A Deep Learning- Image Based Approach for Detecting Cracks in Buildings

A Deep Learning- Image Based Approach for Detecting Cracks in Buildings

Priyadarshini Jayaraju Karthiyaini Somasundaram Adapala Sunny Suprakash Shanmugasundaram Muthusamy

School of Computer Science Engineering, Vellore Institute of Technology, Chennai 600127, India

School of Civil Engineering, Vellore Institute of Technology, Chennai 600127, India

Corresponding Author Email: 
shanmuga.sundaram@vit.ac.in
Page: 
1429-1434
|
DOI: 
https://doi.org/10.18280/ts.390437
Received: 
13 May 2022
|
Revised: 
10 August 2022
|
Accepted: 
20 August 2022
|
Available online: 
31 August 2022
| Citation

© 2022 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).

OPEN ACCESS

Abstract: 

Buildings expand and contract in response to their environment, which results in cracks in the structure. This can pose a serious threat to the people who use it, and these movements are frequently too small to be observed, and thus go unnoticed. Cracks can be caused by a variety of factors, including defects in the construction process, ground movement, foundation failure, and decay of the building fabric. If a structure is unable to accommodate this movement, cracking is likely to occur, posing a serious risk to the building's structural integrity. Only after cracks are identified can they be treated, and existing manual methods of sketching the crack patterns are highly subjective to the person performing the analysis, are frequently constrained by high costs, equipment and tool availability, and are extremely time consuming. In this paper, 40,000 images divided into two and categorized into positive and negative cracks are used as input and the presence of cracks is detected using a deep learning technique. The following crack types are included in the experimentation: hairline, stepped, vertical, and horizontal. In comparison to conventional image processing and other deep learning-based techniques, the proposed Convolutional Neural Network (CNN) achieves significantly higher accuracy than the Recurrent Neural Network (RNN). This paper’s objective is to create a model which can detect the cracks through deep learning methodology, and this will be the innovative region in crack detection using neural net framework.

Keywords: 

concrete surface, CNN, cracks, deep neural nets

1. Introduction

A crack is a line on the surface of something (in this case, a concrete structure) that has split but has not yet broken apart. Cracks erode the building's security, durability, and ultimately result in the building's destruction [1-3]. Hairline, stepped, vertical, and horizontal cracks all have their own unique effect on the structure in question and can occur as a result of a variety of factors such as drought, a weak foundation, an uneven load distribution, ground movement, structure deformation under load, and expansion or contraction of the underlying material. Specifically for buildings, cracks allow dangerous and destructive elements to penetrate the structure, degrading its reliability [4, 5].

Though there are numerous ways to repair cracks once they are found, timely identification of cracks poses a significant challenge. The majority of cracks are identified manually, which requires additional labour in terms of manpower and cost. Human work is never entirely accurate, as humans frequently overlook or misjudge cracks that are necessary for proper building maintenance [6].

In the age of computing, it is extremely crucial to automate the process in order to obtain an accurate analysis of the detection of various cracks, and the literature contains a variety of methods ranging from simple image processing to complex Deep Learnt Models. With accuracy as the criterion, a variety of neural-net models are explored in order to provide an accurate method of identifying the various types of cracks. Damage detection algorithms have been developed using computer vision [7, 8]. Deep Convolutional Neural Networks (DCNN) are one of the most dependable deep learning technologies [9, 10]. The potential causes of cracks like temperature, moisture and other durability behavior in concrete structures are also considered and play a significant role in determining the size of damage detected using CNN [11]. Some researchers have proposed alternative methods, such as RNN, while many deep learning (DL) models, including CNN, use larger datasets to improve their results. Improving the performance of DL classification by enhancing the pertinent characteristics and removing redundant data [12, 13].

Numerous researchers in the field of concrete crack detection classify image patches using Convolutional Neural Networks (CNN) to determine the approximate location of cracks. Certain researchers use the CNN model to regress the crack bounding boxes. Additionally, some researchers view concrete crack detection as a semantic segmentation task, requiring the classification of pixel points in order to determine the location of cracks.

2. Research Significance

This research seeks to develop a model capable of detecting cracks via the deep learning methodology; this will be an innovative area of crack detection utilising the neural network framework. As input, over 40,000 images of cracks (20,000 positive and 20,000 negative) are utilised, and the presence of cracks is detected using a deep learning technique.

3. Literature Survey

The current state of art proposes various analytical methods to solve the problem and below are few standard works driven in the direction using the techniques of image analysis and processing attached with intelligence to solve in an effective way.

External crack identification and examination using image processing techniques are extremely beneficial in supplementing traditional methods of structure examination [14]. The literatures [14-17] demonstrated how image processing techniques were used to detect and evaluate cracks in buildings using pictorial representations. Restoration of buildings can be accomplished using these data and a greater understanding of the cracks [18]. When compared to other image processing methods, image binarization [19, 20] is the most appropriate method for crack identification due to the unique lines and curves found in cracks [5, 21, 22]. The Otsu algorithm is the most frequently used method because it takes image quality, background surface characteristics, and associated parameters into account [14, 23]. Numerous factors affect real-world images, including low contrast at the time the image is taken, changing illumination, noises, and wall imperfections [15]. As a result, it is necessary to improve or modify the standard Otsu method for image binarization [24].

Additionally, there are many other techniques that utilise ultrasounds [25-27], X-rays [28], and Eddy Current (EC) [29]. These papers reconstructed images into 3D (three-dimensional) space [30, 31]. These methods were effective only when the buildings were directly in contact with the concrete surface.

Cai et al. [32] used images captured with an image acquisition system at high magnification, a 2D (two-dimensional) electric cradle, and a laser ranging device that worked in unison to mark the cracks in its observing coordinate system, which was further mapped to observed coordinates, allowing for the spatial location of measured cracks to be determined regardless of device positioning. It was reported that system worked within an average of approximately 16 seconds, with a maximum deviation of 0.07° of crack located. This is extremely convenient if the devices and tools are readily available; however, the test time of 16s was on the longer side.

Liu et al. [33] used multi-scale enhancement and developed visual features to detect cracks. To overcome the limitations of low contrast, a multi-scale enhancement method based on guided filter and gradient information was used first, followed by adaptive thresholding algorithms to obtain a binary image. Cracks were then purified using a combination of morphological processing and visual features. It is demonstrated that the experimental results of various images of real concrete surfaces are validated by the developed technique, where the average TPR reached to 94.22 percent.

To address the issue of real-time crack detection on concrete bridge bottoms, an image processing method was proposed by Tong et al. [34]. The crack's width, depth, and morphology were constrained by the image's pixel intensity distribution. To perform the estimation, the image was converted to 16-bit gray scale and then a mathematical relationship between the intensity distribution and the depth and width of the enhanced image was derived [35].

4. Present Proposed Study

The current study proposes to use Machine Learning (ML) algorithms to detect cracks. As ML techniques improve accuracy and deep learning techniques are an even more powerful tool, the proposed method uses deep neural nets to identify cracks. The system is trained on a variety of images to recognise various types of cracks, and in this paper, various types of cracks are considered, including hairline, stepped, horizontal, and vertical cracks. The proposed layer-wise approach to crack detection is illustrated in Figure 1.

Figure 1. Layer wise approach of the crack detection

4.1 Neural networks

Neural networks are utilised for training and capturing the dataset's characteristics. A neural network is a network of neurons that emits a positive or negative signal in response to inputs, weights, and biases. The learning phase is concerned with determining the weights and biases of the neural network, using the architecture of the network as a hyper-parameter. A neural network consists of a collection of neurons interconnected in a variety of ways to create a miniature model of the brain with an input layer, several hidden intermediate layers, and an output layer where the decision is given as a result.

The neural network consists of a loss function to measure how well a prediction model performs in terms of its ability to predict the expected outcome, an error function to correct its mistake, and a backpropagation algorithm to propagate the error measured.

Neural networks are notoriously known for their capabilities of overfitting and in order to overcome the issue, we have designed system architecture. The considered dataset [36] is expanded from given set, by applying standard image transformations like scaling, skewing, rotations in order to avoid overfitting.

4.2 Convolutional Neural Network

A convolutional neural network is specifically designed to work with images to extract their features. Convolutional neural networks leverage the fact that the input consists of images and constrain the architecture in a more intelligent manner. In particular, unlike a typical Neural Network, the layers of a ConvNet are composed of neurons arranged in three dimensions: width, height, and depth (color dimension) [37].

As depicted in Figure 2, the neural network, given an n-dimensional image (three dimensions in the case of an RGB image), attempts to extract a few features, sending them to the next level while retaining the maximum amount of information in minimal form, before flattening everything out in the final level and applying the final transformation to obtain the final decision output.

Figure 2. CNN architecture

In the design of the proposed system, two distinct activation functions are employed: relu for the hidden layer and sigmoid for the output layer. Binary cross-entropy is used as the loss function, and adam is employed as the optimizer. The conventional back propagation algorithm with a single hidden layer is used to learn the dataset.

4.3 Recurrent Neural Network

A recurrent neural network (RNN) is a type of neural network that displays dynamic temporal behaviour for a time sequence. In contrast to feedforward neural networks, RNNs can utilise their internal state (memory) to process sequences of inputs, resulting in a more accurate approximation of the neuron's understanding of the information it is meant to process. It is useful for handwriting and speech recognition. The RNN principle is utilised in crack detection [38].

Figure 3. RNN architecture

The representation of the recurrent neural network is shown in Figure 3. Each node at a given time step receives an input from the preceding node, which can be represented by a feedback loop. At each time step, an input x_i and a_i-1 (the output of the previous node) are retrieved and processed. The result h_i is generated. This output is collected and passed to the following node. This procedure is repeated until all time steps have been evaluated.

Let at represent the output from previous node

$a_t=f\left(h_{t-1}, x_t\right)$

$g(x)=\tanh x$

$a_t=g\left(W_{h h} \cdot h_{t-1}+W_{x h} \cdot x_t\right)$

$a_t=\tanh W_{h h} \cdot h_{t-1}+W_{x h} \cdot x_t$

$h_t=W_{h y} \cdot a_t$

In recurrent neural networks, backpropagation occurs in the opposite direction of the arrows depicted in Figure 3. As with other back propagation methods, a loss function is evaluated and gradients are obtained to update the weight parameters. The fact that backpropagation in RNNs occurs from right to left is an intriguing aspect. This is known as backpropagation through time because the parameters are updated from the final time steps to the initial time steps.

Two distinct activation functions, relu for the hidden layer and sigmoid for the output layer, are used in the design of the proposed system. Mean-squared is substituted for the error function, and Adam is used as the optimizer. To learn the dataset, the standard back propagation algorithm with a single long term short memory layer and a single hidden layer is applied.

5. About the Dataset

The data set includes images of concrete with cracks. Various METU Campus Buildings serve as collection points for the data. The dataset is separated into negative and positive crack images for classification purposes. Each class contains 20,000 images, for a total of 40,000 images with 227 x 227-pixel dimensions and RGB channels. The dataset is comprised of 458 high-resolution (4032x3024 pixel) images generated using the method proposed by Zhang et al. In terms of surface finish and lighting conditions, high-resolution images exhibit variation. No data enhancement in the form of random rotation or flipping is applied [39]. The size of the data set is maintained with higher image counts and high resolution data set, so that any noise shall be avoided, also, to get more precise model through deep convolutional neural net, such high grade data set are mandatory.

6. Performance Analysis

CNN is the only type of neural network that provides extremely high prediction levels and requires very little computational power, whereas RNN is incapable of achieving high prediction levels when different neural network techniques are applied to achieve the best possible prediction level. Consequently, if the results are observed carefully, RNN attempts to overfit the entire dataset and fails miserably by achieving test accuracy of only 50 percent, whereas CNN achieves test and train accuracies that are equivalent. For the dataset, it can be inferred that CNN has an accuracy of 400 misclassified images out of 40,000, of which 350 are from the training side and 50 are from the testing side [39-42]. Table 1 depicts the performance of the proposed work in training, testing, and prediction.

Table 1. Performance analysis of CNN and RNN

Accuracy

CNN

RNN

Training dataset

99%

100%

Testing dataset

99%

50%

Prediction results

99%

45%

The following Figures 4 shows the results of various types of cracks namely hairline, Stepped, horizontal respectively.

Figure 4. Types of cracks and their detected outcomes

7. Comparative Analysis

Figure 5 compares the outcomes of RNN and CNN. The results indicate that the training sets of both RNN and CNN perform identically, whereas there is a significant difference between the Test Sets. CNN has a 100 percent success rate on the Test set, whereas RNN has only a 45 percent success rate; therefore, CNN influences the RNN model and lowers its performance level. The comparative analysis reveals that CNN is the preferred model for this research. Finally, the prediction results demonstrate that CNN is twice as effective as RNN, making CNN the successful crack detection model identified by this study.

Figure 5. Comparative results of RNN and CNN

8. Conclusion

This study focused on and implemented a model based on computerised machine learning techniques for detecting surface cracks in concrete. Taking into account the aforementioned results, the performed analysis of experiments demonstrated that the use of smaller datasets from conventional neural networks was appropriate. However, the developed trained model was limited to binary classification. From a practical vantage point, AI is an essential path that is identified as computerised review of concrete structures, where the training knowledge reflects the same developed model. In contrast, it demonstrates that human expertise is more necessary to develop and select the appropriate examination tool. As a result, models of machine learning tools are likely to be initially deployed to aid human experts in conducting productive examinations in a secure and expedient manner. Consequently, this facilitates the creation of new opportunities for enhanced infrastructure asset management by enabling periodic damage assessment and structural monitoring.

This study concludes that for structural crack detection through a limited source of available images, the convolution neural net model performs well and detects the crack with 99 percent accuracy, whereas the recurrent neural net model detects the cracks with only 50 percent accuracy. The study can further be extended into its application form, on autonomous crack detection system, through this accuracy. Also, as the high accuracy has been achieved in this study, there lies further opportunities in using this model in drones for supervised structural monitoring system.

  References

[1] Wu, X., Jiang, Y., Masaya, K., Taniguchi, T., Yamato, T. (2017). Study on the correlation of vibration properties and crack index in the health assessment of tunnel lining. Shock and Vibration, 2017: 1-9. https://doi.org/10.1155/2017/5497457

[2] Hoang, N.D., Nguyen, Q.L. (2018). Metaheuristic optimized edge detection for recognition of concrete wall cracks: A comparative study on the performances of Roberts, Prewitt, Canny, and Sobel algorithms. Advances in Civil Engineering, 2018: 1-16. https://doi.org/10.1155/2018/7163580

[3] Talab, A.M.A., Huang, Z., Xi, F., Liu, H. (2016). Detection crack in image using Otsu method and multiple filtering in image processing techniques. Optik, 127(3): 1030-1033. https://doi.org/10.1016/j.ijleo.2015.09.147

[4] Adhikari, R.S., Moselhi, O., Bagchi, A. (2014). Image-based retrieval of concrete crack properties for bridge inspection. Automation in Construction, 39: 180-194. https://doi.org/10.1016/j.autcon.2013.06.011

[5] Hoang, N.D. (2018). Detection of surface crack in building structures using image processing technique with an improved Otsu method for image thresholding. Advances in Civil Engineering, 2018: 1-10. https://doi.org/10.1155/2018/3924120

[6] Hoang, N.D. (2018). Image processing-based recognition of wall defects using machine learning approaches and steerable filters. Computational Intelligence and Neuroscience, 2018: 1-18. https://doi.org/10.1155/2018/7913952

[7] Liu, X., Ai, Y., Scherer, S. (2017). Robust image-based crack detection in concrete structure using multi-scale enhancement and visual features. In Proceedings of the IEEE International Conference on Image Processing, Beijing, China, pp. 2304-2308. https://doi.org/10.1109/ICIP.2017.8296693

[8] Dorafshan, S., Thomas, R.J., Maguire, M. (2019) Benchmarking image processing algorithms for unmanned aerial system-assisted crack detection in concrete structures. Infrastructures, 4(2): 19. https://doi.org/10.3390/infrastructures4020019

[9] Cha, Y., Choi, W., Büyüköztürk, O. (2017). Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil Infrastructure Engineering, 32(5): 361-378. https://doi.org/10.1111/mice.12263 

[10] Dorafshan, S., Thomas, R., Coopmans, C., Maguire, M. (2018). Deep learning neural networks for sUAS-assisted structural inspections: Feasibility and application. In Proceedings of the International Conference on Unmanned Aircraft Systems, pp. 874-882. https://doi.org/10.1109/ICUAS.2018.8453409

[11] Golding, V.P., Munawar, H.S., Ullah, F. (2022). Crack detection in concrete structures using deep learning. Sustainability, 14(13): 1-25. https://doi.org/10.3390/su14138117

[12] Bui, H.M., Lech, M., Cheng, E., Neville, K., Burnett, I.S. (2016). Using grayscale images for object recognition with conventional-recursive neural network. 2016 IEEE Sixth International Conference on Communications and Electronics (ICCE), pp. 321-325. https://doi.org/10.1109/CCE.2016.7562656

[13] Hsieh, Y.A., Tsai, Y.J. (2020). Machine learning for crack detection: Review and model performance comparison. Journal of Computing in Civil Engineering. 34(5). https://doi.org/10.1061/(ASCE)CP.1943-5487.0000918

[14] Kim, H., Ahn, E., Cho, S., Shin, M., Sim, S.H. (2017). Comparative analysis of image binarization methods for crack identification in concrete structures. Cement and Concrete Research, 99: 53-61. https://doi.org/10.1016/j.cemconres.2017.04.018

[15] Zakeri, H., Nejad, F.M., Fahimifar, A. (2016). Image based techniques for crack detection, classification and quantification in asphalt pavement: A review. Archives of Computational Methods in Engineering, 24: 935-977. https://doi.org/10.1007/s11831-016-9194-z 

[16] Mohan, A., Poobal, S. (2017). Crack detection using image processing: A critical review and analysis. Alexandria Engineering Journal, 57(2): 787-798. https://doi.org/10.1016/j.aej.2017.01.020

[17] Koch, C., Georgieva, K., Kasireddy, V., Akinci, B., Fieguth, P. (2015). A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure. Advanced Engineering Informatics, 29(2): 196-210. https://doi.org/10.1016/j.aei.2015.01.008

[18] Rabah, M., Elhattab, A., Fayad, A. (2013). Automatic concrete cracks detection and mapping of terrestrial laser scan data. NRIAG Journal of Astronomy and Geophysics, 2(2): 250-255. https://doi.org/10.1016/j.nrjag.2013.12.002

[19] Chaki, N., Shaikh, S.H., Saeed, K. (2014). Applications of Binarization. In: Exploring Image Binarization Techniques. Springer India, New Delhi, pp. 65-70. https://doi.org/10.1007/978-81-322-1907-1_5

[20] Gonzalez, R.C., Woods, R.E., Eddins, S.L. (2004). Digital Image Processing Using MATLAB. Pearson Prentice-Hall, Upper Saddle River, New Jersey. 

[21] Hoang, N.D., Nguyen, Q.L. (2018). A novel method for asphalt pavement crack classification based on image processing and machine learning. Engineering with Computers, 35: 487-498. https://doi.org/10.1007/s00366-018-0611-9 

[22] Hoang. N.D., Nguyen, Q.L., Bui, D.T. (2018). Image processing-based classification of asphalt pavement cracks using support vector machine optimized by artificial bee colony. Journal of Computing in Civil Engineering, 32(5). https://doi.org/10.1061/(ASCE)CP.1943-5487.0000781

[23] Otsu, N. (1979). A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetic, 9(1): 62-66. https://doi.org/10.1109/TSMC.1979.4310076

[24] Hoang, N.D., Trần, D.K. (2019) Image processing based building crack detection using Surface Crack Analysis program. DTU Journal of Sciences and Technology, 33(2): 17-22. 

[25] Jeune, L.L., Robert, S., Villaverde, E.L., Claire, P. (2016). Plane wave imaging for ultrasonic non-destructive testing: Generalization to multimodal imaging. Ultrasonics, 64: 128-138.

[26] Ultrasonic Concrete Testing Equipment. https://www.pcte.com.au/pundit-pl-200-ultrasonic-pulse-velocity, accessed on 13 August 2022. 

[27] Ultrasonic Testing Product, https://www.canopusinstruments.com, accessed on 13 August 2022.

[28] Eddine, H.S., Pitti, R.M., Badel, E., Joseph, G. (2018). Reconstruction of the 3D crack profile in wood based structures by X-ray computed microtomography. In Proceedings of the JET 2018, Marrakech, Morocco. https://hal.archives-ouvertes.fr/hal-01731148 

[29] Cai, C., Bore, T., Delaine, F., Gasnier, N., Vourc’h, E. (2017). 3D reconstruction of surface cracks using bi-frequency eddy current images and a direct semi-analytic model. 7th International Conference on New Computational Methods for Inverse Problems. Journal of Physics: Conference Series, 904: 012019. https://doi.org/10.1088/1742-6596/904/1/012019 

[30] Liu, Y.F., Cho, S., Jr. Spencer, B.F., Fan, J.S. (2016). Concrete crack assessment using digital image processing and 3D scene reconstruction. Journal of Computing in Civil Engineering, 30(1). https://doi.org/10.1061/(ASCE)CP.1943-5487.0000446 

[31] Ma, Z., Liu, S. (2018). A review of 3D reconstruction techniques in civil engineering and their applications. Advanced Engineering Informatics, 37: 163-174. https://doi.org/10.1016/j.aei.2018.05.005

[32] Cai, Y., Fu, X., Shnag, Y., Shi, J. (2018). Methods for long-distance crack location and detection of concrete bridge structures. IEEE 3rd International Conference on Image, Vision and Computing (ICIVC), pp. 576-580. https://doi.org/10.1109/ICIVC.2018.8492764

[33] Liu, X., Ai, Y., Sebastian S. (2017). Robust image-based crack detection in concrete structure using multi-scale enhancement and visual features. IEEE International Conference on Image Processing (ICIP), pp. 2304-2308. https://doi.org/10.1109/ICIP.2017.8296693

[34] Tong, X., Guo, J., Ling, Y., Yin, Z. (2011). A new image-based method for concrete bridge bottom crack detection. IEEE International Conference on Image Analysis and Signal Processing, pp. 568-571. https://doi.org/10.1109/IASP.2011.6109108 

[35] Vashpanov, Y., Son, J.Y., Heo, G., Podousova, T., Kim Y.S. (2019). Determination of geometric parameters of cracks in concrete by image processing. Advances in Civil Engineering, 2019: 1-14. https://doi.org/10.1155/2019/2398124

[36] Zhang, L., Yang, F., Zhang, Y.D., Zhang, Y.J.Z., Zhu, Y.J. (2016) Road crack detection using deep convolutional neural network. In 2016 IEEE International Conference on Image Processing (ICIP), pp. 3708-3712. http://doi.org/10.1109/ICIP.2016.7533052 

[37] Cha, Y.J., Choi, W. (2017). Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering, 32(5): 361-378. https://doi.org/10.1111/mice.12263 

[38] da Silva, W.R.L., de Lucena, D.S. (2018). Concrete cracks detection based on deep learning image classification. Proceedings, 2(8): 489. https://doi.org/10.3390/ICEM18-05387

[39] AAOSHAT. (2008). Bridging the Gap—Restoring and Rebuilding the Nation’s Bridges, American Association of State Highway and Transportation Officials, Washington DC.

[40] Abdel-Qader, I., Abudayyeh, O., Kelly, M.E. (2003). Analysis of edge-detection techniques for crack identification in bridges. Journal of Computing in Civil Engineering, 17(4). https://doi.org/10.1061/(ASCE)0887-3801(2003)17:4(255)

[41] Ren, Y., Huang, J., Hong, Z., Lu, W., Yin, J., Zou, L., Shen, X. (2019). Image-based concrete crack detection in tunnels using deep fully convolutional networks. Construction and Building Materials, 234. https://doi.org/10.1016/j.conbuildmat.2019.117367

[42] Xu, Y.P. (2022). A deep learning-based cluster analysis method for large-scale multi-label images. Traitement du Signal, 39(3): 931-937. https://doi.org/10.18280/ts.390319