Pavement damage is the main factor affecting road performance. Pavement cracking, a common type of road damage, is a key challenge in road maintenance. In order to achieve an accurate crack classification, segmentation, and geometric parameter calculation, this paper proposes a method based on a deep convolutional neural network fusion model for pavement crack identification, which combines the advantages of the multitarget single-shot multibox detector (SSD) convolutional neural network model and the U-Net model. First, the crack classification and detection model is applied to classify the cracks and obtain the detection confidence. Next, the crack segmentation network is applied to accurately segment the pavement cracks. By improving the feature extraction structure and optimizing the hyperparameters of the model, pavement crack classification and segmentation accuracy were improved. Finally, the length and width (for linear cracks) and the area (for alligator cracks) are calculated according to the segmentation results. Test results show that the recognition accuracy of the pavement crack identification method for transverse, longitudinal, and alligator cracks is 86.8%, 87.6%, and 85.5%, respectively. It is demonstrated that the proposed method can provide the category information for pavement cracks as well as the accurate positioning and geometric parameter information, which can be used directly for evaluating the pavement condition.
Pavement distress is the main factor affecting road performance. Timely and accurate detection of pavement damages is a crucial step in pavement maintenance. Cracks are the initial manifestation of various types of pavement diseases. Pavement cracks will not only affect pavement appearance and driving comfort but also can easily expand to cause pavement structural damage and shorten the overall service performance and life of the pavement [1, 2]. Therefore, early crack detection and timely maintenance of the cracked pavement can reduce the economic cost of pavement repairing and ensure the safety of vehicles and drivers transiting on the pavement.
Crack Fusion Connect 2008 Crack
In 2016, Zhang et al. [18] proposed a crack detection method based on deep learning. They trained a deep CNN based on supervised learning, proving the feasibility of combining deep learning with pavement crack recognition. In 2017, Zhao et al. [19] proposed a pavement crack detection method based on a CNN using images of different scales and taken at different angles for training, achieving the detection of cracks of various shapes. However, owing to road surface interference and noise, the detection accuracy of this system peaked at 82.5%. In 2017, Markus et al. developed the open dataset GAPs for the training of deep neural network and evaluated the pavement damage detection technology for the first time, which is of great significance [20, 21]. In 2018, Nhat-Duc et al. [22] established an intelligent method for the automatic recognition of pavement crack morphology; this study constructs a machine learning model for pavement crack classification that included multiple support vector machines and an artificial swarm optimization algorithm. Using feature analysis, a set of features is extracted from the image projection integral, which can significantly improve the prediction performance. However, the algorithm is complex and programming it becomes very difficult. In 2020, Zhaoyun Sun et al. [23] proposed a method to detect pavement expansion cracks with the improved Faster R-CNN, which can achieve accurate expansion crack location detection through the optimization model. The aforementioned studies only detect and classify pavement cracks and their location but cannot quantify certain crack characteristics, such as crack width and area. On the other hand, there are also many studies on crack segmentation. In 2018, Zhang and Wang [24] proposed CrackNet, which is an efficient architecture based on CNN to predict the class of each image pixel, but its network structure is related to input image size, which prevents the generalization of the method. In the same year, Sen Wang et al. [25] proposed to use the full convolutional networks (FCNs) to detect cracks and built the Crack-FCN model taking into account the shortcomings of the FCN model in the crack segmentation experiment and obtained a complete crack image. However, the highest accuracy obtained by their method is only 67.95%; thus, segmentation performance needs to be improved. In 2019, Piao Weng et al. [26] proposed a pavement crack segmentation method based on the VGG-U-Net model. It solves the problem of fracture in the crack segmentation result in complex background, but its training time is slightly longer and its efficiency is low. In 2020, Zhun Fan et al. [27] proposed an encoder-decoder architecture based on hierarchical feature learning and dilated convolution (U-HDN) detects cracks in an end-to-end manner. The U-HDN method can extract and fuse different context sizes and different levels of feature mapping, so it has high performance. In the same year, Zhun Fan et al. [28] proposed an ensemble of convolutional neural network based on probability fusion for automatic detection and measurement of pavement cracks, and the predicted crack morphology is measured by skeleton extraction algorithm. In summary, these previous studies only use the segmentation method, which cannot achieve accurate crack classification and location determination.
Given the abovementioned problems in pavement crack identification, this paper proposes a method based on a deep convolutional neural network fusion model for pavement crack identification, which is applicable in many crack detection cases (including detector vehicle and smartphone). By training on a learning image data having a variety of sources and sizes, the method can effectively identify cracks, and recognition accuracy can be guaranteed. At the same time, a detected crack can be segmented, and the segmented binary image can be used to calculate the geometric parameters of the crack. Therefore, the proposed model is of great significance for intelligent pavement detection and it can also achieve detection and segmentation simultaneously, thereby significantly improving model efficiency.
In this paper, a crack identification method based on a deep CNN fusion model is proposed. First, the image dataset is established, and the image noise in the dataset is filtered out to increase the contrast between road cracks and background. Next, the processed images are provided as input into an improved single shot multibox detector (SSD) crack detection model and an improved U-Net crack segmentation model for training. Then, the binary image of a crack obtained by the segmentation model is used to calculate the geometric parameters of the crack. By integrating the advantages of the two models, this pavement crack identification method can effectively overcome the single-model limitations of inaccurate positioning and imperfect information. The overall process flowchart is shown in Figure 1. The details of each step are discussed in Section 2.1.
In order to be applicable to crack detection in a variety of scenarios, the proposed method uses a detector vehicle (Figure 2(a)) and a smartphone (Figure 2(b)) to collect crack images. The pixel of the image captured by the detector vehicle is 1024 960, and the pixel of the image captured by the smartphone is 2560 1024.
The pavement crack images are preprocessed before network training to reduce the noise in the images and improve the prediction accuracy. Preprocessing consists mainly of augmenting and denoising the pavement crack images.
First, the number of images needs to be increased. As it is difficult to distinguish the effects of rotation by using an actual crack image, a black-and-white double-arrow picture is used here to exemplify how to increase the number of images (Figure 3). By horizontal reflection, vertical reflection, and clockwise rotation of the image by 45, 90, and 180, the training image dataset can be expanded eightfold.
The images, as taken by the camera, are seriously affected by discrete pulse noise and zero-mean Gaussian noise. Therefore, median and bilateral filters are used to denoise the road images. Then, contrast enhancement is performed to increase the difference between crack information and road background, which improves the quality of the sample images. As seen in Figure 4, the processed crack information is more prominent than in the original image.
Increasing the number of network layers can improve the accuracy of the network in identifying pavement cracks. Therefore, in this study, the feature extraction network structure Visual Geometry Group 16 (VGG16) in the SSD network model was replaced with a deep residual network to improve the pavement crack identification accuracy. The deep residual network [33, 34] solves this problem by fitting a residual map instead of the original map and by adding multiple connections between layers.
The structure of the improved crack classification and detection network is shown in Figure 6. The convolutional layer of different feature map sizes contains two kinds of convolution kernels. One is used for position regression of the prediction box, and the other is used for crack classification.
The reason behind improving the model is to improve crack detection accuracy and to generate a better detection box to surround the cracks. In order to ascertain whether the model prediction results were improved after improving the crack detection model, model results before and after the improvement were compared using the test set (Table 2).
Three types of cracks were randomly selected for testing, and the test results are shown in Figure 9. By comparing the prediction results of the model before and after the improvement for the same category and the same picture, it can be seen that the improved crack classification and detection model provides a higher degree of confidence for identifying the crack category in the pavement image, and the prediction results are more accurate, which demonstrates the effectiveness of model improvement and optimization. 2ff7e9595c
Comments