Enhanced Identification of Internal Casting Defects in Vehicle Wheels Using YOLO Object Detection and X-Ray Inspection

To achieve lighter vehicle weight, reduce energy cost and fuel consumption, aluminum alloy wheels have become the mainstream choice. Automotive aluminum wheel manufacturing technology can be divided into two technologies: “casting” and “forging”. Casting technology can be divided into gravity casting, low-pressure casting, high-pressure casting, etc. according to the production pressure. The vehicle aluminum wheels used in this study were made using the low-pressure casting method [27]. The vehicle aluminum wheel production process is shown in Figure 1 [28]. Before the qualified wheels are sold, they are subjected to relevant safety tests in accordance with regulations. Structural strength tests and external environmental tests are conducted to ensure the wheel does not have structural strength problems and appearance corrosion problems. To ensure that there are no internal organizational problems and to guarantee the wheel structural strength integrity, it is necessary to use X-ray NDT for vehicle aluminum wheel quality inspection.

X-ray non-destructive testing (DR) equipment was used in this study to collect casting defect samples. As shown in Figures 2 and 3, the X-ray inspection process is as follows: X-ray NDT is performed by X-ray tubes with high voltage electricity and a controller to emit X-rays. Then the X-rays penetrate and irradiate the aluminum wheel casting. Different aluminum wheel casting thicknesses were X-rayed and sent to a digital image plate receiver, and image processing was performed real time on an industrial monitor. The image was then converted into a pixel image on the inspection screen. This pixel image shows the quality organization of each part inside the aluminum wheel casting. Finally, the image can be clearly seen by a professional X-ray inspector to directly detect and determine the casting defects. This study also captured three types and samples of casting defects that appear most frequently in the production line for follow-up training and verification. These routine defects are casting holes (ch), cracks (cr), and sponge casting holes (sch). The X-ray inspection technology provides accurate test results without causing any damage to the inspection object.

1.png

Figure 1. Vehicle aluminum wheel production flow chart

2.png

Figure 2. X-ray inspection process and equipment diagram

3.png

Figure 3. X-ray inspection flow chart

The aluminum wheel undergoes X-ray non-destructive inspection. During the image capture process, due to the mutual computer and camera frequency interference, interference ripples will be generated. The shapes include horizontal lines, straight lines and amplitude shapes. These interference ripples are called “Moore fringes” “(Moiré Pattern)”. There are interference removal methods used in the past [29], but this study used adaptive weight morphological filtering. The periodic noise removal algorithm based on the “Composite Cascade Filter” is used for image denoising and optimization [30]. The defect sample image with moiré fringes was selected first, imported into the image sample, and denoised using a composite cascade filter. The results show that after denoising the defect sample image, the moiré fringe can be removed to optimize the image. The YOLO v4 training and testing results show that the defective sample denoised image can significantly help and improve its recognition rate. The defective sample images collected in the future will use this “composite-level series-parallel filter” method for denoising and optimization to achieve the optimized YOLO training and recognition rate. The relevant principles are as follows. The input image is (x) and the cascade filtering result through the structural elements of a certain shape is f_i(x), i=1, 2, ...., n. The output image is F(x). Among them, the structural elements determine the weights through the adaptive algorithm shown in the formula α₁, α_2, ...... α_n, shown as Eq. (1):

$F(x)=\sum_{\mathrm{i}=0}^{\mathrm{n}} \alpha_i f_i(x)$          (1)

When the digital image is denoised using mathematical type filtering, it is possible to use structural elements with dimensions from small to large for processing. Therefore, selecting a series filter for image filtering shape-filters the image by structuring elements of the same shape in ascending order of dimension. This is similar to the series circuit design process, as shown in Figure 4 (a). In addition, the series filters formed by the above structural elements from different shapes can also be connected in parallel and combined with an adaptive weight algorithm to construct a series-parallel composite filter, as shown in Figure 4 (b). The image optimization and denoised process and results are shown in Figure 5.

The neural network is a neural network formed by simulating biological nerve conduction and interconnected by multiple layers of neurons. In the artificial neural network, the neurons in each layer have input and output, and the activation function [31] is when the data is input from the neuron and the output is sent through a linear activation function transmitted to the next neuron up to the final output layer. The activation function allows the neural network to have sufficient ability to capture complex features, thereby improving the model efficiency and the artificial neural network composition. When we input a picture of size 512*512, the input layer of the first layer has 262,144 neurons. The input passes through multiple hidden layers, according to the number of categories to be identified and how many are output in the output layer. Neurons, where the depth of the model is determined by the number of hidden layers. It can be seen from the above that the artificial neural network has two main shortcomings. The first is that it requires a lot of memory. The second is that it is difficult to obtain the characteristics of each part of the object in the image because the input is judged by a single pixel. In the image interpretation, the artificial neural network accuracy is not outstanding.

4.png

Figure 4. Series filter (a) and composite cascade filter (b) flow chart

The convolutional neural network (CNN) is one of the feedforward neural networks, which is often used in the image recognition field. The biggest rolling machine neural network feature is that it can use color, texture, light source, and image size as the neural network input features [32, 33]. In recent years, it has been practically applied to safety detection and epidemic prevention reminders in daily life [34]. When an image is input, the data contains three-dimensional information such as color, horizontal and vertical position. The traditional deep learning network input image is one-dimensional data, in which the traditional artificial neural network easily ignores the image shape during training. The recognition effect is therefore reduced and the convolutional neural network is not like the traditional artificial neural network, which simply extracts data for operation. The convolutional neural network has two more convolutional layers and pooling layers than the traditional artificial neural network. The convolutional neural network includes the first layer-the convolutional layer and the second layer-the pooling layer and a third layer-the fully connected layer (traditional artificial neural network), through feature extraction and system operation output classification results.

5.png

Figure 5. Image denoising optimization process

The convolutional layer function extracts the features in the image. The image features are extracted through a matrix of size N*N. The matrix of size N*N is used to read the input image from top to bottom and produce the inner product. The matrix then calculates and obtains each local feature in the image as the input for the next layer. A 3*3 matrix reads input image sequentially from top to bottom. If the step size is set to 1, the input image features will be obtained. The convolutional layer is characterized by preserving the structure of the photo and extracting features from it.

The pooling layer function reduces the input image size to reduce the feature map of each image and retain the important features. There are three pooling layer advantages. First, it reduces the parameters required for the subsequent layers and speeds up the system efficiency. Second, when some pixels in the image have small offsets or differences in neighboring regions, it has little effect on the pooling layer output. Third, the pooling layer can reduce overfitting occurrence. Like the convolutional layer, the pooling layer also inputs a matrix of size N*N into the image and reads the input image sequentially from top to bottom to extract the values of each area and perform operations. There are three methods used by the pooling layer to reduce the image size, namely maximize, average and random. Taking maximization as an example, a 2*2 matrix is used to set the step size to 2, and read the image from top to bottom and take out the maximum value for each area. In the end, in addition to getting a reduced size image, all of the important feature information is retained.

Object Detection technology is composed of three different algorithms: Object Localization, Feature Extraction, and Image Classification. In the YOLO object detection algorithm series, with the improved version update, the network architecture is also optimized. YOLOv1 was first proposed by Redom et al. in the United States in 2015 [23]. The disadvantage of YOLOv1 is that the recall rate is very low and many small targets are easily missed. In 2016 YOLOv2 was launched. After strengthening, YOLOv2 was able to detect more than 9000 categories, so YOLOv2 can also be called YOLO 9000 [24]. In 2018 YOLOv3 was launched [25]. YOLOv3 features a pyramid network that improves small object detection ability, and the output uses NMS to select the most suitable one among multiple bounding box results as the output. YOLOv4 was jointly developed in 2020 by Liao Hongyuan Distinguished Researcher, Postdoctoral Researcher Wang Jianyao and Russian developer A. Bochkovskiy of the Institute of Information Science, Taiwan Academia Sinica [35]. The YOLOv4 network architecture has a skeleton divided into four parts: CSP Darknet-53, SPP, PANet, and Detection Head. CSPDarknet-53 consists of two architectures, Darknet-53 and Cross-Stage Partial Network (CSPNet), which are used as the base network architecture. This study uses YOLOv5. The network architecture diagram is shown in Figure 6. YOLOv5 is the latest YOLO detection method, proposed by Nepal et al. [36, 37]. The YOLOv5 uses CSPdarknet53 as the backbone and the Focus structure, which consists of modules such as Conv, C3, and Spatial Pyramid Pooling (SPP). The difference between YOLOv5 and previous versions is that it uses PyTorch instead of the YOLOv4 and YOLOv3 Darknet. The neck network also uses Feature Pyramid Network (FPN) and Pixel Aggregation Network (PAN) [36, 37]. In the YOLOv5 model architecture there are four different parameter structures, namely s, m, l, and x, which represent the sizes Small, Medium, Large, and Extra-large. Each of its model architectures has different parameters applied to control the depth and width of the network [38]. As shown in Figure 7, the Focus layer was first introduced in YOLOv5, and the Focus layer replaced the first three layers in the YOLOv3 algorithm. The advantage of using the Focus layer is that it reduces the required CUDA memory, reduces the number of layers and increases forward propagation and backpropagation. The focus module divides the input data into four parts, each of which is equivalent to two down samplings [39]. Furthermore, the head in YOLOv5 is the same as in YOLOv4 and YOLOv3, generating three different feature map outputs for multi-scale prediction. It also helps to effectively enhance predictions for small to large objects in the model.

6.png

Figure 6. Network architecture diagram of YOLOv5

7.png

Figure 7. Focus slice operation diagram

YOLOv4 and YOLOv5 used four output coordinates in the grid unit which are t_x, t_y, t_w, and t_h of the frame. Eqs. (2) to (5) calculate the offset value of the corresponding grid from the upper left corner of the bounding box relative to the center point, C_x and C_y are relative to the upper left corner. When t_x and t_y are calculated, the offset between 0 and 1 will be output. After adding the output offset to C_x and C_y, the center point border position will be obtained. The p_w and p_h of the dashed box are called the width and height of the preset box. After calculating with t_w, t_h, p_w, and p_h, the predicted width and height will be called b_w and b_h.

$b_x=\sigma\left(t_x\right)+c_x$          (2)

$b_x=\sigma\left(t_x\right)+c_x$         (3)

$b_w=p_w e^{t_w}$         (4)

$b_h=p_h e^{t_h}$         (5)

Intersection over union (IoU) is a criterion for detection accuracy. Many object detections use this standard as a reference for accuracy. In order for IoU to detect objects of any size and shape, a ground-truth bounding box and a predicted bounding box are required. In the IoU calculation method the solid line is the real bounding box and the dashed line is the predicted bounding box. The real bounding box and predicted bounding box intersection are divided by the real bounding box and predicted bounding box union, called IoU. When the IoU value is close to 1, it means that the predicted situation conforms more to a real situation.

The confusion matrix method is used as an important quality benchmark for evaluating model training and flaw identification. This method is called a binary confusion matrix, which contains four elements: TP (True Positive), TN (True Negative), FP (False Positive), FN (False Negative). First, TP is the system correctly predicting a successful positive sample. For example, in an image classifier that predicts whether it is a casting defect ch, the image label that successfully classifies the casting defect ch as ch is called TP. Second, TN is the negative sample for which the system correctly predicts success. In the example above, a picture of casting flaw ch is successfully marked as not ch, called TN. Third, FP is when the system misjudges that it is a positive sample, but it is actually a negative sample. For example, a picture with a casting defect ch is incorrectly labeled and classified as a casting defect cr. Ultimately, FN is a negative sample in which the system misjudges (or a positive sample that the system cannot predict), e.g., the system incorrectly marks a picture with a casting defect ch as not a casting defect ch. According to the above explanation, the system model training and defect identification results are better if the TP and TN values are large. The Confusion matrix is shown in Table 1. If you want to look at these values directly, it is difficult to see at a glance whether a defect classification model is good or bad. This study also used recall (RR), precision (PR), F1-score and mAP to evaluate the model quality. After the program calculation, TP, TN, FP, FN can get a variety of different indicators. PR(Precision) represents the ratio of the number of correct recognizers detected by the system to the number of all recognizers detected by the system. RR(Recall) is the ratio of the number of correct recognizers detected by the system to the actual number of recognizers. The F1-score is the Average of weighted and reconciled PR and RR, mAP is calculated by taking the average precision (AP) of all classes. The formulas for PR, RR, F1-score and mAP calculation methods are shown in Eqs. (6)-(9).

$P R(Precision)=\frac{T P}{T P+F P}$            (6)

$R R(Recall)=\frac{T P}{T P+F N}$            (7)

$F 1 score =\frac{2 \cdot P R \cdot R R}{P R+R R}$            (8)

$m A P=\sum_{q=1}^Q \frac{A v e P(q)}{Q}$            (9)

Table 1. Illustration of confusion matrix

Through the X-RAY non-destructive inspection equipment that to inspect the internal casting defects of the vehicle's aluminum wheel is carried out, and the image is imaged on the X-RAY inspection display screen. Three kinds of aluminum wheel internal casting defects are captured, as shown in Figure 11. After optimization through adaptive image de-noise image processing, the open-source frame selection program labelimage is used to draw a rectangular or square frame around the internal casting defects in the image for bad defects. At the same time the casting defects are framed, the labelimage program will output it as a text file (TXT.), which contains the casting defect type name, the center coordinates of the object frame selection, and the length and width of the object frame selection, as shown in Figure 12. A total of 5,700 photo samples were used in this study to create a database of aluminum wheel internal casting defects. Each type has 1,900 internal casting defect photo samples for YOLO training. After the training an additional 100 photo samples of each type will be used, for a total of 300 casting defect photo samples externally tested.

11.png

Figure 11. Internal defect images of three castings

12.png

Figure12. Frame the casting defect with labelimage

The casting defect samples were tested in two different ways. The first is the most original image was taken by X-RAY and inspected. The second was a periodic noise removal image processing method based on weighted adaptive morphology combined with a composite cascade filter. Comparing the two methods, the aperiodic noise removal image processing method based on weighted adaptive morphology produces a composite cascade filter relatively clear to the human eye. The YOLO deep learning network is different from the visual eye. Therefore, the above two methods were used to identify the internal casting defects from three different vehicle aluminum wheels. The results are shown in Table 4. The PR and mAP testing result values for the casting defect identification rates were determined. The image processing method combined with a periodic noise removal image processing method based on weighted adaptive morphology and a composite cascade filter used in this research was better than the original image.

Table 4. Accuracy PR mAP of different filter on YOLOv4 model

Table 5. Accuracy(ch) of different sample sizes on different YOLO models

Table 6. Accuracy(cr) of different sample sizes on different YOLO models

Table 7. Accuracy(sch) of different sample sizes on different YOLO models

The number of iterations was set to classes*2,000 as suggested by the original author of YOLO v4.0. This was the optimal number of iterations. In order to obtain the optimized parameter conditions and the most appropriate training time for YOLO training, this study used two training parameter conditions. The first parameter is that the number of iterations is from 1,000 to 30,000 times. The second is the training angle from 0° to 330°. The YOLOv4 tiny model compares the test result value for PR, the average mAP, and IoU. Considering the final training time and identification results, the overall result is better when the training parameters are overlapped 6000 times and the rotation angle is 0°. The training time is also shorter. In this analysis the optimized parameter conditions and the most appropriate training time for YOLO training were obtained as a follow-up study.

In order to find the correlation between the number of samples and the accuracy rate, this study conducted a test analysis of each number of samples. The number of photo samples was from 150 photo samples to 1900 photo samples, and the number of samples was incremented using 250 photo samples each time. Six different model architectures were used for training and identification, respectively the YOLOv3, YOLOv3-tiny, YOLOv4, YOLOv4-tiny, YOLOv5s and YOLOv5m model architectures. As shown in Tables 5 to 8, according to the final results, when adding the samples and the PR and mAP accuracy rates for related defect types will also crease accordingly for each YOLO module. The recognition performance comparison for each YOLO module can be seen in the mAP values for the final samples for 1900p, as shown in Table 9. The YOLOv3 and YOLOv4 are modules with more convolution layers. The recognition rate performance is good with the mAP value respectively 90.2% and 96.8%, but the disadvantage is that it takes a lot of time to perform research training. YOLOv3-tiny and YOLOv4-tiny are lightweight modules of YOLOv3 and YOLOv4, which use fewer convolutional layers for training. The purpose is to reduce the research and training time, and the relative recognition rate can be maintained. However, due to the use of fewer convolutional layers, the YOLO system can intercept fewer features, it can be seen from the results that the recognition rate is relatively low, with the mAP value respectively 66.7% and 89.7%. The YOLOv5s and YOLOv5m modules combine the advantages of YOLOv4-tiny and YOLOv4. They can not only shorten the system training time like YOLOv4-tiny but also maintain a good recognition rate like YOLOv4. Therefore, the mAP values for YOLOv5s and YOLOv5m respectively were 97.5% and 97.6%, as shown in Figures 13 to 14. Finally, in this study, after the results were compared, the YOLOv5 module has a very good recognition rate and better than the others.

In addition, in order to find the correlation between the number of training and the accuracy rate, test and analyze each training number, the YOLO module selected this time was the latest YOLOv5s and YOLOv5m. The number of training (Epoch) is from 1 to 40, depending on the training times incremented every 5 times. A total of 9 training parameters were used for training and identification. As shown in Tables 10 and 11, according to the final results, when the number of training times is added, the accuracy rates PR and mAP of related defect types will also crease accordingly. According to the final average mAP value, the result from one training session is the worst, the PR range of casting defect recognition rate is 14.9% to 44.9%, and the average mAP value range is 31.3% to 58.8%. Taking 40 training times as the best result, the PR range of each casting defect recognition rate is 93.1% to 99.8%, and the average mAP value range is 97.9% to 98.1%. At the same time, it was also found that when the number of training times was more than 5 times, whether it is YOLOv5s or YOLOv5m module, the PR of casting defect precision rate can reach 88.0% to 99.7%, and the average mAP value can reach 95.2% to 95.4%. The research data results show that the YOLOv5 system can achieve a good precision rate even when the number of training times for casting defects is small, which can be a reference for shortening the research training time.

External casting defect identification system, combined with optimized YOLO weight and Raspberry Pi 4 to run the execute identification, and the identification test results can successfully identify and classify each casting defect type, then according to the inspection result can inform the production line to improve the casting defects and to achieve stable production, as shown in Figure 15. The stability and precision of the identification system are extremely important, which will be one of the indicators of whether it can be imported into production. This study is based on the five core tools of IATF 16949, among which are the Reproducibility and Repeatability of MSA (Measurement Systems Analysis)-confirming the stability and accuracy of the identification system.

Table 8. Accuracy(mAP) of different sample sizes on different YOLO models

Table 9. Accuracy PR and mAP on different YOLO models Total samples: 2000p per each defects (Training:1900p/Test: 100p)

Table 10. Accuracy PR and mAP of different epoch times on YOLOv5s model

13a.png

Figure 13. The mAP@0.5 chart of three casting defects for YOLOv5s

14.png

Figure 14. The mAP@0.5 chart of three casting defects for YOLOv5m

Table 11. Accuracy PR and mAP of different epoch times on YOLOv5m model

Reproducibility: Under different conditions, such as different inspectors, the same program settings, photos of inspected flaws, and environmental conditions, repeat the measurement of the same object under test to ensure the accuracy of the system. In the external experimental test, this study selected casting defect photos, 10 samples of different types of defective products (with defects), 5 photos of different good samples (without defects), and photos of each casting defect A total of 15 samples, 45 samples in total, and was tested by 3 different inspectors for the precision of the system. The judgment is based on whether the system can correctly distinguish defective products from good products, and clearly distinguish and identify them. The test results show that the system can successfully identify and inspect various types of NG and OK products 100%. The system can successfully complete the identification when testing is performed by three different testers, which means that in addition to the identification system accuracy, the reliability degree is OK and the system program will not produce abnormal recognition due to different variations, which will indirectly affect the identification result.

Repeatability: Under the same conditions, such as the same inspector, in the same program setting, defect photos and environmental conditions, repeatable measurement of the same object was conducted to ensure system stability. In the external experimental test, this study selected casting defect photos, 10 photo samples of the same defect, and a total of 30 samples to conduct a systematic stability test. The judgment basis is based on whether the system can clearly and correctly identify and detect the same defective product location, and will not produce identification variation. The test results show that the system can successfully identify and detect defective products and defective positions 100%, and there was no abnormal position identification, which indicates the stability and reliability of the identification system. In the beginning of this research, we focused on the source quality of the training samples, and after multiple tests and verifications, defined the correct parameters of the training samples that managed and implemented them accordingly. Rigorous research and training process can finally achieve optimal and credible research identification results.

15.png

Figure 15. The results of the three casting defects detection