Integration of YOLOv9 and Contrast Limited Adaptive Histogram Equalization for Nighttime Traffic Sign Detection

Single Shot Detection (SSD) is an object detection methodology that identifies many objects inside a single image or video processing instance. This technique differs from previous methods that necessitate multi-stage picture processing. SSD employs a Convolutional Neural Network (CNN) to extract characteristics from images, subsequently utilizing several convolutions to generate a collection of bounding boxes accompanied by confidence values for various item classes.

Traditional nighttime object recognition methods, like Histogram of Oriented Gradient (HOG) and Support Vector Machine (SVM), exhibit computational efficiency but lack resilience in intricate environments. Recent deep learning methodologies, such as Faster R-CNN and YOLO variations, provide enhanced accuracy; nonetheless, they frequently encounter challenges with elevated noise levels and diminished contrast characteristics of nighttime images.

YOLOv9 represents a recent advancement in the YOLO line of models, renowned in the field of object identification [1, 2]. YOLOv9 upholds the legacy of YOLO by delivering exceptionally rapid detection speeds and a high degree of accuracy. YOLOv9 presumably employs more efficient and resilient neural network architecture. This may involve employing more advanced backbones for feature extraction and incorporating cutting-edge technology in machine learning, such as transformers or improved CNNs. Detecting road markers is the foundation upon which autonomous driving and computer vision are built. The objective of these responsibilities is to recognize and adhere to road markers, including lane lines, and to traverse real-time stop lines as they shift positions instantaneously. The use of onboard sensors and computers accomplishes this [3].

The identification of road signs at night is crucial for ensuring safety and efficiency on the road. The following are key reasons for the significance of road sign detection during nighttime: Driver and Passenger Safety: At night, natural sight is markedly diminished, rendering traffic signs less discernible for drivers. Accurate and clear identification of these signals enables drivers to make suitable and safe judgments when operating a vehicle, therefore diminishing the likelihood of accidents. Adhere to road sign Regulations: Road signs convey essential information concerning speed limits, limitations, and additional directives. Effective nighttime detection will enhance driver compliance with traffic regulations, consequently decreasing violations and improving traffic order. Mitigates Driver Fatigue: Nighttime driving can be more exhausting as the eyes exert greater effort to perceive signs and the roadway. A proficient traffic sign-detecting system can alleviate this load by delivering explicit warnings and information to motorists. Support for Autonomous Vehicle Technology: Autonomous cars depend on precise detection systems to identify traffic signs and other roadway components. Accurate nighttime detection is essential for the safe and reliable operation of autonomous vehicles without human oversight. Enhanced Advanced Driver Assistance Systems (ADAS) technology: Advanced driver assistance systems, such as traffic sign recognition, utilize detection technologies to aid drivers by delivering real-time information regarding road conditions and traffic signage. Enhanced nocturnal detection augments the efficacy of this ADAS system [4, 5].

Driving at night presents considerable hazards to road safety owing to diminished visibility, glare from artificial illumination, and heightened dependence on driver perception. The World Health Organization (WHO) reports that road traffic injuries are a predominant source of global mortality, with nocturnal incidents contributing significantly to the death toll. Approximately 1.19 million individuals perish annually due to road traffic collisions. Road traffic injuries constitute the primary cause of mortality among children and young adults aged 5 to 29 years [6].

The hazards of nocturnal driving are exacerbated by inadequate illumination, which hinders the prompt recognition of pedestrians, bicycles, and other vehicles. This is especially crucial in low-resource environments and rural regions, where the infrastructure for sufficient street lighting is frequently deficient. Tackling this issue is crucial for enhancing driver safety and for the progression of technologies such as autonomous vehicles and intelligent transportation systems that depend on precise and reliable object detection algorithms.

Current object detection frameworks, although proficient in well-lit settings, frequently encounter difficulties in low-light scenarios due to issues like low contrast, glare, and inconsistent lighting. This paper introduces an innovative amalgamation of CLAHE with the YOLOv9 detection framework to mitigate these constraints. The objective is to augment object visibility and enhance detection precision in difficult nocturnal settings, consequently fostering safer roadways and more intelligent urban systems. CLAHE is an image processing approach employed to enhance image contrast in a more adaptive and regulated manner than conventional histogram equalization methods [7]. CLAHE is crucial in data preparation due to its capacity to enhance image quality, hence augmenting the efficacy of numerous computer vision and image processing applications [8]. CLAHE is proficient at enhancing visual contrast, particularly in photos characterized by low contrast or extreme brightness or darkness. Enhancing contrast facilitates the recognition and analysis of significant components within a picture. This research will detect road signs, especially at night, using the SSD model, namely YOLOv9 combined with CLAHE. This research can have a significant impact on researchers in the same field.

The suggested YOLOv9 framework utilizes CLAHE to improve image quality, guaranteeing strong performance in low-light environments. This combination rectifies the deficiencies of previous approaches by sustaining elevated detection accuracy while attaining real-time processing velocities. Our method establishes a new standard for nighttime object detection by combining state-of-the-art detection algorithms with advanced preprocessing techniques. This has implications for applications in surveillance systems and autonomous vehicles.

The following section is a succinct summary of the study's principal contributions: This study gathers videos of real drivers maneuvering autos. This creates the Taiwan Road Marking Sign Dataset at Night (TRMSDN), encompassing road markings. (2) A variety of YOLO models are evaluated to ascertain which one is the most effective in identifying Taiwan's road signs. (3) The study analyzes and contrasts the differences among these three methods, examining the impact of CLAHE on photos acquired in nighttime driving conditions. (4) The identification algorithm that was developed in this work is currently being implemented in a variety of YOLOv9c versions to assess its ability to accurately identify road markings in Taiwan. (5) Experiments indicate that CLAHE has the potential to improve the efficacy of all models.

The subsequent sections will delineate the development of this work. In Section II, we examine some comparable works. Our proposed methodology is comprehensively explained in Section III. Section IV delineates the methodology and outcomes. Our findings are thoroughly examined and discussed in Section V. In Section VI, we report our results and propose possible directions for future research.

3.1 Taiwan road marking sign dataset at night

Additionally, we executed our experiment using genuine traffic signs located on conventional highways in Taiwan. The signs included indicators for vehicle turning directions, speed limits, pedestrian crossings, and stop lines. Upon recognizing these signals, vehicles should either adjust their behavior or have their movement restricted. This research sought to tackle the challenge of road sign recognition by recording video footage from the driver's viewpoint in various Taiwanese cities and manually collecting traffic signs to develop a distinctive dataset called the TRMSDN dataset [11]. A total of 4,386 distinct photos were collected for training purposes. The photos were classified into 15 distinct categories, with 80% designated for the training set and 20% for the testing set. A 512 by 288-pixel image is utilized.

Table 1 and Figure 1 exhibit the training and testing labels for each category of road marking sign, respectively. The document has both categories of labels. The TRMSDN dataset indicates an average of 399 to 409 occurrences per class. Access the following URL: https://drive.google.com/drive/folders/1e5xbkuqnN2EeoCleV9JQ3zOTOB8F_YU1?usp=sharing.

Table 1. TRMSDN dataset

Class	Name	Training	Testing	Total Image
P1	Turn Right	277	69	405
P2	Turn Left	301	75	401
P3	Go Straight	283	71	407
P4	Turn Right or Go Straight	307	77	409
P5	Turn Left or Go Straight	272	68	403
P6	Speed Limit (60)	276	69	400
P7	Zebra Crossing (Crosswalk)	327	82	401
P8	Slow Sign	275	69	399
P9	Overtaking Prohibited	310	77	404
P10	Barrier Line	295	74	409
P11	Cross Hatch	302	76	398
P12	Stop Line	283	71	403
	Total	3509	877	4386

1.png

Figure 1. TRMSDN dataset instances

3.2 Experiment setting

In this experiment, we classify our dataset into two categories: Dataset 1 and Dataset 2. Dataset 1 consists of the unaltered image without any image enhancement methods applied. Dataset 2 is the conclusive set, comprising the CLAHE-enhanced original image. Figure 2 exhibits an example of the image enhancement method with (a) the original image and (b) CLAHE. Moreover, Figure 3 shows our research workflow. Our works will implement YOLOv9c to train and test our Dataset 1 and Dataset 2. YOLOv9 seeks to mitigate information bottlenecks with an auxiliary supervision architecture termed Programmable Gradient Information (PGI). PGI is primarily intended as a training aid to enhance the efficiency and accuracy of gradient backpropagation through connections to preceding layers, utilizing a detachable branch that allows for the elimination of these supplementary computations during inference, hence optimizing model compactness and inference speed. To enhance these interconnections, it employs multi-level auxiliary information with integration networks that aggregate gradients from various convolutional stages to combine significant gradients for propagation.

21.png

22.png

Figure 2. Example of the image enhancement methods with (a) Original image, and (b) CLAHE image

3.png

Figure 3. Research workflow

3.3 Training result

Table 2 shows the training result in the TRMSDN dataset with YOLOv9c. Among the datasets, Dataset 1 has an average accuracy of 74.5 percent, while Dataset 2 demonstrates the best accuracy of 76.6 percent. The CLAHE algorithm in Dataset 2 earns the highest score and has the potential to increase the mAP performance by 2.1% when compared to the original images included in Dataset 1.

Table 2. Training result in TRMSDN dataset with YOLOv9c

Furthermore, Figure 4 explains the training process of YOLOv9c with CLAHE (Dataset 2). The results of the training for each category are presented in a manner that is proportional to the number of times the neural network repeated the training process. Following more than thirty cycles of training, the various loss functions started to decrease, which was an indication that the training of the network's parameters had reached a point of convergence. All the loss functions moved in the same direction and eventually decreased, which is an indication that the parameters of the network were trained appropriately.

4.png

Figure 4. Training process YOLOv9c with CLAHE dataset

YOLOv9 is a substantial advancement in real-time object identification, delivering significant advantages in terms of efficiency, accuracy, and adaptability. It is a vital development in the field. YOLOv9 establishes a new standard for research and application that will be conducted in the sector in the future by tackling crucial difficulties using innovative technologies such as PGI and GELAN. Considering the ongoing development of the artificial intelligence community, YOLOv9 serves as a powerful illustration of the role that collaboration and creativity play in propelling technological advancement. YOLOv9c experiment during validation batch 0 with CLAHE Dataset 2 is shown in Figure 5. In our experiment, we just implemented the default configuration with YOLOv9 data augmentation approach. The performance and robustness of YOLO models can be significantly enhanced by incorporating a variety of augmentation methods, including HSV augmentation, image angle/degree, translation, perspective transform, image scale, flip up-down, and flip left-right, as well as more advanced techniques like Mosaic, CutMix, and MixUp.

5.png

Figure 5. Validation batch 0 with YOLOv9c and CLAHE dataset

The Intersection over Union (IoU) quantifies the overlap ratio between the predicted bounding box (pred) and ground truth (gt), as shown in Eq. (1) [25, 26].

$I o U=\frac{\text {Area}_{\text {pred}} \cap \text {Area}_{\text {gt}}}{\text {Area}_{\text {pred}} \cup \text {Area}_{\text {gt}}}$       (1)

However, the output examples can be categorized into three kinds. True positive (TP) refers to accurately identified samples, false positive (FP) indicates samples that were incorrectly identified, and true negative (TN) represents samples that were not recognized. Precision and Recall are represented by Yuan et al. [27] in Eqs. (2)-(3).

${Precision} (P)=\frac{T P}{T P+F P}$              (2)

${Recall}(R)=\frac{T P}{T P+F N}$                   (3)

Another evaluation index, F1 [28], is shown in Eq. (4).

$F 1=\frac{2 \times{Precision} \times {Recall}}{{Precision}+ {Recall}}$             (4)

Yolo loss function based on Eq. (5) [28].

$\begin{gathered}\lambda_{\text {coord }} \sum_{i=0}^{s^2} \sum_{j=0}^B l_{i j}^{o b j}\left[\left(x_i-\hat{x}_i\right)^2+\left(y-\hat{y}_i\right)^2\right] \\ +\lambda_{\text {coord }} \sum_{i=0}^{s^2} \sum_{j=0}^B l_{i j}^{o b j}\left[\left(\sqrt{w_i}-\sqrt{\widehat{w}_i}\right)^2+\left(\sqrt{h_i}-\sqrt{\widehat{h}_i}\right)^2\right] \\ +\sum_{i=0}^{s^2} \sum_{j=0}^B l_{i j}^{o b j}\left(C_i-\hat{C}_i\right)^2+\lambda_{\text {noob } j} \sum_{i=0}^{s^2} \sum_{j=0}^B l_{i j}^{n o o b j}\left(C_i-\hat{C}_i\right)^2 \\ +\sum_{i=0}^{s^2} l_i^{o b j} \sum_{c \epsilon c l a s s e s}\left(p_i(c)-\hat{p}_i(c)\right)^2\end{gathered}$         (5)

where, $l_{i j}^{o b j}$ denotes if the object appears in cell i, and $l_{i j}^{o b j}$ notes that the $j^{t h}$ bounding box predictor in cell i is responsible for the prediction. Next, $(\hat{x}, \hat{y}, \widehat{w}, \hat{h}, \hat{c}, \widehat{\text { and } p})$ are represented as the predicted bounding box's center coordinates, width, height, confidence, and category probability. The symbols lacking the cusp represent authentic labeling. Furthermore, our research has determined the λ_coordvalue to be 0.5, indicating that the errors in width and height are less significant in calculations. A λ_noobjvalue of 0.5 is implemented to mitigate the influence of several grids devoid of objects on the loss value.

The primary focus of this research is to examine the ways in which image enhancement methods can enhance the performance of the original image. Using techniques such as CLAHE, our study blends the original image with other image-enhancing techniques. To train, we use a variety of images, both in terms of quantity and size. As part of our research, we study and investigate CNN models that have been integrated with a variety of backbone architectures and extractor features, notably YOLOv9c, for the purpose of identification of road markings during the night.

Additionally, we derived the subsequent findings from our experiments: Experimentally, Dataset 2, which consists of the original image augmented by CLAHE, is the most effective dataset. This study endorses the utilization of CLAHE's YOLOv9c as the superior model, with a mean Average Precision (mAP) of 76%. Augmenting the noise during training will prolong the training duration and diminish the frequency of general errors. Consequently, enhancing item recognition efficacy can be achieved by integrating the dataset with CLAHE images and the original photographs. The results of this study have significant significance for practical applications, including autonomous vehicles, which indicates the improved detection capabilities of the suggested framework can boost the reliability of autonomous driving systems, especially in nocturnal conditions, hence mitigating accident risks and assuring safer navigation.

In the future, one of our objectives is to enhance the dataset that we have in Taiwan, namely by collecting data in a wider range of circumstances than merely at night. Because we want to emphasize the advantages of image enhancement, we are going to analyze it in comparison to other standards for road marking signs. Future work will involve extending the dataset to include diverse nighttime conditions and urban environments across different regions, starting with additional data collection in Taiwan. This expansion will enable the model to generalize better across varied scenarios.