A Novel Deep Learning Architecture for Real-Time Autonomous Driving Perception with Fuzzy Graph-Cut and Metaheuristic Optimization

The intelligent vehicle is the bearer of the thorough integration of numerous technologies and is a crucial component of intelligent transportation systems. Despite the promising results of vision-based autonomous driving, the challenge of analyzing the complex traffic scenario using the data gathered remains. Recently, numerous tasks related to autonomous driving have been developed independently utilizing various models, including the object detection task and the intention identification job [1]. Designing and developing successful autonomous driving systems (ADS) involves many obstacles, particularly in developing nations with inadequate road infrastructure. For sophisticated ADS to determine whether to change lanes, slow down, or even stop, it must be able to detect barriers in lanes. Poor or non-existent lane line demarcation, inadequate traffic control, and a variety of driving habits are some of the difficulties faced by developing nations like India. This makes it difficult to create reliable lane-keeping or lane-changing decision systems for an ADS [2]. In actual driving situations, autonomous driving necessitates the accurate and dependable detection and recognition of nearby objects. Even though several object detection algorithms have been put out, not all of them are reliable enough to identify and detect things that are obscured or truncated [3]. Road safety and traffic management may benefit from the system's ability to make decisions in real time, which is essential for maximizing autonomous vehicle maneuvering [4]. Intrusion for the autonomous car concept to become a reality, lane detection is essential. The necessity for substantial hand-crafted features and post-processing approaches in traditional lane recognition methods makes the models feature-oriented and vulnerable to instability due to fluctuations in road situations. Convolutional neural network (CNN) models, in particular, have been suggested and used in recent years for pixel-level lane segmentation using Deep Learning (DL) models [5]. Stakeholders in autonomous vehicles (AVs) are still looking for proof of the new technology's safety performance through AV testing on in-use roads, AV-specific road networks, and AV test tracks. Some public trust in the safety of AV operations has been damaged, nevertheless, by recent AV-related fatalities on in-service roads. Furthermore, the real-world driving environment cannot be sufficiently described by test tracks. Driving simulators are still a popular way to test for AV because of this [6].

The following is the primary contribution of the suggested method:

(1) The researchers used the GRAZ dataset and nuScenes, which offers high-resolution and varied road scene images, to conduct thorough training and evaluation for lane line and object detection in challenging driving situations.

(2)  nuScenes provides richer data (1,000 scenes, 1.4 million photos) from cameras and other sensors, while the original study depends on the GRAZ dataset (1115 images, camera-only). This allows for prolonged assessment without the need to completely rebuild the model.

(3) Gaussian filtering was used to improve edge information and reduce noise, while Min-Max normalization was used to scale image intensities. This improved the input quality for tasks involving detection and segmentation.

(4) A hybrid segmentation technique that handles ambiguity and maintains spatial continuity by combining Graph-Cut optimization with fuzzy Gaussian Mixture Models. This allows for the precise extraction of lane lines and vehicle regions.

(5) HBE-CSA, a novel metaheuristic algorithm, was presented. It optimizes deep learning parameters to increase detection precision and convergence speed by combining the exploration strengths of the Bald Eagle Search and Crow Search algorithms.

(6) The entire system achieves 99.5% detection accuracy and exhibits strong robustness and generalization across a variety of scenarios.

The following is how the following sections will be formatted: Our suggested approach's literature review and other pertinent data are covered in Section 2. Section 3 explains the suggested approach; Section 4 describes the experiment and data analysis; and Section 5 summarizes the study's findings and recommendations for further research.

For autonomous driving systems, the suggested architecture offers a complete deep learning solution for real-time object and lane line recognition. First up is the GRAZ dataset and nuScenes, which provides a wide variety of rich driving situations. Image quality is improved by applying pre-processing methods like Gaussian Filtering and Min-Max Normalization [29]. A new segmentation technique called Graph-Cut Optimized Fuzzy GMM (GC-GMM) combines spatial optimization and probabilistic modeling to precisely extract lane and object sections [30]. To improve detection performance even more, a new Hybrid Bald Eagle-Crow Search Algorithm (HBE-CSA) is suggested to efficiently optimize deep learning parameters. Figure 1 shows the five successive phases that make up the suggested framework.

Figure 1. Workflow for the suggested approach

3.1 Image pre-processing

Noisy values that are too high, too low, or almost the same as a certain number are swapped out for zero by default. Using unsupervised instance-based filtering, duplicate instances are eliminated. In order to fill in the missing values, the modes and means of the attribute values are used. Across various numerical ranges, the cleaned samples' values fluctuate. After that, the data is transformed by applying the Min-max normalization procedure, which is outlined in Eq. (1), to normalize the numerical features that they fall between 0 and 1.

$X=\frac{X-M I N}{M A X-M I N}$            (1)

where, X is the numerical value for each feature sample, MIN is the feature's lowest value, and MAX is its greatest value.

The parameters used for Gaussian filtering must be specified in order to guarantee experiment reproducibility. With a sigma value of 1.0 regulating the Gaussian distribution's standard deviation and a kernel size of 5×5 pixels defining the smoothing process's region of action, the Gaussian filter is applied. These parameters are essential because they affect the level of noise reduction and the retention of significant image characteristics. Clarity on the noise reduction step and easier experiment replication will result from including this comprehensive information on the Gaussian filtering procedure. Using a Gaussian kernel, the image is convolved.

$G(x, y)=\frac{1}{2 \pi \sigma^2} \exp \left(-\frac{x^2-y^2}{2 \sigma^2}\right)$            (2)

The degree of smoothing and the breadth of the kernel are determined by the standard deviation of the Gaussian distribution, σ, where G(x,y) is the value of the Gaussian function at coordinates (x,y).

3.2 Graph-cut optimized fuzzy GMM segmentation (GC-GMM)

The next stage of our innovative segmentation model is the use of the fuzzy Gaussian Mixture Model (GMM) to divide the images into areas or segments following pre-processing with contrast stretching and homomorphic filtering techniques. In contrast to classic GMM, which places every pixel in a single cluster, our fuzzy GMM permits pixels to be a part of several clusters with different levels of affiliation. By taking into consideration the uncertainty in pixel membership, this addition offers a more adaptable and realistic depiction of image segmentation.

Initializing the Gaussian mixture model's parameters is the first step in processing the fuzzy GMM segmentation. Cluster-describing parameters include the number of clusters, covariance matrices, mean vectors, and the degree to which each pixel belongs to each cluster. Using fuzzy membership functions, like the Gaussian membership function, the degree of membership i, j of a pixel i to a cluster j is calculated. This function uses a pixel's similarity to the cluster's mean vector and covariance matrix to determine how likely it is that the pixel belongs to that cluster.

$U_{I J}=\frac{1}{\sum_{k=1}^K w_k} \sum_{k=1}^K w_k \cdot N\left(\frac{x_i}{U_k} \sum_k\right) \cdot N\left(\frac{x_i}{U_j} \sum_j\right)$            (3)

The degree of membership of pixel i to cluster j is represented by ‘j’ the mixing coefficients are indicated by w_k, the Gaussian probability density function of pixel i given the mean vector k and covariance matrix k is represented by $N\left(\frac{x_i}{v_k} \Sigma_k\right)$ and the same is represented for cluster j  by $N\left(\frac{x_i}{v_j} \Sigma_j\right)$ We present a unique graph-cut optimization-based post-processing method to enhance boundary adherence and refine the segmentation results. In this stage, we use a graph to represent the segmentation problem, with nodes standing for pixels and edges for pairwise associations between pixels. By rephrasing the segmentation problem as an energy minimization effort, graph-cut optimization can be used to maximize each pixel's label assignment.

Both a data term and a smoothness term are combined in the graph-cut optimization energy function, or E_graphcut. The smoothness term promotes smooth borders and spatial coherence, whereas the data term describes the probability that each pixel belongs to a certain label. Here's how to define the energy function:

$E_{\text {graphcut }}=\sum \varphi_d\left(i, l_i\right)+\sum_{(i, j) \in \text { edges }} \varphi_s\left(i, l_i\right)$            (4)

The label for pixel I is represented by l_i, the data term potential function is indicated by φ_d (i, l_i), and the smoothness term potential function for adjacent pixels I and J is indicated by φ_s (i, l_i).

After graph-cut optimization, we use label fusion to improve the segmentation accuracy even more. The knowledge from several segmentation results, such as the original fuzzy GMM segmentation and alternative segmentation outputs from various algorithms or parameter settings, is included into this stage. We obtain strong and accurate segmentation findings by giving greater weight to labels that are more dependable and consistent.

This is a representation of the label fusion process:

$l_{\text {final }}(i)=\frac{\sum\left(w_{\text {seg. }} l_{\text {seg }}(i)\right)}{\sum w_{\text {seg }}}$                (5)

The final label applied to pixel I is indicated by l_final (i), the weight allocated to each segmentation result is represented by w_seg, and the label applied to pixel I in a particular segmentation result is represented by l_seg (i).

3.3 Proposed hybrid optimisation algorithm

The successful optimization of the GMM-UBM's hyper-parameters is necessary for its log-likelihood. BESA and CSA mutualism is demonstrated by the suggested hybrid algorithm. The predation tactic of the bald eagle is replicated by the BESA. There are three main components to it: choosing a precise search area, searching inside that area, and determining the optimal opportunity to attack the target. The bald eagle's most recent movement is connected to the search space that was chosen during the select step. Eagles look for the ideal spot in the search area to carry out their hunting maneuvers. In the following section, eagles go in different directions within the first chosen search zone in an attempt to find prey. During the diving stage, the eagles attack the prey to capture it by diving from the best position. BESA is good at exploitation in general. However, it lacks the capacity to explore the entire world. As a result, it occasionally struggles to complete the specified optimization task effectively and falls short of reaching the global optima.

The local optima can be avoided by the CSA. Crow's activities, including as identifying members of a group and warning one another in unfavourable circumstances, are replicated by the CSA. Additionally, they can remember their food hiding spots for longer and employ a variety of gimmicks to communicate with one another. Figure 2 illustrates how CSA and BESA's hybridization may aid in the discovery of better solutions by taking into account their respective advantages and disadvantages.

Figure 2. The hybrid bald eagle-crow search algorithm (HBE-CSA) flow diagram

3.4 Performance evaluation criteria

Evaluates the performance of the proposed hybrid HBE-CSA using four popular classification metrics: accuracy, recall, precision, and F1-score. The suggested model's usefulness is fully evaluated by these measures, with accuracy being a key component in determining the model's overall precision. This assessment measure can be calculated using Eq. (6):

Accuracy $=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{FP}+\mathrm{TN}+\mathrm{FN}}$                (6)

To calculate the precision, divide the number of accurately predicted positive cases by the total number of anticipated positive cases. As shown below, Eq. (7) is used to calculate this evaluation metric:

Precision $=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$                (7)

Among all outcomes having a true value of positive, the recall rate is the percentage of accurate model predictions:

Recall $=\frac{T P}{T P+F N}$                (8)

$\mathrm{mAP} @ 0.5=\frac{1}{N} \sum_{i=1}^N A P_i$                (9)

where, mAP is mean average precision, @0.5 is intersection over union threshold, and N is averaging over multiple classes.