Real-Time Vehicle Classification Using LSTM Optimized by Oppositional-Based Wild Horse Optimization

Urban and highway traffic analysis and planning tools rely on key elements like vehicle classification (cars, trucks, buses, autos, and two-wheelers) on urban roads and highways, as well as statistical traffic flow estimation [1]. Real-time highway traffic flow monitoring is still a challenging undertaking, nevertheless, in this day of rapidly advancing technology and urbanisation. Traditional approaches, such as human observers, fall short as they are inadequate for vehicle recognition, classification, and generating real-time traffic flow data [2]. Human observers can be error-prone and subjective, leading to inaccurate and inconsistent results. They may also work at a slower pace, be affected by fatigue, and struggle in challenging conditions like bad weather or low-light situations. Additionally, employing human observers for large-scale traffic monitoring can be inefficient and costly, making automation a more practical choice. Inadequate road/highway traffic management leads to traffic law violations, congestion, and accidents. Conventional techniques (e.g., RADAR, LIDAR, RFID, or LASAR) require significant time, money, and effort [3]. The challenge lies in achieving automatic vehicle classification from traffic surveillance camera recordings [4]. Although there are publicly available vehicle datasets, not all are suitable for training traffic surveillance algorithms [5]. The research's key contributions include addressing real-time vehicle classification for traffic management, handling diverse environmental conditions, combining YOLO-v2, ResNet50, and optimized LSTM models, streamlining hyperparameter optimization with OWHO techniques.

Traffic surveillance photographs that are captured have a lesser resolution and are affected by different lighting, weather, and occlusion circumstances [6]. Modern artificial intelligence techniques, especially those based on deep learning and machine learning, are harnessed by online video processing systems [7]. Convolutional neural networks (CNN) - ResNet-50 are utilized in the study for automatic feature extraction. Compared to conventional feature extraction methods, this approach is more reliable and discriminative. Models based on ResNet-50 have the ability to share parameters and have local connectivity. The features from various layers are extracted utilizing the ResNet-50's layered architecture [8]. There is still a lot of room for classification study even if LSTM has dealt with time series issues adequately. Vehicle identification using the conventional LSTM network model alone cannot produce better classification results [9].

LSTM networks are well-suited for handling temporal sequence data like traffic surveillance. Integrating hyperparameter optimization is necessary to adapt the model to the specific challenges and complexities of traffic data, improve its performance, and ensure its ability to generalize effectively to real-world scenarios.

To enhance the LSTM model, the Paper aim to identify optimal hyper parameters. It is possible to think about hyper parameter optimization as an optimization issue where the goal is to choose a value that maximises performance and produces the desired model [10]. The goal of the scientific field of optimization is to maximise or minimise particular objective functions. It is present in almost all disciplines, including economics, text clustering, and pattern recognition. Recent attempts to resolve these issues have been made by academics employing metaheuristics, a novel class of approximation method [11]. The prior swarm-based optimization techniques have some flaws, worries, and issues. This paper analyses the WHO optimizer algorithm, which was motivated by wild horses' social behaviours [12]. Therefore, in order to explain complex problems and functions, this algorithm needs to be enhanced. It is a concept of computational opposition that aims to accelerate the convergence of soft computing algorithms. It is based on the idea of opposed interactions between things. Calculating both the initial outcome and its reverse is required [13].

The following sections are organised as follows: Section 2 illustrates the recent literature reviews in this study context. The subsequent research demonstrates the methodology in section 3, results, and discussion with interpretations in section 4, followed by the conclusion.

A precise and effective VD method is crucial because it is the first stage in the process of classifying a vehicle type. The accuracy of traditional VD and classification techniques decreases dramatically due to the backdrop, changing lighting, climatic conditions, and varying vehicle sizes in a frame. The motion-based VD system may identify moving cars in video frames. The method, however, is unable to recognise items in a single image and fails to recognise stationary objects. Furthermore, feature-based VD is not universal because finding suitable features to describe the thing is challenging. This research involves a popular real-time object detection algorithm named YOLOv2 for object detection. ResNet-50, a CNN with 50 layers, is then exploited in the study for the feature extraction procedure. The extracted features from ResNet50 are used as input for LSTM in order to classify vehicle types. This research includes four classes: Light Moving Vehicle (LMV), Heavy Moving Vehicle (HMV), Auto, and Two-Wheeler (TW). Furthermore, the research intends to configure the LSTM’s hyperparameters suitable for classifying vehicle types. Deciding the optimal hyperparameters is crucial in determining the overall LSTM performance. Because they have a tremendous influence on the final output, it is necessary to consider optimal hyperparameters carefully. However, it is complicated and time-consuming to identify the optimal hyperparameters through a manual or trial and error process. Therefore, the research integrates WHO to resolve the computational complexity while configuring the ANN architecture. Fig. 1 shows the overall research methodology starting from video to frame conversions and getting into object detection using Yolo-v2, then feature extraction using ResNet50 and finally getting into classification with LSTM.

3.1 Dataset description

This study employs real-time vehicle surveillance videos collected from the Surveillance Camera of the Transport Department in Mysuru, India. A total of 36 videos were selected for this research, which represent daytime lighting, cloudy, and nighttime lighting conditions. Out of the 36 real-time vehicle surveillance videos, 30 were utilized for training purposes, with 11 representing daytime lighting conditions, 14 representing cloudy conditions, and 5 representing night-time lighting conditions. Each video has duration of 3 seconds and contains 100 frames, resulting in a total of 1100 frames for the daytime lighting condition, 1400 frames for the cloudy condition, and 500 frames for the night-time lighting condition. The remaining 6 videos were reserved for testing purposes. Figure 1 illustrates overall proposed method.

1.png

Figure 1. Overall research methodology

3.2 Yolo-v2

The YOLO-v2 object detector is used in this research as a methodology as a part of identifying and classifying vehicles in real-time. The role of YOLO-v2 in this research is to detect and classify objects in the videos, which provides valuable data for the analysis and interpretation of road traffic conditions also helps in improving the road safety and traffic management. The YOLO-v2 process was performed on a total of 3000 frames, comprising 1100 frames representing daytime lighting conditions, 1400 frames representing cloudy conditions, and 500 frames representing night-time lighting conditions. These frames were utilized for the detection of vehicles. The YOLOv2 model can process frames under different lighting conditions, such as cloudy conditions, night-time lighting conditions, and day lighting conditions, in the same way as it processes frames under any other lighting conditions. For each frame, the YOLOv2 model processes the image in a single forward pass, generating predictions for the presence of vehicle and their bounding boxes. The model outputs the coordinates of the bounding boxes, the class probabilities for each box, and the confidence scores for each prediction. The raw predictions from the YOLOv2 model need to be post-processed to filter out low-confidence predictions, and to merge overlapping bounding boxes. The processed frames can be displayed with the detected objects and their bounding boxes drawn on them. Under cloudy conditions, the input frames may have lower overall brightness, making it harder for the model to differentiate between objects and the background. During night-time lighting conditions, the input frames can have very low overall brightness, and may have areas of high contrast, such as streetlights. This can make it harder for the model to identify objects, especially if the objects have similar colors or textures to the background. Under day lighting conditions, the input frames will have higher overall brightness and more consistent lighting, making it easier for the model to identify objects and their bounding boxes. YOLOv2 is capable of detecting vehicle that is partially occluded by other objects or by the boundaries of the image. This makes it a good choice for VD under different lighting conditions, as objects may be partially obscured by shadows or other objects in the scene. YOLOv2 is an end-to-end learning system that predicts object locations and class probabilities in a single forward pass. This makes it a good choice for object detection under different lighting conditions, as it does not require separate detection and classification steps. By using YOLOv2 for VD under different lighting conditions, the advantage is real-time processing speed, single pass detection, end-to-end learning, and robustness to partial occlusions, and improved accuracy. The following Figure 2 demonstrates the working flow of Yolo-v2.

ria10_2.png

Figure 2. YOLO-v2 process flow

3.3 Convolution neural network (CNN)

CNN, neural network architecture, excels in image recognition [27] and various other visual data processing tasks. ResNet50 is a specific architecture of CNN that was introduced in a 2015 paper by researchers at Microsoft. ResNet50 uses a technique called "residual connections" to allow the network to learn more effectively even as it becomes deeper. This allows ResNet50 to achieve better performance than other CNN architectures on image recognition tasks. In this study, ResNet 50 was used for the feature extraction process in order to identify and classify vehicles in real time. ResNet 50, a deep residual network, was utilized for feature extraction due to its ability to effectively extract features from images, even in low-light conditions. The ResNet 50 model architecture is able to learn deep residual representations by using identity mappings, which allows for the network to be trained with much deeper architectures than traditional approaches. This helps to improve the accuracy of the feature extraction process and allows for more accurate vehicle classification.

3.4 Recurrent neural network (RNN)

LSTM is a type of RNN that is able to capture long-term dependencies in sequential data. LSTMs are able to overcome the vanishing gradient problem that occurs in traditional RNNs by introducing a memory cell, input gate, forget gate, and output gate. These gates allow LSTMs to selectively store or discard information in the memory cell, which allows them to maintain a useful representation of the past for long periods of time. LSTM is a variation of the RNN architecture which is able to learn the context of the data, and capture the dependencies of the input elements over time. While RNN's are designed to capture the sequence information and LSTM are designed to capture the long-term dependencies.

An RNN is an ANN with arbitrary connections between neurons, often fully connected between neighbouring layers. The inputs that the network nodes receive are the current data point x (t) and the hidden state values of the hidden layer in the previous state h (t-1). As a result, by virtue of recurrent connections, inputs at time t have an effect on the network's future outputs. A standard RNN  with an input vector ve=(ve₁, ..., ve_T) calculates a hidden vector hd=(hd₁, ..., hd_T) and an output vector oy=(oy₁, ..., oy_T) by iterating Eq. (1) and Eq. (2) over t=1, …, T.

$h d(t)=A_Q\left(W g t_{(h x)} x^{(t)}+W g t_{(h h)} h d^{(t-1)}+b i_h\right)$     (1)

$o y^{(t)}=\sigma\left(W g t_{(h y)} h d^{(t)}+b i_y\right)$     (2)

where, bi_y and bi_h are vectors of biases, Wgt_(hx)Wgt_(hh) and Wgt_(hy) are weights matrices of input-hidden layer, hidden-output layer and recurrent connections respectively. A_Q is an activation function [24]. Standard neural networks are trained using the backpropagation via time process over a number of time steps.

3.5 Long term short term memory networks

RNN from the 1980s are where LSTM networks got their start. To solve the disappearing and expanding gradient issues of conventional RNNs, Hochreiter and Schmidhuber created the RNN architecture [28]. Their architecture makes it possible to gather information while in use and employs feedback to keep track of prior network call conditions. In comparison to traditional RNNs, the LSTM model has demonstrated exceptional ability to learn long-range needs in real-world applications. As a result, the LSTM model is used in the majority of cutting-edge applications. The LSTM typically consists of a few memory blocks. Memory cells and gates are found in memory blocks. Information flow is controlled by gates, and memory cells are capable of remembering the network's temporal state through self-connections. A forget gate, an output gate, and an input gate are all included in every memory block. The output gate regulates how cell activations are distributed throughout the rest of the network. The LSTM cell's architecture is depicted in Figures 3 and 4.

3.png

Figure 3. Flowchart for the LSTM's classification of vehicle types

4.png

Figure 4. RNN-LSTM layer model

LSTM is used as a classifier in conjunction with YOLO-v2 and ResNet50 for identifying and classifying vehicles in real-time. The LSTM is configured optimally to increase the accuracy of the VD and classification process. Additionally, the research utilizes OWHO techniques to identify the optimal hyperparameters for LSTM, which helps to improve the performance of the overall method. The use of LSTM allows for the detection and classification of vehicles under various factors like Rainy weather, cloudy weather, Low illumination, etc., which can be challenging for other algorithms.

Hyperparameter optimization is essential in LSTM because it allows the model to find the best set of parameters for the specific task and dataset being used. LSTMs have many hyperparameters, like the number of hidden units, the number of layers, and the learning rate, that can greatly impact the performance of the model. By optimizing these hyperparameters, the model can be fine-tuned to achieve better results, such as higher accuracy or lower loss. Without proper optimization, the model may not perform as well or may take longer to converge. Optimization techniques are essential in hyperparameter optimization because they allow for efficient and automated search for the optimal set of hyperparameters. Without optimization techniques, finding the best set of hyperparameters would likely require a significant amount of time and resources, as it would involve manually testing different combinations of hyperparameters.

3.6 Opposition based Wild Horse Optimization (OWHO)

WHO is a metaheuristic optimization algorithm that is inspired by the behavior of wild horses utilized for identifying the hyperparameters for LSTM. It is a population-based optimization algorithm that can be used to solve optimization problems in various fields. WHO uses a group of individuals, called a population, to represent possible solutions to the optimization problem. The algorithm iteratively updates the population by moving the individuals towards better solutions. WHO has been utilized in different applications, like image processing, signal processing, and machine learning. Oppositional-based learning is a technique that involves adding an opposite strategy to the traditional optimization algorithm. In the context of WHO, adding an opposite strategy can help improve performance by allowing the algorithm to explore a wider range of solutions. This can be particularly useful when dealing with complex, high-dimensional problems that have multiple local minima. The advantages of using opposition-based learning include faster convergence, increased robustness, and improved global search capabilities. Additionally, the opposition-based approach can help to mitigate the effects of premature convergence and stagnation, which are common problems in traditional optimization algorithms. Overall, incorporating opposition-based learning into traditional optimization methods like WHO can lead to more efficient and accurate solutions for a wide range of optimization problems.

Horse social structure: horses are separated into territorial and non-territorial groups. Social grouping, bonding and grazing, mating behaviour, leadership hierarchy, and dominance are only a few of the many contrasts between these two types of organisations. In this essay, we'll be concentrating on equine extraterrestrials. Herds of non-terrestrial horses form stable family units known as harems, comprising a stallion, one or more mares, and their young. Stallions are positioned near mares for communication, and mating can occur at any time. Foals start grazing within their first week of life, increasing grazing and reducing rest as they grow older. As foals reach adolescence, they leave their parent groups, and male horses old enough to breed join single groups, while female foals join other family groups. This separation prevents mating between fathers and their offspring. The suggested wild horse optimizer approach involves five primary phases, as shown in Figure 5 of the flowchart.

Establishing the initial population, setting up horse groups, and choosing leaders:

·Horse grazing and mating.

·The leader (stallion) leads and guides the group.

·Leaders are exchanged and chosen.

·Keep the finest solution.

5.png

Figure 5. Flow chart of oppositional based wild horse optimization

3.6.1 Creating an initial population

Every optimization algorithm shares the same fundamental structure. The algorithm begins with an initial random population of ( $\vec{x}$ )={ $\overrightarrow{x_1}$, $\overrightarrow{x_2}$, …, $\overrightarrow{x_n}$ }. This random population is evaluated by the target function multiple times, yielding the target value ($\overrightarrow{\mathrm{Ot}}$)={Ot₁, Ot₂, …, Ot_n}. A set of guidelines for an optimisation strategy was also useful. There is no guarantee that a solution will be found in a single run because population-based optimisation approaches look for the appropriate number of optimisation problems. However, the likelihood of discovering the best global solution rises with enough random solutions and optimization stages (iteration). We first divide this initial population into a number of categories. If N is the population's total size, then G=⌈N×PS⌉ is the total number of groups. We refer to the PS, which is the percentage of stallions in the overall population, as a control parameter for the suggested approach. The leader G (stallion) and the other members (N-G) are split evenly among the dissimilar groups, according to the number of them. Prior to being chosen by an algorithm based on fitness (the best fitness function) among the group members, group leaders are originally chosen at random.

3.6.2 Opposition based solution

The opposition-based strategy works by creating an "oppositional" solution that is the opposite of the best solution found so far. This oppositional solution is then used as a starting point for the next optimization iteration. The idea behind this is that by exploring the opposite of the best solution, the algorithm can discover new, previously unexplored regions of the solution space that may contain even better solutions. The opposition-based strategy is particularly useful for hyperparameter optimization problems because it allows the algorithm to explore a wide range of possible solutions. This can be especially beneficial when the solution space is complex, highly nonlinear, or has many local optima.

$O H_{i, G}^j=x_i+y_i-H_{i, G}^j$     (3)

In Eq. (3), OH represents the opposition-based solution, and H denotes the randomly generated solution with x_i and y_i representing the minimum and maximum values, respectively. Both the randomly generated solution and the opposition-based solution are used in the fitness computation for process evaluation.

3.6.3 Fitness computation

In LSTM models, accuracy serves as the fitness function for hyperparameter optimization, measuring how effectively the model classifies vehicle types based on input data. Typically, accuracy is computed by comparing the model's predictions against the true outputs on a test dataset. The OWHO algorithm utilizes accuracy as a guide to find optimal hyperparameters that yield the highest accuracy on the test set. This process involves iteratively testing various hyperparameter combinations and adjusting them to enhance the model's accuracy.

$Accuracy =\frac{T_r P+T_r N}{T_r P+T_r N+F_a P+F_a N}$     (4)

In Eq. (4), T_rP refers True Positive; T_rN refers True Negative; F_aP refers False Positive; F_aN refers False Negative.

3.6.4 Grazing behaviour

Foals often graze about their group for the majority of their time, as was indicated in the preceding section. To carry out grazing behaviour, we consider the stallion to be the centre of the grazing area. Group members then search the region around the stallion (graze). To imitate grazing behaviour, we presented Eq. (5), which leads group members to move and search in a dissimilar radius around the leader.

$\begin{aligned} \bar{X}_{i, G}^j= & 2 F \cos (2 \pi R F) \times \left( Stallion^j-H_{i, G}^j\right)+Stallion^j\end{aligned}$     (5)

where, $H_{i, G}^j$ is the current position of the group member (foal or mare), Stallion^j is the position of the stallion (group leader), F is an adaptive mechanism calculated by Eq. (6), R is a uniform random number in the range [−2, 2] that causes The grazing of horses at dissimilarangles (360 degrees) of group leader, π is the same as the pi number equal to 3.14,The COS function by combining π and R causes the movement in dissimilar radius, and finally $\bar{X}_{i, G}^j$ is the new position of the group member when grazing:

$\begin{aligned} & P_v=\vec{R}_1<T D R ; \operatorname{IRV}=\left(P_v==0\right); \\ & F=R_2 \Theta \operatorname{IRV}+\vec{R}_3 \Theta(\sim \operatorname{IRV}), \end{aligned}$     (6)

where, P_v is a vector consisting of 0 and 1 equal to the dimensions of the problem, $\overrightarrow{R_1}$ and $\overrightarrow{R_3}$ are random vectors with uniform distribution in the range [0, 1], R₂ is a random number wit uniform distribution in the limit [0, 1], IRV indexes of the random vector $\vec{R}_1$ returns that satisfy the condition (P_v==0). TDR is an adaptive parameter that initiates with a value of 1 and progressively decreases as the algorithm is executed following Eq. (7) until it reaches a value of 0 at the end of the algorithm's execution.

$T D R=1-iter \times\left(\frac{1}{M_{iter}}\right)$     (7)

where, M_iter represents the maximum number of times the algorithm can be executed, while iter indicates the current run.

3.6.5 Horse mating behaviour

Horses separate their foals from the group and breed them, which is one of their distinctive behaviours in comparison to other animals. Female foals join another family group once they reach puberty in order to find a partner, while male foals leave the family group before they reach adulthood and join the group of single horses. To stop the father from mating with the daughter or other family members, he is leaving.Here's how we put this behaviour into practise: A foal joins a temporary group after leaving group I, while another departs group j.Since these two foals have no familial ties, we'll presume they are both male and female and can mate once they reach adolescence. The offspring is required to leave the temporary group and join another group, denoted as group k. This cycle of emigration, mating, and reproduction is a common occurrence in all equine species. To model the behavior of horses leaving and mating, Eq. (8) is introduced, which corresponds to the Crossover operator of the mean type.

$$

\begin{aligned}

& H_{G, k}^p=Crossover\left(H_{G, i}^q, H_{G, j}^z\right) \\

& i \neq j \neq k, p=q=\text { end, }

\end{aligned}

$$

Crossover=Mean     (8)

In the context of the equation, $H_{G, k}^p$ represents the position of horse p from group k, which leaves the group to create space for a new horse whose parents are horses from group i and j, having reached puberty and mated to reproduce. These horses have no family relationship. Additionally, $H_{G, i}^q$ indicates the position of foal q from group i, which also leaves the group and, after reaching puberty, mates with the horse z in position $H_{G, j}^s$ who then departs from group j.

3.6.6 Group leadership

The group leader is responsible for directing the group to the proper location. This appropriate location is what we refer to as the water hole. This water hole must be approached by the group. In a same manner, other groups approach this water hole. In order for the domination group to use this water hole, leaders compete for it. No other groups may utilise the water hole until the dominating group has left. The group leaders must direct their members to the watering hole, use it if their group is the dominant one, and migrate away from it if another group is. Eq. (9) is suggested for this technique and distance.

$\overline{Stallion_{G_i}}=\left\{\begin{array}{ll}2 F \cos (2 \pi R F) \times\left(WH-Stallion_{G_i}\right)+W H & \text { if } R_3>0.5 \\ 2 F \cos (2 \pi R F) \times\left(WH-Stallion_{G_i}\right)-W H & \text { if } R_3 \leq 0.5\end{array}\right\}$     (9)

$Stallion _{G_i}=\left\{\begin{aligned} H_{G, i} & \ if  \cos t\left(H_{G_i}\right)<\cos t\left(Stallion_{G_i}\right) \\ Stallion_{G_i} & \ if  \cos t\left(H_{G_i}\right)>\cos t\left(Stallion_{G_i}\right)\end{aligned}\right\}$     (10)

where, $\overline{\text { stallıon }}_{\mathrm{Gi}}$ is the next position of the leader of the i group, WH is the position of the water hole, stallion_Gi is the current position of the leader of the i group, F is an adaptive mechanism calculated by Eq. (10), R is a uniform random number in the range [−2, 2], π is the same as pi number equal to 3.14.

3.6.7 Exchange and leader selection

To retain the randomization of the method, we initially choose the leaders at random. Leaders are chosen later in the process based on fitness. Eq. (10) will alter the dynamic between the group leader and corresponding member if one of the group members is more physically fit than the group leader.