Ensemble-Based Traffic Sign Recognition Using SVM, K-NN, and Decision Tree Classifiers for ADAS Dataset

The proposed methodology in this paper has been applied to address the objectives of this study which is employed on The Chinese Traffic Sign Database, which available online (Traffic Sign Recognition Database). The proposed method uses the histogram of oriented gradients (HOG) as feature extraction and three classification algorithms then at the end multiply the result of these three algorithms by its weight and take the mean by using the voting algorithm, in order to achieve high accuracy. Before this step, the performance evaluation for the three algorithms should already exist. The development of the implementation was performed based on the OpenCV predefined and NumPy libraries that provide several images processing tasks. The evaluation of the algorithms based on image accuracy. In the hardware, the code done by Anaconda3 Python then attach the dataset and the code to the Raspberry Pi in order to assess the embedded system vs Spyder software (python) using execution time, the paper Schematic diagram is shown in Figure 2.

image004.png

Figure 2.​ Schematic diagram of system operation

The proposed work includes three stages; image processing, classification, recognition as illustrated in Figure 3 the stage of image processing involved to convert the RGB image to gray scale image in order to avoid low vision and reduce the accuracy of image classification before the images need to be resized, in this process all the images be the same size for easy extracting feature and classification. The histogram of oriented gradient (HOG) is used to recognize the road sign by extracting features. Further one of the most important aspects training the images, that used thousands of images, this method used machine learning is to evaluate an algorithm by splitting a data set into two, one of those sets (the training set), on which learned some properties; which is the other set (the testing set), finally used three algorithms to classify trained images, by comparing the support vector machine (SVM) with other algorithms which are K-nearest neighbor and Decision Tree. Eventually, voting algorithm used to get high accuracy, the result of this algorithm and SVM compared with each other.

image005.png

Figure 3. The process image recognition

2.1 Image dataset

The Traffic Sign Recognition Database (TSRD) consists of traffic sign images which are 6164 in number and comprises of 58 categories of traffic sign. The database where images are stored divided into two sub-databases as training database and testing database. The testing database includes 1994 images while the training database contains 4170 images. All images on the database annotated, i.e. the four coordinates of the sign and the category [13, 14]. To clarify the dataset and ensuring a fair assessment of model performance, the dataset is split into roughly 68% for training and 32% for testing.

A photo and image resizer is a superb tool that helps maintain websites, send images via email or resize large images for printing purposes. Aside from aiding in determining the size (in pixels) of an image, it also reduces the file size. In this paper, the image is resized to facilitate the process of feature extraction. All pictures used resized to the same x-dimension and y-dimension of (150*150) respectively.

2.2 Convert a color image to grayscale

An RGB image is also known as a color image. The pixels of a color image are determined by three values with each of the RGB (Red, Blue, and Green) components having one of the pixel scalar value. An M-by-N-by-3 array belonging to uint16, uint8, single, or double class whose pixel values defines the value of the intensity. For double or single arrays, the values range from 0 to 1 while uint8 values are within the range of 0 to 255 and class uint16 values range from 0 to 65535.  It is also known as an intensity, grayscale, or gray level image. For the int16 class, values range from [-32768, 32767] [15, 16].

An image gray level denotes the quantization interval number in grayscale image processing. Presently, the most frequently used storage technique is 8-bit storage. An 8-bit grayscale image comprises of a number of gray levels (256) where Each pixel intensity varies from 0 to 255, with 0 representing black and 255 representing white. Another storage method that commonly used is 1-bit storage. This method consists of two gray levels, with which 0 is black, and 1 is white, and is mostly used in bio-medical images and referred to as binary image.  Due to the fact that it is easy to work with binary images, images of other formats are mostly converted to binary format when used for edge detection or recognition, enhancement and so on [3]. In this study, RGB images were converted to grayscale, especially in traffic sign images because of weather fluctuation which might make the extraction of image features difficult. Furthermore, memory size reduced. There are several methods to convert an image to grayscale, but this work used a simple method which is a common method as shown in the equation below where the average value of the three colors (RGB) taken. Therefore, for a color image, the value of r is added to that of g and b then divided by 3 in order to achieve the desired image in grayscale format.

$Grayscale=\left( R+G+\frac{B}{3} \right)$                            (1)

2.3 Histogram of Oriented Gradients (HOG)

This is a feature descriptor that is used to find target objects in image processing and computer vision. The HOG descriptor counts the incidences of gradient orientation in the area localized in an image such as the Region of Interest (ROI) [17]. The implementation of HOG executed in some steps. Firstly, the image divided into small regions that are connected called cells. For every cell, HOG edge orientations or directions for pixels in the cell computed. After that, an individual cell discretized into angular bins in accordance with the orientation of the gradient. The pixel for individual cell delivers a weighted gradient to its matching angular bin. Moreover, sets of adjacent cells treated as blocks which are spatial regions. The grouping of cells into blocks is fundamental for the grouping and normalization of histograms. Lastly, histogram groups are normalized and denote the block histogram as shown in Figure 4.

image006.png

Figure 4​. The process of HOG

Practically, vertical and horizontal gradients are first calculated. The image split to cells of dimension 8×8, each of which HOG is derived. One of the major reasons feature descriptors are used to describe an image patch is because it provides a compact representation. The patch gradient comprises of two values (direction and magnitude) per pixel summing up to 8x8x2 = 128 numbers, to construct histogram of gradients in these 8×8 cells. This technique requires filtering the data intensity or color of the image with the following filter kernels as shown in Figure 5 and the HOG parameter listed in Table 1. To calculate the magnitude and direction of the gradient using the following formula

$\text{g}=\sqrt{\text{g}_{\text{x}}^{2}+\text{g}_{\text{y}}^{2}}$                  (2)

$\text{ }\!\!\theta\!\!\text{ }=\text{arctan}\frac{{{\text{g}}_{\text{y}}}}{{{\text{g}}_{\text{x}}}}$                 (3)

image007.png

Figure 5​. Training images for traffic sign recognition

Table 1. Configuration of HOG parameters in feature extraction

2.4 Classification

Image classification algorithms used since the early 1970s when the information on land observation became accessible. Nearly all these algorithms range from the methods of visual interpretation to advanced machine learning algorithms. In this section, comparison between three algorithms done, and lastly, a voting algorithm is used to combine those three algorithms in order to enhance accuracy as shown in Figure 6.

image008.png

Figure 6​. Initialization of three exemplary classifiers for soft voting

2.4.1 Support Vector Machine (SVM)

Support Vector Machine (SVM) constructs a hyperplane or a collection of hyperplanes in a large or infinite-dimensional space applicable for various applications, including regression and classification. Intuitively, the hyper-plane which possesses the greatest distance to the closest training data nodes of any object class achieves a good separation which is referred to as functional margin. In general, the more the functional margin, the lesser the classifier’s generalization [18]. The SVM classifier defines as:

$min\frac{1}{2}{{w}^{T}}w+C\underset{i=1}{\overset{n}{\mathop \sum }}\,{{\zeta }_{i}}$                    (4)

${{y}_{i}}\left( {{w}^{T}}\phi \left( {{x}_{i}} \right)+b \right)\ge 1-{{\zeta }_{i}},$                         (5)

where, ${{\text{ }\!\!\zeta\!\!\text{ }}_{\text{i}}}\ge 0,\text{i}=1,\ldots .,\text{n}$,  C > 0 is the upper bound, and (X, Y) stands for a group of feature vectors and a set of class labels in the training dataset respectively. W denotes the weight vector for learned decision hyperplane while b referred to as the model bias. ${{\zeta }_{i}}$ are the slack variables that demonstrate the distance of a particular instance from its proper side of the decision boundary in a geometric viewpoint, and it is a non-zero value for cases violating the constraint $\text{yi }\!\!~\!\!\text{ }\left( \text{w}.\text{xi }\!\!~\!\!\text{ }+\text{ }\!\!~\!\!\text{ b} \right)\text{ }\!\!~\!\!\text{ }\ge \text{ }\!\!~\!\!\text{ }1$. The C parameter is a trade-off amid minimization of error and maximization of margin, with a larger value of C putting more concentration on error minimization.

In this system, an SVM trained with Radial Basis Function (RBF) kernel, where two parameters C and gamma must be put into consideration. The parameter C, which is prevalent to all kernels in SVM, trades-off the misclassification of training samples against decision surface simplicity. A high C aims to correctly classify all training examples, while a low C smoothens the decision surface. Gamma defines the degree of influence of a sole training example. The bigger gamma is the nearer other samples must be for them to be affected. Where an appropriate choice of parameter gamma and C is significant to the performance of the SVM. To compute the RBF kernel between two vectors. The kernel defines as equation below.

$k\left( x,y \right)={{e}^{-\gamma {{\left| \left| y-x \right| \right|}^{2}}}}$                     (6)

where, the x and y are the input vector and $\text{ }\!\!\gamma\!\!\text{ }$ is a slope, most of the time, the data will be composed of n vectors $\text{xi},$ for each xi will be related with a value$\text{ }\!\!~\!\!\text{ yi}$ showing whether the element belongs to the class (+1) or not (-1) as in Figure 7. Yi can only have two possible values -1 or +1. Moreover, most of the time, for instance when the text needs to be classified, the vector xi ends up having a lot of dimensions.$\text{ }\!\!~\!\!\text{ }xi$ is a p-dimensional vector provided it has p-dimensions as in Table 2. So, a dataset D is the set of n couples of elements $xi,yi$. Thus, the formal definition of an initial dataset in set theory is as follows:

$D=\left\{ \left( xi,yi \right)xi\in {{\mathbb{R}}^{p}},yi\in \left\{ -1,1 \right\} \right\}~{{i}^{2}}=1$                     (7)

image009.png

Figure 7​. Data separated by an RBF kernel

Table 2. SVM parameter

2.4.2 Decision Trees (DTs)

Decision Trees is a method of supervised learning with no parameters which used for regression and classification [19, 20]. The aim of the Decision Tree is to create a model that guesses target variable values by studying simple decision guidelines that were deduced from the data features which can manage both categorical and numerical data and also requires little data preparation. Furthermore, it is adept in identifying multiple classes on a given dataset. Given the data at node m is denoted by $\text{Q}$, for any candidate split $\text{ }\!\!\theta\!\!\text{ }=\left( \text{j},\text{tm} \right)$ containing threshold $\text{tm}$ and a feature$\text{ }\!\!~\!\!\text{ j}$, segregate the data into $\text{Qleft}\left( \text{ }\!\!\theta\!\!\text{ } \right)$ and $\text{Qright}\left( \text{ }\!\!\theta\!\!\text{ } \right)$ subsets.

 $Qleft\left( \theta  \right)=\left( x,y \right)|{{x}_{j}}\le {{t}_{m}}$                      (8)

$Qright\left( \theta  \right)=Q\backslash Qleft\left( \theta  \right)~$                    (9)

2.4.3 K-Nearest Neighbour (KNN)

K-Nearest Neighbour is a well-known and often utilized algorithm in machine learning. Classification is performed based on the proximity of a chosen feature to the nearest one. The K value enables us to ascertain the volume of data to be evaluated based on the extent of data the categorization will generate. Eq. (11) delineates the distances between objects. Where the value of the n-neighbour is 10. Furthermore, the goal of KNN is to learn an optimal linear transformation matrix of size (components, features), This, as shown in the Eq. (10), maximizes the total across all samples i of an opportunity pi that i is properly identified.

$\begin{matrix}  argmax  \\   L  \\\end{matrix}\underset{i=0}{\overset{N-1}{\mathop \sum }}\,pi$                     (10)

$d\left( i,j \right)=\sqrt{\underset{k=1}{\overset{p}{\mathop \sum }}\,{{({{X}_{ik}}-{{X}_{jk}})}^{2}}}$                                       (11)

2.5 Voting classifier

Conceptually combine diverse machine learning-based classifiers and use the mean predicted probabilities soft vote or majority vote to foretell the class labels. Hence, this classifier is suitable for a set of evenly well-performing models to stabilize their weaknesses. The class label returned as $\text{argmax}$ of the total predicted probabilities in soft voting. Figure 8 and Figure 9 indicate the voting steps.

image010.png

Figure 8​. Main steps of voting classifier

image011.png

Figure 9​. The voting classifier process

Certain weights could be allocated to every classifier via the weight’s parameter. As the weights allocated, the probabilities of the envisioned class for every classifier are collected, then multiplied by the weight of the classifier and lastly, the average is determined. Weights were assigned proportionally to each classifier’s standalone accuracy on the validation set. The resulting class label is obtained from the class label having the largest average probability. Three classifiers taken with three-class classification problems where several values of weights assigned to all classifiers. The class with the highest average probability chosen as the predicted class label.

2.5.1 Hard voting

In hard voting, they assign the final class label as the class label most commonly expected by classification models.  Hard voting is the easiest vote-majority scenario. Which predicts the class tag $\hat{y}$ by majority vote (plurality) of each Classifier ${{C}_{j}}$.

 $\hat{y}=mode\left\{ {{c}_{1}}\left( x \right),{{c}_{2}}\left( x \right),\ldots .,{{c}_{m}}\left( x \right) \right\}$                    (12)

Assuming that three classifiers classify a training sample as below:

• SVM -> class 0
• KNN -> class 0
•  DTs  -> class 1

$\hat{y}=mode\left\{ 0~,0~,1 \right\}=0$                    (13)

Through majority vote, labels the sample as "class 0." Besides the simple majority vote (hard vote), as mentioned in the previous section, we can measure a weighted majority vote by adding a weight ${{\text{w}}_{\text{j}}}$ with classifier ${{\text{C}}_{\text{j}}}$:

$\hat{y}={{i}^{argmax}}\underset{j=1}{\overset{m}{\mathop \sum }}\,{{w}_{j}}{{x}_{A}}\left( {{C}_{j}}\left( x \right)=i \right) $                (14)

where, ${{\text{x}}_{\text{A}}}$ is the function characteristic [${{\text{C}}_{\text{j}}}\left( \text{x} \right)=\text{i}\in A$, and A is a specific class set labels. Assignment of weights {w1, w2, w3} which yields a prediction

 ${{i}^{argmax}}\left[ w1*i0+w2*i0+w3*i1 \right]=1$                     (15)

In this paper the method of hard voting is determined by the above equation, the SVM given highest weight because of its high accuracy, the remain values gives to KNN and Decision Tree.

2.5.2 Soft voting

In soft voting, class labels are determined depending on the predicted classifier p parameters; this method is recommended only if the classifiers are good-calibrated.

$\hat{y}={{i}^{\arg max~}}\underset{j=1}{\overset{m}{\mathop \sum }}\,{{w}_{j}}{{p}_{ij}}$                   (16)

where, Wj is the weight allocated to the j. By using the weight, the average probabilities can determine as in equation below.

$p\left( i0|x \right)=~\sum classifie{{r}_{acc}}*wj$                         (17)

$p\left( i1\mid x \right)=~\sum \left( 1-classifie{{r}_{acc}} \right)wj$                (18)

However, the weights {w1, w2, w3} which yield a prediction $\hat{y}$ =1.

$\hat{y}={{i}^{argmax}}\left[ p\left( i0x \right)+p\left( i1\mid x \right) \right]=1$               (19)

The vote classifier used in this paper to enhance the accuracy of Chinese traffic sign recognition which used different classifiers, SVM, KNN, Decision Tree. The ensemble classifier depends on the averages of the classifier accuracy and its probability.