Comparative Analysis of Classification of Dry Nut Types Using the Support Vector Machine and Linear Discriminant Analysis Methods

Nuts are an important product in the agricultural and food industry throughout the world where they contain vegetable protein, fibre and other important nutrients for the human diet. Each type of nut has its characteristics, so it is very important to identify and classify the type of nut. Accurate classification is crucial for minimizing post-harvest damage, improving processing efficiency, and ensuring high-quality products for commercialization. By selecting and classifying peanut samples based on attributes like size and colour, producers can enhance the marketability and consumer acceptance of these products [1].

The main objective of this research is to compare the effectiveness of two machine learning algorithms, Support Vector Machine (SVM) and Linear Discriminant Analysis (LDA) in classifying various types of dry nuts. The motivation behind using these algorithms stems from their proven performance in other fields such as image processing [2, 3] medical diagnostics [4, 5] and financial forecasting [6, 7]. However, there has been limited exploration of their application in the agricultural domain, particularly for the classification of nuts. This research aims to address that gap by analyzing their accuracy and suitability for this specific task.

The SVM method is a classification method used to separate two data classes where the SVM method works by finding the largest margin between different data classes which is determined from the distance between the dividing line and the closest point of the two classes [8]. The SVM method aims to find the line separator that has the largest margin, so this makes it better at abstracting unknown data.

The way SVM works starts by visualizing each data point in dimensional space according to the features in the data. Then, a decision boundary or hyperplane will be formed which is used as a decision boundary that maximizes the distance or margin. The point that is closest between these classes is called the support vector. SVM projects data into dimensional space using a kernel function where this kernel function can help the SVM in creating an effective hyperplane. The way to determine the best hyperplane is to look for ideal lines that are divided into these two classes where the lines are not close to each other and are far from the two points. In other words, the line must be in the middle of both classes.

Several examples of the application of the SVM method which can be used in various application fields such as image classification [9], traffic casualties [10], genome classification [11], financial analysis [12], detecting cyberbullying on social media [13], identifying black-hole attacks to enhance Mobile Ad Hoc Network (MANET) security [14], and others. SVM has been previously used to predict the moisture content of Macadamia nut-in-shell [15].

LDA is a classification method used to separate two or more groups of data. LDA works by finding a linear combination of features that maximizes the differences between groups of data, making it easier to classify new data [16]. The way LDA works starts by calculating the mean, calculate the covariance matrix each class, calculate the in-class covariance matrix, and calculate the formation of linear discriminant functions.

Several examples of the application of the LDA method which can be used in various application fields such as facial recognition [17], disease detection [18], and others. LDA has also been employed in research to identify shelled and unshelled walnuts. LDA has been applied in research aimed at classifying shelled and unshelled walnuts [19].

Apart from the SVM and LDA method, there are several other methods that can be used to classify types of nuts, namely Decision Tree [20], Naïve Bayes [21], Artificial Neural Network (ANN) [22], and Logistic Regression [23]. Each of these methods needs to consider the type of data characteristics and the desired analysis objectives. The reason the researcher did not use this method was that there were shortcomings in each of these methods and they were not very suitable for this research, so the researcher decided to use the SVM and LDA methods in this research. This can be seen from the shortcomings of Naïve Bayes, namely that it lies in the predictors of the independent variables, which can slow down the classification performance [24], Random Forest also has a disadvantage, namely that it is difficult to interpret the influence of each feature in decision making [25], Logistic Regression also has a disadvantage, namely that it is sensitive to underfitting in datasets whose classes are not balanced, which will cause low accuracy values [26].

In this research, the dataset was obtained from the UCI Machine Learning Repository website through this link, https://archive.ics.uci.edu/datasets. This dataset contains 17 variables listed in Table 2.

Table 2. Table of variables in the dataset [27]

This dataset contains seven types of dry nut used in this research with data totalling 13, 611 considering the type, structure, shape and characteristics. This data will be used in developing a classification model that can differentiate between different types of dry nuts. Each type of dry nut is photographed using a high-resolution camera and will go through a segmentation process along with feature extraction including 12 dimensions and 4 types of shapes that describe the geometric characteristics of the dry nut. This data was provided to the UCI Machine Learning Repository on September 13, 2020 [29].

3.1 Load dataset

The dataset is provided in .xlsx format. After downloading, it can be uploaded to the notebook by dragging and dropping the file. Once the upload is complete, the dataset becomes accessible for further processing.

This dataset comprises 13,611 records and includes 17 columns or variables. These variables represent various attributes, as illustrated in Table 3.

Table 3. Dry bean dataset

Table 3. Dry bean dataset (Continue)

3.2 Cleaning data

From the results of checking the dataset, it can be said that there are no null values in each column value, as shown in Table 4.

Table 4. Null value checking result [27]

3.3 Exploratory data analysis

First, the results of the data type for each column, as shown in the Table 5.

Then, we attach the histogram image of each column in the dataset apart from the Class column, as shown in the Figure 2.

There is information obtained, namely that the ShapeFactor1, ShapeFactor2, ShapeFactor3, AspectRation and Compactness columns have normal distribution values which have symmetrical skewness. This indicates that the data is spread evenly without a strong tendency in a particular direction. This means that the mean, median and mode have the same value and the data is easy to analyze using common statistical methods. The result is shown in the Figure 3.

Table 5. Data type of each column [27]

The positively skewed distribution information obtained is that the Area, Perimeter, ConvexArea, EquivDiameter, MajorAxisLength and MinorAxisLength columns have positive skewness. This indicates that the data distribution tends to the right because the average value is higher in the right direction of the data distribution. This means that the data is not normally distributed and the average value is greater than the median and mode, so it has to be careful when using general statistical methods.

There is information obtained, namely that the Eccentricy, Extent, Solidity, Roundness and ShapeFactor4 columns have negative skewness. This indicates that the data distribution tends to the left because the average value is higher in the left direction of the data distribution. The data is not normally distributed, and the mean value is smaller than the median and mode, so it has to be careful when using general statistical methods (Figure 4).

2a.png

2b.png

2c.png

Figure 2. Histogram ShapeFactor1, ShapeFactor2, ShapeFactor3, AspectRation, Compactness

3a.png

3b.png

3c.png

Figure 3. Positively skewed distribution

4a.png

4v.png

4c.png

Figure 4. Histogram eccentricity, extent, solidity, roundness, ShapeFactor4

3.4 Class imbalance assessment

The 'Bombay' class had the highest representation, while other classes, such as 'Sira', were underrepresented. To address this imbalance, we applied techniques such as random oversampling of the minority classes and under sampling of the majority class during the training process. This method aimed to create a more balanced training dataset, ensuring that the classifiers would not be biased towards the more prevalent classes. Additionally, we monitored the performance metrics, such as precision, recall, and F1-score, for each class to evaluate how well the models performed across all nut types.

3.5 Training data using SVM

The training data and testing data were separated by 80% and 20%. A model will be created using the SVM method, as shown in the Figure 5.

5.png

Figure 5. SVM model

After the separation was carried out, the data that would be trained to form the model was 10,888 and the data used for testing was 2723. The SVM model was trained using the Radial Basis Function (RBF) kernel, which is particularly effective for non-linear classification problems. To optimize the model, we employed hyperparameter tuning using Grid Search to find the best values for the regularization parameter (C) and the kernel coefficient (gamma). Specifically, we tested C values in the range of 0.1 to 10 and gamma values from 0.01 to 1. The best combination was selected based on cross-validation accuracy, ensuring that the model was well-tuned to the dataset. Additionally, a stratified k-fold cross-validation with 10 folds was implemented to maintain the class distribution in each fold, providing a robust assessment of the model's performance.

3.6 Testing data using SVM

At this stage, testing data uses a model that has been trained in the data training process but uses data that has never been used at all, as shown in the Figure 6.

6.png

Figure 6. Prediction result of testing data

3.7 Training data using LDA

At this stage, a model will be created using the LDA method, as shown in the Figure 7.

7.png

Figure 7. LDA model

For the LDA model, the training process involved calculating the mean vectors and covariance matrices for each class in the training dataset. Prior to training, we confirmed that LDA assumptions, such as data normality and equal class covariance, were met using exploratory data analysis. Additionally, the number of projection components was set to the standard value of one less than the total number of classes, optimizing the model's ability to distinguish between classes. The LDA model was tested using the same 80/20 training and testing split as the SVM model, ensuring a consistent and fair comparison between the two approaches.

3.8 Testing data using LDA

At this stage, testing data uses a model that has been trained in the data training process but uses data that has never been used at all, as shown in the Figure 8.

8.png

Figure 8. Prediction result of LDA model

3.9 Discussion

To ensure the reliability and stability of the results, multiple experiments were performed using both the SVM and LDA models. Each model was trained and tested on 10 different random splits of the dataset, and the average accuracy along with other performance metrics was computed across these iterations to ensure a comprehensive evaluation.

Additionally, we analyze feature importance to identify the key characteristics of dry nuts that contributed to accurate classification. For the SVM model, the feature contributions were assessed using the coefficients of the decision boundary, while for the LDA model, we examined the weights assigned to features in the linear discriminant functions. The results showed that features such as 'Area', 'Perimeter', and 'MajorAxisLength' were the most significant in distinguishing between different nut types. This indicates that geometric properties play a critical role in classification. Highlighting feature importance not only improves the interpretability of the models but also offers valuable insights for enhancing data collection and feature engineering in future studies.

3.10 Confusion matrix

The final step is to measure the accuracy of the model that has been developed using the confusion matrix. The formula for calculating accuracy with the confusion matrix is as follows: Accuracy = $\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}} * 100 \%$ [30].

The results are shown in Figures 9-22.

9.png

Figure 9. The result of class Barbunya SVM model

$\begin{gathered}\text { Accuracy }=\frac{240+2445}{240+2445+17+21} * 100 \% \\ \text { Accuracy }=98 \%\end{gathered}$

10.png

Figure 10. The result of class Bombay SVM model

$\begin{gathered}\text { Accuracy }=\frac{117+2606}{117+2606+0+0} * 100 \% \\ \text { Accuracy }=100 \%\end{gathered}$

11.png

Figure 11. The result of class Cali SVM model

$\begin{gathered}\text { Accuracy }=\frac{300+2387}{300+2387+19+17} * 100 \% \\ \text { Accuracy }=98 \%\end{gathered}$

12.png

Figure 12. The result of class Dermason SVM model

$\begin{gathered}\text { Accuracy }=\frac{621+1994}{621+1994+58+50} * 100 \% \\ \text { Accuracy }=96 \%\end{gathered}$

13.png

Figure 13. The result of class Horoz SVM model

$\begin{gathered}\text { Accuracy }=\frac{391+2306}{391+2306+9+17} * 100 \% \\ \text { Accuracy }=99 \%\end{gathered}$

14.png

Figure 14. The result of class Seker SVM model

$\begin{gathered}\text { Accuracy }=\frac{393+2298}{393+2298+12+20} * 100 \% \\ \text { Accuracy }=98 \%\end{gathered}$

15.png

Figure 15. The result of class Sira SVM model

$\begin{gathered}\text { Accuracy }=\frac{481+2122}{481+2122+65+55} * 100 \% \\ \text { Accuracy }=95 \%\end{gathered}$

16.png

Figure 16. The result of class Barbunya LDA model

$\begin{gathered}\text { Accuracy }=\frac{217+2453}{240+2445+9+44} * 100 \% \\ \text { Accuracy }=98 \%\end{gathered}$

17.png

Figure 17. The result of class Bombay LDA model

$\begin{gathered}\text { Accuracy }=\frac{117+2606}{117+2606+0+0} * 100 \% \\ \text { Accuracy }=100 \%\end{gathered}$

18.png

Figure 18. The result of class Cali LDA model

$\begin{gathered}\text { Accuracy }=\frac{302+2375}{302+2375+31+15} * 100 \% \\ \text { Accuracy }=98 \%\end{gathered}$

19.png

Figure 19. The result of class Dermason LDA model

$\begin{gathered}\text { Accuracy }=\frac{570+2022}{570+2022+30+101} * 100 \% \\ \text { Accuracy }=95 \%\end{gathered}$

20.png

Figure 20. The result of class Horoz LDA model

$\begin{gathered}\text { Accuracy }=\frac{381+2309}{381+2309+6+27} * 100 \% \\ \text { Accuracy }=98 \%\end{gathered}$

21.png

Figure 21. The result of class Seker LDA model

$\begin{gathered}\text { Accuracy }=\frac{383+2303}{383+2303+7+30} * 100 \% \\ \text { Accuracy }=98 \%\end{gathered}$

22.png

Figure 22. The result of class Sira LDA model

$\begin{gathered}\text { Accuracy }=\frac{508+2025}{508+2025+162+28} * 100 \% \\ \text { Accuracy }=93 \%\end{gathered}$

Following the evaluation of the confusion matrix, we conducted a detailed analysis of misclassifications to understand which types of nuts were most frequently confused by the models. The confusion matrices for both SVM and LDA indicated that certain nut types, such as 'Barbunya' and 'Dermason', were often misclassified as 'Seker'. This pattern suggests that these nuts may exhibit similar geometric characteristics, leading to confusion during classification. Hypothetically, factors such as slight variations in colour, shape, or size, which may not be adequately captured in the features used for modeling, could contribute to these misclassifications. Additionally, the similarities in texture or visual features between these nut types may also play a role in their frequent confusion.

In our analysis of misclassified samples, we observed that the misclassifications were not uniformly distributed across all classes. For example, a notable number of samples from the 'Sira' class were misclassified as 'Dermason'. This could be attributed to the similarity in their geometric features and possibly their size. Moreover, the confusion matrices showed that the model struggled to accurately classify samples from classes with fewer instances, such as 'Sira', leading to higher misclassification rates for these types. This highlights the impact of class imbalance on model performance, where models may not learn adequately about underrepresented classes. Future work could focus on exploring advanced techniques like synthetic data generation for minority classes to mitigate these issues.