Genetic Algorithm Based Ensemble of Deep Learning Models for Performance Enhancement of Bird Species Classification

Several researchers have attempted to develop techniques for effective classification of bird species. A review of these key contributions is essential for identifying research gaps, addressing the limitations of existing systems, and proposing methodologies to enhance efficiency and effectiveness. In light of this, the following paragraphs summarize the significant works related to appearance-based bird species classification.

2.1 CNN and Vision Transformer (ViT)-based approaches

CNN and ViT-based approaches for bird species classification have been discussed. The following paragraphs present a brief description and critical analysis of these approaches.

Chen et al. [1] proposed a hierarchical gate network for fine-grained visual recognition. HGNet uses the interconnections among hierarchical classes. LSTM like mechanism is used to propagate dependencies among classes, which boosts classification accuracy. Experiments were conducted on datasets such as Stanford Dogs, CUB-200-2011, Aircraft, NABirds, iNaturalists, DeepFashion2 and DeepFashion. The proposed methodology achieved 88.7%, 88%, 92.8%, 86.4%, 78.2%, 58.5%, and 68.4% accuracy for the Stanford Dogs, CUB-200-2011, Aircraft, NABirds, iNaturalists, DeepFashion2 and DeepFashion datasets respectively.

Branson et al. [2] proposed a classification approach based on pose-normalized deep convolutional neural networks (DCNNs). The architecture estimates the pose of a bird to compute the local image features for classification. A graph-based clustering algorithm and higher-order geometric warping functions were used to estimate the normalized bird pose. Experiments conducted on the CUB-200-2011 dataset showed an accuracy of 75.7%.

Jaderberg et al. [3] proposed a spatial transformer network by introducing a learnable module that could be integrated into any CNN architecture with very little modification. When incorporated, this module equips the CNN with the ability to handle image transformations, such as translation, scaling, and rotation. The experiments conducted on the CUB-200-2011 dataset achieved a classification accuracy of 84.1%.

Manna et al. [4] proposed a various convolutional neural network, namely Inception V3, DenseNet 201, MobilenetV2, and Resenet152V2, for bird species classification based on images. The model was trained on 58388 images and tested on 2000 images belonging to 400 species. According to the study, ResNet152V2 achieved the highest accuracy of 95.45% and a loss of 0.8835. Similarly, DenseNet achieved a classification accuracy of 95.0% with a loss of 0.6845.

Zhang et al. [5] proposed a part-based CNN for fine-grained category detection to explicitly identify significant differences in the appearance of specific object parts. Semantic part localization was utilized for fine-grained categorization, with bounding box annotations considered during testing to address the object annotation challenges. The model leveraged convolutional features obtained through bottom-up region proposals, learning both whole-object detection and part detection while enforcing geometric constraints between the foreground and background of the interest. This approach enables fine-grained category detection using post normalized representations. Experiments on the CUB-200-2011 dataset demonstrate a performance of 76%.

Huang et al. [6] proposed a recognition of endemic birds using deep-learning models. A transfer learning-based method using Inception-ResNetV2 was used, which outperformed all the other methods, namely InceptionV3, ResNet101, MobileNetV2, Xception, and ResNet101. A five-fold cross-validation was used to review the results. A total of 790 images of birds consisting of 29 species were used to evaluate the performance of the model. The proposed model achieved accuracies of 100% for bird identification and 98.39% for bird classification.

Wei et al. [7] proposed a mask CNN without fully connected layers, which localizes parts and selects descriptors for fine-grained bird species categorization. The fully convolutional network locates discriminative parts and generates weighted object masks. A three-stem-masked CNN model was used to build the descriptors. The experiment was conducted on the CUB-200-2011 and Birdsnap datasets. The proposed model achieves 85.7% of accuracy for CUB-200-2011 dataset and 77.3% of accuracy for Birdsnap dataset, respectively.

Kumar and Kondaveeti [8] proposed a bird species recognition using a hybrid hyperparameter optimization scheme (HHOS). A few selected models were trained in the HHOS, which enhanced the classification performance. The results showed that EfficientNetB0 achieved superior performance compared to the other classification models, with the highest accuracy of 99.12%. The authors increased the size of the CUB-200-2011 dataset from 11788 to 40000 by adding 29,000 more images, which are the augmented versions of the original dataset.

Ngo et al. [9] proposed a survey of different deep-learning models such as InceptionV3 and EfficientNetB4. The model was evaluated on a wide range of datasets, including CUB -200-2011, Kaggle 325 bird species, and Kaggle-510 bird species, and a self-generated dataset (100 bird species from Malaysia) was used in this study. The EfficientNetB4 model outperformed the models in terms of classification performance. For the CUB-200-2011 dataset, the result was 74% for the EfficientNetB4 model.

Bold et al. [10] proposed a multiple kernel learning framework for bird species classification using a CNN was proposed. A multiple kernel learning technique was used to combine features from audio and video data. The CUB-200-2011 dataset and audio data were combined for classification. 78.15% was the highest performance achieved using the proposed methodology.

Discriminative features for bird species classification were proposed by Pang et al. [11]. First, the images were cropped based on patches. The patches are used to form codebooks, and finally, the code books are used to generate intermediate features based on the sparse coding algorithm. Intermediate features were concatenated to form the final feature representation, which was used for training and classification. The CUB-200-2011 dataset was used in this study. The proposed model has achieved 64.6% of performance.

CNN works best in processing spartial patterns while ViT works best in processing global context and long-range dependencies. Limitations of the ViT are high computational cost, low inductive bias and needs huge data. Whereas limitations of the CNN are high inductive bias, lower computational cost and weak in handling global context.

CNN-based models utilize convolutional kernels that have a local receptive field, meaning they process only a small window of pixels at a time. While stacking multiple layers allows the network to eventually see the entire image, its global understanding remains inefficient and implicit. This limitation can be addressed by transformer-based methods, which use a self-attention mechanism for a more holistic view. However, transformer models are extremely dependent on large amounts of data and require high computational power. Consequently, the choice between a CNN and a ViT involves a trade-off: CNNs offer greater efficiency and lower computational demands, while ViTs deliver higher performance and superior holistic modelling—requires adequate data and significant computational resources as shown in Table 1.

Table 1. Literature review for CNN and ViT based approaches

2.2 Genetic algorithm-based approaches

Genetic algorithm-based approaches for bird species classification were discussed in studies [12, 13]. The following paragraphs presents a brief description and critical analysis about these approaches.

Mousavi and Eftekhari [12] proposed a technique for image classification based on hybridization of classifier ensemble approaches. The model was evaluated using both static and dynamic ensemble methodologies, leveraging NSGA-II as a multi-objective genetic algorithm to optimize both error and diversity for selecting the best classifier ensemble. A Pareto optimal solution is suggested to balance the trade-off between these objectives. The experiments were conducted using six combined rules and 46 individual classifiers. The proposed approach outperformed all other ensemble methods across the 14 datasets in terms of classification accuracy.

Dharmawan et al. [13] proposed a novel waste classification approach that incorporates genetic algorithm hyperparameter optimization with data augmentation and data down sampling. To obtain the best hyperparameter configuration, multiple scenarios were tested with augmentation and different image dimensions. This configuration enhances the accuracy from 94% to 97%.

Advantages of genetic algorithm are it supports global optimization, handles nondifferentiable and complex problems, it is flexible and versatile, it provides a set of good solutions, it is intuitive and inspired by nature.

Limitations of the Genetic algorithm-based approaches are, computationally expensive for training model at each iteration, no guarantee of optimality, problem specific parameter tuning, premature convergence and not suited for all kinds of solutions as shown in Table 2.

Table 2. Literature review of genetic algorithm-based approaches

2.3 Machine learning based approaches

Machine learning-based approaches for bird species classification are discussed in studies [14-19]. The following paragraphs presents a brief description and critical analysis about these approaches.

Kong and Fowlkes [14] proposed a low-rank bilinear pooling method for fine-grained classification. The study revealed that a high-dimensional bilinear feature vector, created by pooling the results of second-order local feature statistics, outperformed contemporary methods across various fine-grained bird classification tasks. To address the complexity of high-dimensional feature spaces, this study proposes covariance features and applies them to a low-rank bilinear classifier. The model was further compressed using a classifier co-decomposition approach, where a collection of bilinear classifiers was factorized into a common factor and compact per class terms. This co-decomposition was implemented using two convolutional layers and was trained in an end-to-end architecture. Additionally, an effective initialization strategy was suggested to avoid the explicit training and factorization of larger bilinear classifiers. Several experiments were conducted on various public datasets to evaluate the efficacy of the proposed methodology, and the results demonstrated a superior performance. The experiment conducted on the CUB-200-2011 dataset achieved the highest accuracy of 84.21%.

Rai et al. [15] proposed various machine learning techniques for bird species classification were proposed. In this study, a CNN model was used for feature extraction, whereas Naive Bayes, SVM, Random Forest, LDA, Decision Tree, KNN, and Logistic Regression were employed for classification. Trail experiments were conducted on the CUB-200-2011 dataset, and it was reported that the KNN classifier achieved the highest classification accuracy of 33%. These studies highlight the progress made in appearance-based bird species classification and underscore the need for further improvements, particularly in handling complex pose variations and enhancing classification accuracy in real-world settings.

Yang et al. [16] proposed an improved transfer learning methodology for the classification of protected Indonesian birds based on images. In this study, batch normalization dropout fully connected (BNDFC) layers were proposed, which can be incorporated into any CNN network to further enhance classification accuracy. This study was conducted using an Indonesian bird dataset. The experiment was conducted using MobilenetV2 CNN and on average, the performance was enhanced by 19.88% accuracy, 24.43% F-measure, 17.93 G-mean, 23.41 of sensitivity, and 18.76 of precision. The highest accuracy achieved was 88.07% for the validation set.

Naranchimeg et al. [17] proposed a multi-model bird species classification for using audio-visual data. CUB-200-2011 was used to build the visual data module, and originally collected audio data were used for building the audio module. The overall classification performance was enhanced by combining audio and video data using a CNN. The model achieved a classification accuracy of 78.9%.

Yang et al. [18] proposed an automatic bird image classification with feature enhancement and contrastive learning. The proposed methodology includes multiscale feature fusion to extract information at different scales. An attention feature enhancement module was integrated to address occlusion and noise. Furthermore, the Siamese network was used to compare two images belonging to the same class and different classes. CUB-200-2011 dataset was used in the study. With only 5% of the training data, the model performed fairly well, with a recognition accuracy of 65.2%. Limitations of the machine learning based approaches, depends mainly on feature engineering, struggles with unstructured data, curse of dimensionality and sensitive to data scaling as shown in Table 3.

Table 3. Literature review of machine learning based approaches

2.4 Optimization based approaches

Optimization-based approaches for classification were discussed in studies [19, 20]. The following paragraphs presents a brief description and critical analysis about these approaches.

Kittler et al. [19] proposed a theoretical framework was for combining multiple classifiers using various combinations of rules to improve pattern classification performance. Each classifier utilizes a unique pattern representation technique to make decisions. Four decision classifiers such as structural, Gaussian, neural network, and hidden Markov model were studied. Accuracy of the proposed classifier ensemble was evaluated through handwritten digit classification, which achieved a classification accuracy of 98.19% with the median combining rule. The approach is computational expensive and suffers from the risk of overfitting and noise sensitivity.

Kumar and Kondaveeti [20] proposed an explainable artificial intelligence to increase the clarity of a deep learning model for bird species classification. Popular deep learning models such as EfficientNetB0, DarkNet53, DenseNet201 SqueezeNet, InceptionResNetV2 and NasNetLarge were trained using CUB-200-2011. The authors have claimed a highest accuracy of 99.51% for EfficientNetB0.

Explanations given by XAI are approximations and incomplete as the single statement rarely tells the complete story. As explanations are filtered through cognitive biases, Explainable models are less accurate, creates false sense of trust and hard to audit, as shown in Table 4.

Table 4. Literature review of other optimization-based techniques

In the present study optimal chromosome has been achieved which outperforms all the individual classifiers. It’s the best combination of the available classifiers, which has enhanced the classification performance. The classifiers have not overfitted because of the availability of the validation split. Genetic algorithm has been best suited in the present scenario.

The ensemble model using genetic algorithm does not overcome the individual limitations, instead it enhances the overall performance of the ensemble.

4.1 Dataset

The Indian bird image dataset was used to conduct experiments and corroborate the performance of the proposed methodology for classifying bird species. The dataset comprises 6150 images splitted into 41 categories, each of which has approximately 150 images. The dataset contained non-segmented bird images. There were significant intra-class variations in the dataset as a single bird was captured at different angles and poses. The dataset is quite complex when compared to available datasets, as the birds are captured with complex backgrounds, bird images are occluded with other objects, and the birds are captured in different poses and angles.

4.2 Experimental setup

Several experiments were conducted to study the performance of the proposed methodology and fine-tune the parameters to enhance the classification accuracy and robustness of the classification approach. Segmented and non-segmented images were studied to understand the performance of the proposed methodology. The following subsections provide details of the parameters used for training the model.

4.2.1 Parameters used for training deep learning models

All the bird images (Figure 3) are resized to size of (224,224,3). A batch size of 32 was used in the experiment. ‘Include top’ parameter was set as False, as the fully connected layers were not included. Stochastic gradient descent (SGD) was used as the optimizer of the model. Accuracy was used as the metric for the optimization of the model. The SGD learning rate was initialized as 0.001, the SGD decay as 1e-6, and the SGD momentum as 0.9. The callbacks of the model were declared with ‘Restore best weights’ as TRUE and with a patience of 5. The loss was categorized as categorical_crossentropy, as it was a multiclass classification problem.

3.png

Figure 3. Examples of images from the Indian bird dataset

4.2.2 Results on different ratios of training and testing data

Several experiments were conducted to study the classification accuracy of the proposed methodology by considering the parameters considered in the previous sections.

Table 7 presents the outcomes of the experiments conducted with different percentages of training and testing for bird images.

Table 7. Classification results for Indian bird images

4.2.3 Parameters used in genetic algorithm

Some parameters associated with the genetic algorithm proposed in this study need to be initialized and fine-tuned during its execution. New chromosomes are generated in each iteration by performing genetic operations, such as mutation and crossover. It has been observed that new chromosomes generated in this manner will be stable after 10 generations. Therefore, the maximum number of iterations required to produce stable chromosomes in this experiment was 10. Thus, the search space computation using a genetic algorithm is equal to the product of the number of generations and population size (10 × 30). As 1536 > 300, the search space for the selection of the best classifier ensemble is reduced using a genetic algorithm when compared with the brute force technique.

The population size should be as large as possible so that a larger diversity of chromosomes can be considered in the experiment. The population size should be as small as possible so that the product of the number of generations and population size is smaller than the computations done by the brute force technique. Hence, the population size is a trade-off between these two conditions. In the experiment, the population size was empirically chosen as 30, as it optimizes the trade-off criteria. The point of crossover is chosen as 1, as it is the standard value used in most genetic algorithm implementations.

The mutation rate should be as large as possible so that diverse solutions are explored with fewer generations. The mutation rate should be chosen as small as possible so that solutions with local optima can be explored. Hence, the mutation rate is empirically chosen as 0.5, as it optimizes the trade-off criteria. Table 8 lists the parameters used in the experiments.

Table 8. Parameters used in the experiments

The parameters used in the genetic algorithm are initialized as mentioned in Table 8. The performance of the individual deep architectures and their ensemble was observed and is presented in Table 9. It can be observed from the results presented in Table 9 that the ensemble of classifiers using the proposed genetic algorithm has enhanced the accuracy of the classification by selectively choosing the right combination of deep classifiers and the right combining rule. Although it is possible to come up with the right combination of deep classifiers with brute force methodology, it is a very cumbersome task, and the complexity of such a technique is quite high. Time complexity can be drastically reduced by applying a genetic algorithm, as discussed.

Table 9. Performance of individual deep architecture classifiers and their ensembles

Figure 4 represents performance of individual deep architecture classifier and their ensembles. Figure 5 represents graph depicting training accuracy and validation accuracy. Figure 6 represents graph depicting training loss and validation loss. Based on Figures 5 and 6, it is easy to analyze that the model is not overfitting and not lacking generatability.

4.png

Figure 4. Accuracies of individual and ensemble techniques

5.png

Figure 5. Graph plotted between accuracy and epochs

6.png

Figure 6. Graph plotted between loss and epochs

4.3 Analysis of fit chromosome

In this context, the fitness of any chromosome is measured in terms of its classification performance by combining the different deep architectures considered for the analysis and the combining rule. The best combination of the deep classifier ensemble and the best combining rules were analysed by conducting experiments and fine-tuning the parameters of the genetic algorithm. Experimental results revealed that the combination of an EfficientNetV2L and ViT with a median combining rule produced the highest accuracy of 97%. The details are presented in Table 10.

Table 10. Best performing chromosomes in terms of the fitness of the genetic algorithm

From the experiments, the best performing top 10 chromosomes were observed and are presented in Table 11. The structure of each chromosome indicates the presence (1) or absence (0) of a particular deep learning model and the combining rules for ensembling. For example, chromosome 10000011101 indicates that the deep learning models considered for ensembling are Xception, EfficientNet and ViT as mentioned in Table 6 and the ‘product’ combining rule as described in Table 5. From Table 11, it can be observed that all the top 10 chromosomes have 7th and 8th bit as one, which indicates that the combination of EfficientNetV2L and ViT has shown a better classification accuracy of 97% when compared to other combinations.

Table 11. Best performing top 10 chromosomes for Indian bird dataset

The top chromosomes work better because they have the highest fitness. The researcher does not have any control over the output of the genetic algorithm, and the genetic algorithm decides the output.

The pattern of the best-performing chromosome states that the combination of ViT and Efficient with Median combining rule is most suited for bird species classification.

Individually, ViT and EfficientNetV2 performed very well. Hence, any combination of these two models performs better. Table 11 presents all the top 10 chromosomes, where all the chromosomes have ViT and EfficientNet.

The performance of the proposed ensemble classification approach was also studied by conducting several experiments and computing class-wise precision, recall, and F-measures, and the results are presented in Table 12. From the results, it was observed that for most of the classes, the results were good, with high precision, recall, and F-measure. However, for a few classes, the results must be improved. One of the reasons for low precision, recall, and F-measure for some of the classes is that the samples for those classes are insufficient, and many more samples with significant intraclass variations need to be considered for training the model. Such an analysis will be conducted in future work.

Species such as Swifts, Prinias, Robins, Plovers, Pigeons, Munias, Nightjars, Flycatchers, Cuckoos, and Doves performed very poorly, with low F1 scores.

Table 12. Class-wise precision, recall and F1-score obtained for the Indian bird dataset using the best classifier ensemble for 90% training, 5% validation, and 5% testing