Medicinal Plants Recognition Using Deep Transfer Learning Models

The rapid progress in technology, specifically artificial intelligence, has made exploiting technology to automatically identify and classify medicinal plants essential considering their importance in protecting human health and preventing disease. The medicinal plants contain natural compounds, phytochemicals, vitamins, antioxidants, and minerals used as important components in pharmaceutical manufacturing [1, 2].

The number of plants in the world is approximately 350,000 species [3], varying in size, shape, and leaves, which makes the conventional identification is error-prone, slow, exhausting and needs an experienced specialist while plant identification through deep learning can be a method to reduce human errors and produce accurate results [4].

Musyaffa et al. [5] carried out study on identification of medicinal plants, used three transfer learning models: ResNet34 [6], DenseNet121 [7], VGG11_bn [8] and scratch model on a dataset of their made which is Indonesia Medicinal Plant dataset. The best accuracy achieved was 100% by DenseNet121 with a validation accuracy 87.4% then ResNet34 accuracy 99.82% and validation 85.65%, VGG11_bn accuracy was 96.33% and validation accuracy was 82% and the scratch model accuracy was 65.97% with validation 43.53%. By observing their work, it was noticed that the methods used had a high training accuracy with significantly lower validation accuracy which might indicate overfitting with complexity in terms of model and learning. Also, they didn't show their methods effectiveness by other performance metrices such as precision, recall and F1-score. Therefore, this study explores other transfer learning models in order to achieve more reliable results.

Deep learning models especially transfer learning in the current time is highly popular and widely utilized in many fields [9], seed Taxonomy by transfer learning [10], in the medical field, for example, it was used as a tool for automatic diagnosis of the COVID-19 [11-13] also used to build intelligent system aids the blind people to distinguish the objects around them [14] and so many applications. In the field of construction there are applications in which deep learning plays an important role as demand forecasting tool of industrial companies [15]. One of the widely used applications of deep learning in other fields is fingerprint recognition [16].

Below is a summary of our contributions of this study:

1. Evaluation of various transfer learning models to create an automated framework for medicinal plant recognition.
2. In order to improve the model performance, the hyperparameter of the suggested models are systematically modified and fine tuned.

The following of study is divided as follows: Section 2 reviews the related works. Section 3 expresses methodology and proposed method, Section 4 presents experimental results. Finally, the conclusion is discussed in the last section.

In this study, a transfer learning method is proposed for utilizing pre-trained models, trained on a large datasets. Thus, the cost and time of training new deep learning models are reduced. As shown in Figure 1, six steps are implemented in the workflow. At first, selecting a suitable dataset and the choice fell on Indonesia Medicinal Plant dataset [5] after that apply preprocessing techniques on the dataset, splitting it into 60% training data and 40% validation data. Then applying augmentation processes such as rotation, translation and zoom. The next step is the most important step which is selecting a pretrained model. The pretrained model is automatically extracts the features from the training data then validates and tests the validation data. Finally, competitive the best hyperparameters are selected by manual comprehensive experiments. The selected hyperparameters are displayed in Table 2 below.

Eventually, the medicinal plant will be recognized, and the performance will be measured by precision, f1 score, recall and accuracy metrics.

image003.png

Figure 1. Block diagram of the proposed classification process

Table 2. Hyperparameter set (same for all 6 models)

3.1 Dataset description

Indonesia Medicinal Plant Dataset introduced by Musyaffa et al. [5] is balanced dataset have been utilized to train and evaluate the proposed model. This dataset contains 10,000 images with a total 100 classes of herbal plant in Indonesia, each class has a total 100 images most of them are collected from google images search. And the others resulted from augmentation processes such as horizontal flip, vertical flip for the sake of meet the criteria for each class. All the images are 128×128 pixels. Figure 2 represents six classes of this dataset. Indonesia Medicinal Plant dataset comprises images of various medicinal plants native to Indonesia, captured under different conditions and angles to ensure variability and robustness in the training process. This variability in imaging conditions helps to mimic real-world scenarios, enhancing the model's ability to generalize to new, unseen data.

image004.png

image005.png

image006.png

Figure 2. Four images selected from six classes of Indonesia medicinal plant dataset

3.2 Model description

The study investigated six deep learning models; Bit_s-r50x1 [23], Inception_v3 [24], Inception_ResNet_v2 [25], Nasnet_large [26, 27], ResNetv1_152 [6], mobilenet_v2_130_224 [28]. These models were selected based on their proven performance in image recognition tasks and their distinct architectural advantages.

3.2.1 Bit_s-r50x1

A pre-trained deep learning model developed by Kolesnikov et al. [23]. the notation "Bit_s" denotes Big Transfer family-s, while "r50x1" denotes ResNet-50x1, which indicates that it is a ResNet-50 architecture has been pre-trained on ImageNet-1k. Bit_s-r50x1 is known for its robust performance on a variety of image classification tasks due to its extensive pre-training on large datasets. It was chosen for its ability to generalize well to new tasks with limited fine-tuning. Figure 3 shows the architecture of BiT-Small model.

tu_pian_3.png

Figure 3. BiT-Small (BiT-S) architecture [23]

3.2.2 Inception v3

Inception-v3 is part of inception family, it is a convolutional neural network (CNN) architecture Introduced by Szegedy et al. [24] for many different computer vision applications, such as object identification, image segmentation, and image classification, Inception-v3 has been extensively utilized. On benchmarks like the ImageNet Large Scale Visual Recognition Challenge (ILSVRC), it has demonstrated state-of-the-art performance. Inception v3 architecture illustrated in Figure 4, consisting of 42 layers.

tu_pian_4.png

Figure 4. Inception v3 architecture [29]

3.2.3 Inception_ResNet_v2

The Inception module and residual connections and benefits were combined in the Inception-ResNet-v2 deep convolutional neural network architecture. Introduced by Szegedy et al. [25], resulting in a powerful deep neural network that achieves state-of-the-art performance on various computer vision tasks, including image classification, object detection, and image segmentation. Inception-ResNet-v2 has the same schema of Inception-ResNet-v1 but differ in the underlying components, their schema is displayed in Figure 5.

tu_pian_5.png

Figure 5. Inception-ResNet-v1 and Inception-ResNet-v2 networks diagram [25]

3.2.4 Nasnet_large

NASNet is a family of CNN for image classification. The architecture of its convolutional layers has been found by Neural Architecture Search (NAS). NASNets by studies [26, 27] comes in various sizes, nasnet_large of NASNet-A for ImageNet, which begins with 168 convolutional filters and employs 18 Normal Cells as shown in NASNet-A architecture in Figure 6. Nasnet_large is an architecture designed using Neural Architecture Search (NAS), which optimizes the network structure for better performance. It was included to evaluate if this automated design process could outperform manually designed models.

tu_pian_6.png

Figure 6. NASNet architecture [27]

3.2.5 ResNet_v1_152

A deep convolutional neural network design called ResNet_v1_152, for Residual Network version 1 with 152 layers, was carried out by He et al. [6]. It belongs to the ResNet family, which developed residual learning as a solution to the very deep neural network degradation issue. It was known for its exceptional performance in deep learning tasks due to its residual connections that mitigate the vanishing gradient problem. This ResNet_v1_152 block consists of multiple residual blocks. Each residual block contains a series of convolutional layers, followed by batch normalization and ReLU activation functions as shown in Figure 7.

image011.png

Figure 7. Block for ResNet-50/101/152 [6]

3.2.6 MobileNetV2_130_224

MobileNetV2_130_224 is a special version of the MobileNetV2 architecture. Developed by Sandler et al. [28] for efficient image classification tasks. "130" in MobileNetV2_130_224 refers to the width multiplier, which controls the number of channels in each layer of the network and "224" denotes the input resolution of the network. It is a lightweight and efficient solution suitable for deployment in resource-constrained environments such as mobile devices, IoT devices, and edge computing platforms.

MobileNetV2_130_224 architecture achieves a good balance between efficiency and accuracy by employing depthwise separable convolutions, inverted residual blocks, and skip connections. Its lightweight architecture was included to explore the feasibility of deploying plant recognition models on resource-constrained devices. Figure 8 displayed MobilenetV2 architecture.

The input size of Bit_s-r50x1, Inception v3, ResNet_v1_152 and MobileNetV2_130_224 models was 224×224 pixels. While Inception_ResNet_v2 was 299×299 pixels and NASNets has an input size of 331×331 pixels.

image012.png

Figure 8. MobilenetV2 architecture [28]

The power of plants to sustain life is crucial, especially medicinal plants. In this study, six pretrained models were experimented and utilized to identify medicinal plants: Bit_s-r50x1, Inception_v3, Inception_ResNet_v2, Nasnet_large, ResNetv1_152 and mobilenet_v2_130_224. The experimental findings exhibit superior performance over the models were used in previous research on Indonesia Medicinal Plant dataset. Nasnet_large model achieved the highest validation accuracy, the model achieved 92.45% outperformed the highest validation accuracy of the earlier study by 5.4% afterwards was Bits-r50x1 which showed a percentage of improvement 5.2%. In the future, we plan to apply an alternative model in an attempt to increase the validation accuracy. Simultaneously, the accuracy of some of the most misclassified classes will be increased by adding more images to the dataset belonging to that category, improving feature extraction or using better methods. our study demonstrates that transfer learning models significantly enhance the accuracy of medicinal plant identification. This advancement not only supports the immediate needs of plant cataloging and conservation but also holds promise for long-term impacts. As deep learning models continue to evolve, their integration into more sophisticated and user-friendly applications could revolutionize the field, making accurate medicinal plant identification accessible to a broader audience. This could lead to greater preservation of plant biodiversity and more effective utilization of medicinal plants in healthcare.