Divya Anbalagan*
| Sungeetha Dakshinamurthy
© 2024 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
Feeding a growing global population and meeting nutritional needs requires farming. It provides food and clothing reliably. It boosts the economy and employs many, especially in distant areas. Early plant disease diagnosis is crucial to agricultural system sustainability. It provides farmers with the knowledge they require to make informed decisions, which helps to reduce crop losses and contributes to an agriculture industry that is more efficient and sustainable. Many methods have been developed recently to identify and categorize plant diseases at an earlier stage, however it is essential that these methods be as precise and reliable as possible. Deep learning is one of the greatest methods for early detection of plant diseases. The purpose of this study piece is to suggest a revolutionary design that is referred to as the Ensemble ResNet LSTM for Horticulture plants (ERLSTMH). The LSTM network and dropout layer are both incorporated into the ResNet-50 architecture through the process of this suggested design. In order to do an analysis of calculated metrics, such as precision, recall, F1-score, sensitivity, and specificity, the proposed design is utilized. This architecture has provided the maximum level of accuracy. In contrast to all of the earlier structures, the architecture that was recommended was able to attain an exceptionally high level of accuracy, which was 97.147% effective.
image processing, deep learning, ERLSTMH, downy mildew, ergot, rust, DenseNet- 121, ResNet-152
Pearl millet, also known as Pennisetum glaucum, is a type of annual grass crop that is grown during the warm season and is often cultivated in arid and semi-arid parts of Africa and Asia. They have heights that range from two to four meters, making them rather tall. Their leaves are tall and thin, and they are either green or purple in colour. They have a sturdy stem throughout. It is able to absorb water and nutrients from deeper soil layers because to its vast and deep root system, which enables it to thrive in regions that are prone to drought. The inflorescence of pearl millet consist of a panicle, which is a spike-like structure that is compact and cylindrical in shape. Because of the variation, the panicle can be of different lengths and have different densities. As a result of its strong resistance to drought, heat, and poor soil conditions, it is a crop that is able to thrive in locations that are difficult to cultivate. There are millions of people in Africa and Asia who rely on pearl millet as their primary source of nutrition, particularly in areas where other crops have difficulty emerging. This ingredient is utilized in the production of a wide range of culinary items, including fermented drinks, flatbreads, and porridge. Minerals, carbs, protein, and fiber are just some of the nutrients that are abundant in this food. The high levels of iron and zinc that it contains, both of which are essential for human health, are the primary reasons for its high value.
The production of pearl millet in India makes a substantial contribution to the country's efforts to ensure its citizens have access to sufficient food, particularly in areas with difficult meteorological and agricultural circumstances. Pearl millet production can be optimized with the help of regionally relevant instructions and advice from agricultural extension agencies and specialists in the field. The pearl millet plant is vulnerable to a number of illnesses, including those that are caused by fungi, bacteria, and viruses. A number of diseases, including downy mildew, ergot, leaf blight, rust, and smuts, are among the most prevalent diseases that affect pearl millet. More than twenty illnesses have been identified as having an influence on pearl millet, however the incidence of these diseases and the extent of their damage might vary depending on the location and the conditions under which they are grown.
Continuous monitoring of plant diseases is made more difficult by a number of obstacles like, Plant diseases can vary in their symptoms, severity, and spread, which makes it difficult to monitor them continually. Additionally, various diseases may require different monitoring approaches so that they can be monitored effectively. It is necessary to incorporate decision support systems that are able to assess the data and make recommendations that farmers may put into action. In order to do this, the monitoring system needs to be calibrated and validated with great care. The Modified U-Net segmentation model outperforms the U-Net model, while InceptionNet1 achieves high accuracy for binary and multi-class classification. The potential of artificial intelligence in improving disease detection and management in plants, specifically in tomato plants [1]. A software solution for automatic detection and classification of plant leaf diseases, which improves upon a previously proposed solution. The solution consists of four main phases. with two additional steps added after the segmentation phase [2]. Research and breeding efforts are still concentrated on generating superior varieties of pearl millet with improved characteristics such as higher yield potential, resistance to disease, and nutritious content. The purpose of these improvements is to further increase the productivity and profitability of cultivating pearl millet. The development of a segmentation technique for the automatic identification and classification of plant leaf diseases is vital work that should be done so that illnesses can be found early. Extracting features and then using selected features to train support vector machine (SVM) and artificial neural network (ANN) classifiers are both steps in the classification process [3]. The algorithm that was built is able to accurately diagnose 13 distinct forms of plant diseases based on leaf visuals and differentiate between healthy leaves and diseased leaves.
The Caffe framework was utilized in order to accomplish the deep CNN training [4]. The framework leverages a custom-designed model known as "Custom-Net" for disease prediction.
The data is collected automatically from the farmland, sent to the cloud server, and stored on Raspberry Pi. Transfer learning is utilized so that feature extraction can be improved [5]. The accuracy of the traditional manual diagnosis procedures as well as the availability of resources are both limited. The Custom Centre Net framework, with DenseNet-77 serving as the base network, is at the heart of an effective plant disease categorization system [6]. Poor keeping quality and the presence of anti-nutritional elements limit its current application. In order to realize pearl millet's full potential for both farmers and consumers, this review will focus on its nutritional profile, processing methods, health benefits, and obstacles [7].
Pearl millet is sensitive to a wide variety of illnesses and pests, which can be detrimental to both the yields and the quality of the grain. The diagnosis of these illnesses in a timely manner and with precision is very necessary for efficient management and output that is sustainable. One approach that might be taken to solve these difficulties is the method that has been offered, which makes use of imaging techniques and deep learning. In order to improve the early diagnosis and monitoring of illnesses in pearl millet crops, the solution intends to make use of recent technological advancements such as deep learning algorithms and high-resolution imagery. Through the enhancement of disease detection and monitoring, the suggested solution has the potential to assist farmers in the implementation of timely interventions, such as the application of targeted pesticides or modifications to agricultural techniques, in order to reduce the effect of illnesses. An innovative design incorporating LSTM and ResNet-50 networks, as well as dropout layers with a value of 0.2, is introduced in this study. The ideal dropout rate depends on model design, dataset, and training parameters. When aiming for regularization, a low dropout rate is typically a good starting point; if necessary, you may easily increase it. This was done to demonstrate how the combination enhances accuracy, leading to easier and more precise sickness classification.
Contribution of Research:
(1) To offer a new approach to illness classification in horticultural images using deep learning methods.
(2) Disease Classification is carried out using ERLSTMH model.
The following is the article's outline: section 2 discusses related works based on image classification of plant diseases, section 3 explains the preprocessing techniques involved, section 4 explains proposed deep learning technique in image classification, section 5 discusses performance analysis and concludes the research with future scope in section 6.
Coulibaly et al. [8] proposed a method for identifying pearl millet mildew disease by combining transfer learning with feature extraction. In precision agriculture, deep learning enables a rapid and engaging data analysis. Support for stakeholders (researchers and farmers) through information and knowledge gained through the reasoning process is the anticipated benefit of the project. Positive results can be seen in the f1-score (91.75%), recall (94.50%), and accuracy (95.00%) from the experiments.
Ramathilagam and Haldar [9] aimed to evaluate the potential of multi-temporal Sentinel-1 and RADARSAT-2 data for discriminating pearl millet. Overall accuracy (OA) and kappa coefficient results demonstrate that the RF classifier is superior to the other classifiers. For Sentinel-1 and RADARSAT-2 data, the RF classifier achieved the highest Overall Accuracy (OA) of 89.1% and the highest kappa values of 0.822 and 0.814, respectively. Mohameth et al. [10] discussed that due to the increasing availability of networks and sophisticated cellphones, as well as recent developments in artificial intelligence, a new method known as "smartphone-assisted disease diagnosis" has arisen. Scientists have trained a wide range of models at excellent accuracy levels (above 99.53%). According to the results, SVM is the best classifier for spotting leaf illnesses.
Sladojevic et al. [1] explored a novel strategy for building a plant disease detection model using deep neural networks to classify images of leaves to detect disease. The cutting-edge methodology and innovative approach to training practical use of a system. The deep CNN training was carried out using Caffe, a deep learning framework created by the Berkley Vision and Learning Centre. Based on independent class testing, the developed model's experimental findings showed an average accuracy of 96.3%. Tomato late blight, Septoria spot, bacterial spot, bacterial canker, and tomato leaf curl are only some of the illnesses examined in this study by Sabrol and Satish [11] along with photographs of healthy tomato plants. Image segmentation is used to extract color, shape, and texture traits from pictures of both healthy and diseased tomato plants. A classification tree is then trained using these features. The results demonstrate that across all six categories of tomato photos, the overall categorization accuracy is a staggering 97.3%.
Amoda et al. [12] proposed a method for developing an image processing software solution for automated detection and categorization of plant leaf diseases is quick, easy, cheap, and accurate. Making use of color space transformation and a custom color transformation framework for RGB leaf photos. K-means clustering to divide the photos into smaller pieces. The affected segments will have their texture features computed. Using a neural network that has already been trained to classify the retrieved features. Wang et al. [13] told that wheat stripe rust from wheat leaf rust and grape downy mildew from grape powdery mildew, this study looks at four different types of neural networks (backpropagation, radial basis function, extended regression, and probabilistic neural networks). Prediction accuracy for wheat diseases is maximized by BP networks, GRNNs, and PNNs, and is maximized by 97.50% by RBF neural networks. Prediction accuracy for grape illnesses ranges from 100% for BP networks, GRNNs, and PNNs to 94.29% for RBF neural networks.
For the purpose of preparing pictures for analysis by deep learning models, preprocessing is a key step. This helps to increase the performance and efficiency of the models in tasks such as classification, detection, and segmentation respectively. Data analysis and machine learning activities cannot be guaranteed for accuracy, compatibility, or peak performance without first undergoing preprocessing. The quality of the data is improved, the data is made ready for analysis or modelling, and the performance of the algorithms is enhanced. For image classification with CNNs, Zero Component Analysis (ZCA) proves superior than Mean Normalization and Standardization across all three networks [14].
When the training dataset is small and more training samples are needed to increase the model's performance and resilience, affine data augmentation can be a helpful tool. Affine transformations are used to increase the size of the dataset so that the model may learn to be more invariant to different transformations and better equipped to deal with fluctuations in the test data that occur in the real world. Extensive experiments were performed, and the results informed the development of implementation recommendations [15]. The general transformation like flipping, rotating, translation, scaling, zooming, shearing is applied in order to enhance the data, and examples of the resulting enhanced images are shown in Figure 1.
Figure 1. Scaled and enhanced images
Sharpening an image serves to improve its overall quality by drawing attention to the sharp transitions between areas of an image. The goal of image sharpening is to create an impression of greater sharpness and improve the image's aesthetic appeal. Because of its ease of use and small computing footprint, Contrast Limited Adaptive Histogram Equalization (CLAHE) is widely deployed. Image capturing produces noise, thus enhancing methods are needed to counteract it. Three picture channels are used for the enhancing procedure. Authors discussed a blood vessel in the retina, which can be seen with the naked eye [16]. Root Mean Square Error and entropy are both easily measured with the CLAHE method, making it the preferred method of picture improvement [17]. CLAHE is favoured for image improvement activities because to its ability to successfully improve image details, maintain a natural appearance, and adapt to different lighting conditions. Figure 2 shows the pictures after they have been made sharper and their colors have been made brighter. These Eqs. (1) and (2) are used to derive the clip limit from the normalized clip limit value:
$x=ceil(y/z)$ (1)
$clip\lim it=x+round(normClipLimit*(Y-X))$ (2)
Figure 2. Displays the pictures after they have been sharpened and their color brightened
Images of varying sizes were captured on handheld devices in Saveetha School of Engineering's pearl millet farm geographical region. The devices used to capture the photographs were of diverse sizes. As a result, it is essential to scale all of the visuals to the same dimensions, which are 224 × 224 × 3 pixels. Some more photos have been taken from a Kaggle dataset and used the multi-labelling approach to manually classify all the images. Using this method, a picture displaying symptoms of both Ergot and Smut may be annotated with the words "Ergot" and "Smut" simultaneously. Figure 3 displays these images with annotations:
Following the completion of data augmentation, a total of 20,000 images is used in order to make it suitable for usage by deep learning architectures. Despite ResNet-50's poor training accuracy, in comparison to DenseNet-121 and ResNet-152, the training and validation data are well-merged, and the majority of the testing data were correctly predicted. The results of the ResNet-50 and ResNet-152 designs are very similar when thinking about ResNet-152.
Adding more layers, though, makes the process more time-consuming and complicated. DenseNet-121 design achieves 99% training accuracy in 10 epochs. As noted in DenseNet-121 architecture, this is visible in the plot. Consequently, ResNet-50 will serve as the foundational framework for our project.
In this research, a novel architecture that integrates an LSTM network with ResNet-50 and incorporates dropout layers with a value of 0.2 is presented. Model design, dataset, and training parameters all have a role in determining the optimal dropout rate. A low dropout rate is usually the best place to start when trying to achieve regularization, and you can always raise it later if needed. This was done to show how the combination improves accuracy, which makes illness classification easier and more precise.
Figure 3. Pictorial representations of annotated
4.1 ResNet-50 architecture
ResNet-50 has outperformed shallower networks in a number of image recognition problems, showing outstanding performance. Many computer vision jobs favor it due to its capacity to train very deep networks and handle deeper designs. Since its inception, other ResNet-50 depth variants (ResNet-18, ResNet- 34, ResNet-101, etc.) have been developed to better fit a variety of use cases and computing environments. ResNet-50 is a deep neural network design with 50 layers and four stages such as Convolutional layer, Residual blocks, Global Average Pooling, Fully Connected Layer. Spinach, a popular and nutritious meal, is susceptible to Cercospora, a disease that attacks plant leaves. Consumption of contaminated spinach can cause illness in people and animals. ResNet-50 architecture, to identify and properly categorize spinach leaves contaminated with Cercospora. Training and testing on publicly available datasets demonstrate great accuracy, paving the way for accurate diagnosis and prompt treatment to contain the disease [18]. The ResNet-50 model has been judged to be superior to Alex net, which is CNN's top model. In the context of this comparison, the findings obtained using ResNet-50 are more ideal [19]. Even without using state-of-the-art hardware like GPUs or Tensor Processing Units (TPUs), Transfer learning for malaria image classification has yielded promising results. Using this cutting- edge hardware could improve precision and significantly decrease processing time [20]. The normal condition of the plant and its growth can be disrupted by a number of different plant diseases. Plant diseases are one of the primary contributors to decreased productivity, which ultimately results in economic losses [21]. The amount of parameters and processing time needed to produce classification results are both reduced when using ResNet-50 [22]. In order to merge the multidimensional identification findings, MDFC-ResNet recognizes from three different dimensions: species, coarse-grained disease, and fine- grained illness [23].
A convolutional layer's output feature map may be computed as Eq. (3):
$Y=conv2D(X,W)+b$ (3)
Normalization of the output of a convolutional layer is accomplished through the use of batch normalization.
The following Eq. (4) shows how to compute the output after normalization:
$Y=(X-\mu )/(\sqrt{{{\sigma }^{2}}}+s)$ (4)
Elements of the feature maps are activated using the rectified linear unit (ReLU) function, which converts all negative values to zero. This is given by Eq. (5):
$Y=\max (0,X)$ (5)
Through max pooling, the spatial dimensions of the feature maps are reduced. The final feature map is derived by taking the maximum from each pooling region. The Eq. (6) shows for max pooling:
$Y=MaxPool(X)$ (6)
ResNet-50 computes the output of a residual block using the following series of convolutional layers and shortcut connections:
$Y=F(X)+X$ (7)
With global average pooling, the median of all feature maps in all space dimensions was found. This expression is given in Eq. (8):
$Y=AvgPool(X)$ (8)
A fully connected layer receives the final result of the global mean pooling layer and uses it to make a classification. The logits Z of the output are computed by Eq. (9):
$Z=(W*Y)+b$v (9)
The final class probabilities are computed by applying the softmax activation function to the logits. This is given by expression (10):
$P=soft\max (z)$ (10)
Figure 4. Accuracy and loss analysis of training and validation using ResNet-50 model
In the ResNet-50 architecture, all the aforementioned equations reflect the most basic processes. Our 100-epoch analysis for multi- image categorization with the ResNet-50 architecture is displayed in Figure 4. Batch size of 32 has been considered for this analysis with the learning rate of 0.01. Accuracy measurements during training and validation have been made. With a 0.9% loss in training, the model has achieved an accuracy of 84.8%.
The parameters for the ResNet-50 model have been computed, but because our attention is solely focused on the disease, it is determined that the accuracy of the model has to be enhanced. The model has been used to estimate a test accuracy of 85.13 percent and a loss of 0.91 percent. The expected outcome is depicted in Figure 5.
Figure 5. Predicted outcome using ResNet-50
4.2 DenseNet-121 architecture
The dense block, which connects each layer to every other layer in a feed-forward way, is the most important contribution that DenseNet-121 makes to the field of computer networking. Because of this extensive connectedness, each layer is guaranteed to receive the feature maps from all of the layers that came before it, which results in more effective parameter sharing, feature reuse, and improved gradient flow while the model is being trained. As a direct consequence of this, DenseNet-121 is able to go with a smaller set of parameters than conventional designs such as ResNet-50.
This procedure was carried out for a total of 100 epochs with a batch size of 32, and a loss function based on categorical cross entropy has been utilized because it is multi-classification. DenseNet-121 is a broad model in Deep Learning architectures that can identify 15 different types of plant diseases more precisely and with a higher level of accuracy [24]. However, in order to achieve high classification accuracy, the computational complexity of the DenseNet-121 model has to be simplified, as does its error rate. As a result, the MHGSO optimization approach is the one that is utilized in order to improve its parameters [25]. This algorithm contributes to the provision of an effective result in the detection of plant diseases, which, in turn, contributes to the improvement of the economy of the nation. After converting the original picture to HSV colour form, the thresholding process is used to create the masked image, which is then sent to the proposed model for the purposes of training and classification [26]. ImageNet weights are put to use in this transfer learning procedure in order to diagnose plant illnesses that are provided as inputs. As a result, it is now considered the most important of all machine learning algorithms [27]. Hyperparameter tweaking for widely used pre-trained models including DenseNet-121, ResNet-50, VGG-16, and Inception V4. Plant Village, a widely used dataset, was used for the studies. It contains 54,305 picture samples of various plant disease types, organized into 38 classifications. Classification accuracy, sensitivity, specificity, and F1 score were used to assess the model's efficacy. In addition, the results were compared to those of other, similar, state-of-the-art investigations. The results shown that DenseNet-121 outperformed the state-of-the- art models with a 99.81% increase in classification accuracy [28].
ResNet-50 was analysed alongside DenseNet- 121, and the outcomes of the latter’s training and testing are depicted in Figure 6. The results reveal a 99.66% accuracy during training and a 98.15% accuracy during validation. There were around 1.14 percent training-set losses and 7.7 percent validation-set losses. It has been determined that the correctness of the tests is 80.23 percent, and that the same-architecture loss is 14.22 percent.
Figure 6 demonstrates that the network has been adequately converged for only the previous twenty-five epochs. Later on, the convergence halts, and the waveforms no longer correspond with one another, indicating overfitting.
Therefore, the precision of the tests also fell short of expectations. This is due to the fact that the amount of images used by ResNet- 50 is insufficient for the DenseNet-121 model, despite the fact that the model performs well across a wide variety of datasets.
As a result, there is overfitting. Since we’re testing all of the models on the same data, this one doesn’t perform well with the present sample size.
Figure 6. Accuracy and loss analysis of training and validation
4.3 ResNet-152 architecture
ResNet-152 has 152 layers, making it an even more complex neural network than ResNet-50. Like previous ResNet versions, ResNet-152 solves the vanishing gradient problem in training very deep networks by making use of residual blocks. The network can be trained more efficiently and with better results because of the residual blocks' ability to keep gradient flow steady at greater depths. It is similar to ResNet-50 in that it contains four phases to compute the accuracy for the input that is provided.
On large-scale datasets like ImageNet, ResNet-152 has shown state-of-the- art performance in a number of image recognition and computer vision applications. ResNet-152's depth, however, makes it more computationally costly than shallower topologies and may necessitate additional memory and processing capacity for training. ResNet-152 has been applied to the rice picture dataset by the authors for the purpose of classifying four distinct illnesses, and the results demonstrate that this model will deliver a high level of accuracy [29]. The dataset was ran for the same number of epochs as ResNet-50, and the batch size remained at 32. The loss function categorical cross entropy was used throughout. The outcomes of the ResNet-50 and ResNet-152 models are not distinguishable from one another in any significant way. As a result, in order to decrease the amount of computing complexity involved in our study, ResNet-50 has been taken for research. Figure 7 depicts the training and validation analyses, which will reveal the convergence of the network in greater detail.
Table 1. Accuracy comparison between three architectures
|
Architecture |
Epochs |
Accuracy (in %) |
|
ResNet-50 |
100 |
84.8 |
|
DenseNet-121 |
10 |
99 |
|
ResNet-152 |
100 |
86 |
Although ResNet-50 has poor accuracy, it is selected as the base architecture due to the fact that DenseNet-121 produces overfitting errors after 10 epochs and ResNet-152 achieves nearly the same level of accuracy as ResNet-50. This comparison is shown in Table 1.
For accurate disease classification in horticulture images, a novel ResNet-50 model that combines an LSTM network with a 0.2% dropout layer after the convolutional layers of the classic ResNet-50 design was proposed. Figures 7 and 8 present a unique model that has been proposed, along with the procedure for it.
The residual blocks are the primary components that go into the construction of ResNet-50. Each residual block consists of a number of convolutional layers in addition to shortcut connections. Because of these connections, the network is able to bypass one or more layers, which contributes to the solution of the vanishing gradient problem that occurs during the training of very deep neural networks. The Proposed model was implemented with a 0.2 percent dropout layer and an LSTM network at the end of the residual blocks. While adding Long Short-Term Memory (LSTM) units to a ResNet-50 architecture can drastically alter the network's characteristics and the tasks it is capable of performing, but it also helps to decrease memory consumption in specific cases. First and foremost, the incorporation of LSTM is for the purpose of processing sequences of features retrieved by ResNet-50. In the context of a classification task, for instance, the ResNet-50 may be used as a feature extractor for images, while the LSTM could be used to represent the temporal dependencies observed in the derived features across time. Thorough training and optimization are necessary to integrate LSTM with ResNet-50. To make sure both parts learn well, it has to figure out how to initialize the LSTM weights, aggregate their losses, and train them in a balanced way. In situations where training data is scarce or noisy, dropout can regularize the LSTM layers, lowering the danger of overfitting. This, in turn, might encourage the two components to acquire more robust and generalizable features, which could lead to an improvement in overall performance.
Figure 7. Workflow of ERLSTMH model
Figure 8. Proposed novel ERLSTMH model
The original mobile-collected dataset is then scaled and supplemented. In a later step, sharpening and color enhancement using CLAHE technique are applied to these images. There are six different labels, which will refer to as Downy Mildew, Blast, Ergot, Healthy, Rust, and Smut. These clusters have been divided into three categories: 70% Train, 20% Test, and 10% Validation. Our suggested model receives the input, and then it operates for a total of 100 epochs with a batch size of 32.
The illness detection parameters for each disease class and a comparison of the amount of accuracy are discussed in results along with the total accuracy of the analysis performed on the test set. It is a reflection of the model's ability in reliably recognizing illness-affected pixels and separating them from healthy ones that the disease detection accuracy is so high across all classes.
5.1 Optimal use of computing power
On a normal hardware configuration, an examination of computational efficiency is carried out in order to estimate the amount of time required for inference and the computing needs of the model. A computer equipped with an NVIDIA GeForce RTX 4050 graphics processing unit (GPU) was used for the research.
5.2 Disease overview
Within the scope of our investigation, not one but five unique illness datasets have been combined, in addition to one that contains information on plants that are in good condition. Downy mildew, blast, ergot, rust, smut, and healthy are the illnesses that are being discussed here. One of the most destructive diseases that may affect pearl millet (Pennisetum glaucum) is called downy mildew, and it is caused by the pathogen Sclerospora graminicola. The obligatory pathogen Sclerospora graminicola can't live or multiply without the host organism. Although other types of grass are susceptible, pearl millet is the most common host. The illness mostly manifests itself as a downy growth on the underside of leaves, which can range in color from white to grayish-white. This gives the leaves a "downy" or fuzzy look. There is also the possibility that infected leaves will exhibit yellowing, stunting, and ultimately, premature death. Pathogens are able to live in the soil and on agricultural wastes that have been contaminated. This organism generates spores, which can be dispersed by the wind, rain, and irrigation water. As long as the conditions are suitable (high humidity and moderate temperatures), infection will take place when spores settle on vulnerable plant portions. A devastating disease of pearl millet (Pennisetum glaucum) that affects several phases of plant growth, including seedlings, vegetative stages, and panicle development, is called blast. This disease is caused by the fungus Magnaporthe grisea. In pearl millet, the signs of blast include lesions that range from elliptical to spindle-shaped and appear on the leaves, stems, and panicles. In the beginning, these lesions have a light brown color with a black border, and they have the potential to combine to produce bigger necrotic regions. Infected panicles may not grow to their full potential, which might result in a decreased grain production. As well as other types of grass, pearl millet (Pennisetum glaucum) is susceptible to ergot, which is a disease that is caused by the fungus Claviceps. A structure known as sclerotia is produced by the fungus, and it is responsible for replacing the seeds in the flower head. Pearl millet plants that have been infected with ergot often have black, elongated formations called sclerotia in place of the seeds that are found in the panicle. These sclerotia are body structures that are tough and black in color, and they carry fungal spores. As an additional symptom, infected panicles may exhibit discoloration and deformation. Pearl millet grains that have been infected with ergot are frequently tainted with harmful alkaloids that are generated by the species of fungus. Ergotism is a disease that can affect both people and animals, and it can be caused by the consumption of grains that have been infected with ergot. It is possible to experience hallucinations, convulsions, and gangrene when you have ergotism. Pearl millet is also susceptible to a disease known as rust, which is caused by the fungus Puccinia substriata. The unique reddish-brown spores that are formed on diseased plant tissues are what contribute to its widespread recognition. When pearl millet is affected by rust, the leaves, stems, and sometimes the panicles will have tiny, reddish-brown pustules that range in size from circular to elongated. These pustules are called uredinia. It is possible for these pustules to give the plant a rusty color since they contain multitudes of spores each [29]. Yellowing of the leaves, early leaf drop, and a decrease in grain output are all potential outcomes of severe infections. Pearl millet (Pennisetum glaucum) is susceptible to a disease called Smut, which is caused by the fungus Tolyposporium penicillariae. This disease affects pearl millet and can result in severe production reductions. It is distinguished by the appearance of enormous, black, powdery masses of spores known as teliospores on panicles that have been infected with the disease. The development of smut symptoms includes the appearance of black, powdery masses of spores on the panicles of the plant. If the panicles are infected, they may become deformed and fail to produce seeds that are viable [30]. In extreme circumstances, smut can result in the full reduction or elimination of grain yield in plants that are impacted.
5.3 Algorithm of ERLSTMH model
# Import necessary libraries and modules
# Set the number of classes and image dimensions
# Load ResNet-50 base model
# Freeze the base model's layers
# Create the top layers for classification # Add LSTM layer
# Add Dropout layer (0.2)
# Add dense layers for classification # Compile the model
# Set up data augmentation for training images # Load and augment the training images
# Set up data augmentation for validation images
# Load and augment the validation images
# Train the model
for epoch in range (1, num_epochs+1): for step in range (1, num_steps+1):
# Get a batch of training data from the train_generator
# Perform forward pass through the model # Calculate the loss
# Backpropagate gradients and update the weights
# Evaluate the model on the validation data and store the history for this epoch
# Access the epoch history
# Plot loss and accuracy over epochs
# Load and augment the validation images for visualization
# Display the randomly selected images with true and predicted labels
# Set up data augmentation for testing images # Load and augment the testing images
# Generate predictions for the testing data # Get the true labels and predicted labels # Get the file paths of all testing images
# Randomly select images for prediction
# Display the randomly selected images with true and predicted labels
# Evaluate the model on the testing data and get testing loss and accuracy
# Plot the testing accuracy and loss over epochs
# Generate predictions for the validation data
# Randomly select images to display
# Assuming `predictions` is a 2D array of shape (num_samples, num_classes)
# Get the predicted labels for the randomly selected validation images
# Set the paths to your train, test, and validation data
# Check if the train, test, and validation directories exist
# Randomly select images for prediction
# Generate predictions for the randomly selected validation images
# Display the randomly selected images with true and predicted labels
# Load and augment the validation images for calculating the confusion matrix and classification report
# Generate predictions for the validation data
# Get the true labels and predicted labels
# Compute the confusion matrix
# Display the confusion matrix
# Calculate the classification report
# Print the classification report
# Calculate Sensitivity and Specificity for each class
# Print Sensitivity and Specificity for each class
Each network was trained over the span of a total of 100 epochs, with each epoch comprised of 236 iterations and a batch size of 32. The learning rate for training was set at 0.005 for the first 60 epochs, 0.001 for the subsequent 20 till 80, and 0.0001 for the last 20. In order to train the models, a machine that has a CPU with an Intel Core i5-RTX 4050 and 15.6 gigabytes of RAM is utilized. Using the options that are provided, it is possible to train any particle form, whatever its size or aspect ratio.
In Figure 9, it clearly shows that Analysis of training and validation has been done perfectly. In a similar manner, the validation function was compared to the testing dataset and analyzed, and these results have been presented in Figure 10. The true and predicted outputs have been taken and these outputs using the proposed model has been given in Figure 11.
A good indication of this can be seen in Figure 10, which demonstrates that the loss is decreasing in both the training dataset and the testing dataset. The accuracy of disease prediction is illustrated in Figure 11, which includes both the actual names of diseases and the names that were predicted for them.
Figure 9. Analysis of training and validation using a proposed ERLSTMH model
Figure 10. Analysis of testing and validation using a proposed ERLSTMH model
The parameter analysis results for the ResNet-50 and ERLSTMH designs are summarized in Table 2. The table demonstrates that for every condition, the ERLSTMH design performs better than the RESNET-50 architecture. Figure 12 shows the statistical analysis of parameters chosen.
Table 3 and Figure 13, gives the comparison of existing and proposed architecture and corresponding statistical analysis respectively.
The confusion matrix gives a more complete picture of how well, model is performing for each class and assists in identifying regions in classification issues in which the model may be making mistakes more frequently.
The true positive (TP) values for each class are shown along the main diagonal of the confusion matrix, while the other values reflect the false positive (FP), false negative (FN), and true negative (TN) rates.
Table 2. Parameter analysis using ResNet-50 and RLSTMH architectures
|
Architecture |
Diseases |
Precision |
Recall |
F1-score |
Sensitivity |
Specificity |
|
RESNET-50 |
BLAST |
0.169 |
0.183 |
0.175 |
0.183 |
0.779 |
|
DOWNY MILDEW |
0.161 |
0.221 |
0.186 |
0.221 |
0.829 |
|
|
ERGOT |
0.184 |
0.162 |
0.172 |
0.162 |
0.835 |
|
|
HEALTHY |
0.159 |
0.192 |
0.174 |
0.192 |
0.851 |
|
|
RUST |
0.083 |
0.066 |
0.073 |
0.066 |
0.865 |
|
|
SMUT |
0.202 |
0.167 |
0.183 |
0.166 |
0.825 |
|
|
ERLSTMH (Proposed) |
BLAST |
0.90 |
0.86 |
0.88 |
0.9809 |
0.8623 |
|
DOWNY MILDEW |
0.93 |
0.95 |
0.94 |
0.9840 |
0.9487 |
|
|
ERGOT |
0.81 |
0.98 |
0.88 |
0.9663 |
0.9761 |
|
|
HEALTHY |
0.96 |
0.95 |
0.96 |
0.9914 |
0.9541 |
|
|
RUST |
0.91 |
0.78 |
0.84 |
0.9815 |
0.7778 |
|
|
SMUT |
0.92 |
0.95 |
0.93 |
0.9835 |
0.9517 |
Table 3. Comparison of accuracy
|
Architecture |
Accuracy (in %) |
|
RESNET-50 |
84.8 |
|
DENSENET-121 |
99 |
|
RESNET-152 |
86 |
|
ERLSTMH(Proposed) |
97.147 |
Figure 11. True and predicted images using ERLSTMH model
Figure 12. Statistical analysis of parameters
Figure 13. Statistical analysis of accuracy
Figure 14 shows the confusion matrix for this classification problem using proposed model and Precision, Accuracy, Recall, Fi-Score, Sensitivity, and Specificity are some of the metrics used to compile a thorough categorization report and graphical representations.
Figure 14. Confusion matrix
As an inference, it is essential to ensure that plant diseases are accurately diagnosed and detected at an early stage in order to maintain agricultural output. Deep learning, and more specifically the Ensemble ResNet LSTM for Horticulture plants (ERLSTMH) model that was suggested in this work, has shown tremendous potential in terms of obtaining accurate and dependable disease diagnosis. The ERLSTMH model outperformed previous models such as ResNet-50, Dense-121, and ResNet-152, demonstrating greater accuracy in forecasting diseases such as Downy Mildew, Ergot, Rust, Blast, and Smut. This was accomplished by utilizing labeled datasets of Pearl millet plant illnesses and photos of healthy plants. Farmers may be empowered with timely information through the use of modern technologies such as ERLSTMH, which enables them to make decisions that are informed, reduces crop losses, and contributes to an agriculture industry that is more efficient and sustainable when it is implemented. In comparison to all of the previous structures, the suggested architecture achieved an extremely high level of accuracy, which was 97.147% effective.
We would like to use this opportunity to extend our appreciation to everyone who participated in this study with the purpose of utilizing deep learning techniques to detect plant illnesses at an earlier stage. I would like to extend my sincere gratitude to the farmers and agricultural professionals who contributed their important thoughts and knowledge to the development of this study. We also like to express our gratitude to the individuals who were responsible for the collection and labeling of the datasets that were utilized in this study. Without their unwavering commitment and unwavering support, our endeavor definitely would not have been feasible.
[1] Sladojevic, S., Arsenovic, M., Anderla, A., Culibrk, D., Stefanovic, D. (2016). Deep neural networks based recognition of plant diseases by leaf image classification. Computational Intelligence and Neuroscience, 2016(1): 3289801. http://dx.doi.org/10.1155/2016/3289801
[2] Albattah, W., Nawaz, M., Javed, A., Masood, M., Albahli, S. (2022). A novel deep learning method for detection and classification of plant diseases. Complex & Intelligent Systems, 8: 507-524. https://doi.org/10.1007/s40747-021- 00536-1
[3] Kundu, N., Rani, G., Dhaka, V.S., Gupta, K., Nayak, S.C., Verma, S., Ijaz, M.F., Wo´zniak, M. (2021). IoT and interpretable machine learning based framework for disease prediction in pearl millet. Sensors, 21(16): 5386. https://doi.org/10.3390/s21165386
[4] Shoaib, M., Hussain, T., Shah, B., Ullah, I., Shah, S.M., Ali, F., Park, S.H. (2022). Deep learning-based segmentation and classification of leaf images for detection of tomato plant disease. Frontiers in Plant Science, 13: 1031748. https://doi.org/10.3389/fpls.2022.1031748
[5] Al-Hiary, H., Bani-Ahmad, S., Reyalat, M., Braik, M., Alrahamneh, Z. (2011). Fast and accurate detection and classification of plant diseases. International Journal of Computer Applications, 17(1): 31-38.
[6] Vamsidhar, E., Rani, P.J., Babu, K.R. (2019). Plant disease identification and classification using image processing. International Journal of Engineering and Advanced Technology, 8(3): 442-446.
[7] Satankar, M., Kumar, U., Patil, A.K., Kautkar, S. (2020). Pearl millet: A fundamental review on underutilized source of nutrition. Multilogic Sci, 10: 1081-1084.
[8] Coulibaly, S., Kamsu-Foguem, B., Kamissoko, D., Traore, D. (2019). Deep neural networks with transfer learning in millet crop images. Computers in Industry, 108: 115-120. https://doi.org/10.1016/j.compind.2019.02.003
[9] Ramathilagam, A.B., Haldar, D. (2022). Evaluation of different machine learning algorithms for pearl millet discrimination using multi-sensor SAR data. Geocarto International, 37(17): 5116-5132. https://doi.org/10.1080/10106049.2021.1914744
[10] Mohameth, F., Bingcai, C., Sada, K.A. (2020). Plant disease detection with deep learning and feature extraction using plant village. Journal of Computer and Communications, 8(6): 10-22. https://doi.org/10.4236/jcc.2020.86002
[11] Sabrol, H., Satish, K. (2016). Tomato plant disease classification in digital images using classification tree. In 2016 International Conference on Communication and Signal Processing (ICCSP), Melmaruvathur, India, pp. 1242-1246. http://doi.org/10.1109/ICCSP.2016.7754351
[12] Amoda, N., Jadhav, B., Naikwadi, S. (2014). Detection and classification of plant diseases by image processing. International Journal of Innovative Science, Engineering & Technology, 1(2): 211-217.
[13] Wang, H., Li, G., Ma, Z., Li, X. (2012). Image recognition of plant diseases based on principal component analysis and neural networks. In 2012 8th International Conference on Natural Computation, Chongqing, China, pp. 246-251. http://doi.org/10.1109/ICNC.2012.6234701
[14] Pal, K.K., Sudeep, K.S. (2016). Preprocessing for image classification by convolutional neural networks. In 2016 IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT), Bangalore, India, pp. 1778-1781. http://doi.org/10.1109/RTEICT.2016.7808140
[15] Wei, J., Zou, K. (2019). Eda: Easy data augmentation techniques for boosting performance on text classification tasks. arXiv preprint arXiv:1901.11196. https://doi.org/10.48550/arXiv.1901.11196
[16] Setiawan, A.W., Mengko, T.R., Santoso, O.S., Suksmono, A.B. (2013). Color retinal image enhancement using CLAHE. In International Conference on ICT for Smart Society, Jakarta, Indonesia, pp. 1-3. http://doi.org/10.1109/ICTSS.2013.6588092
[17] Garg, D., Garg, N.K., Kumar, M. (2018). Underwater image enhancement using blending of CLAHE and percentile methodologies. Multimedia Tools and Applications, 77: 26545-26561. https://doi.org/10.1007/s11042-018-5878-8
[18] Ramkumar, M.O., Catharin, S.S., Ramachandran, V., Sakthikumar, A. (2021). Cercospora identification in spinach leaves through ResNet-50 based image processing. In Journal of Physics: Conference Series, 1717(1): 012046. https://doi.org/10.1088/1742-6596/1717/1/012046
[19] Ajra, H., Nahar, M.K., Sarkar, L., Islam, M.S. (2020). Disease detection of plant leaf using image processing and CNN with preventive measures. In 2020 Emerging Technology in Computing, Communication and Electronics (ETCCE), Bangladesh, pp. 1-6. https://doi.org/10.1109/ETCCE51779.2020.9350890
[20] Reddy, A.S.B., Juliet, D.S. (2019). Transfer learning with ResNet-50 for malaria cell-image classification. In 2019 International Conference on Communication and Signal Processing (ICCSP), Chennai, India, pp. 0945-0949. http://doi.org/10.1109/ICCSP.2019.8697909
[21] Khandelwal, I., Raman, S. (2019). Analysis of transfer and residual learning for detecting plant diseases using images of leaves. In book: Computational Intelligence: Theories, Applications and Future Directions - Volume II, 295-306. http://doi.org/10.1007/978-981-13-1135-2_23
[22] Ganatra, N., Patel, A. (2020). Performance analysis of fine-tuned convolutional neural network models for plant disease classification. International Journal of Control and Automation, 13(3): 293-305.
[23] Hu, W.J., Fan, J., Du, Y.X., Li, B.S., Xiong, N., Bekkering, E. (2020). MDFC–ResNet: An agricultural IoT system to accurately recognize crop diseases. IEEE Access, 8: 115287-115298. http://dx.doi.org/10.1109/ACCESS.2020.3001237
[24] Dubey, N., Bhagat, E., Rana, S., Pathak, K. (2022). A novel approach to detect plant disease using DenseNet-121 neural network. In Smart Trends in Computing and Communications: Proceedings of SmartCom 2022, pp. 63-74. https://doi.org/10.1007/978-981-16-9967-2_7
[25] Nandhini, S., Ashokkumar, K. (2022). An automatic plant leaf disease identification using DenseNet-121 architecture with a mutation-based henry gas solubility optimization algorithm. Neural Computing and Applications, 34(7): 5513-5534. https://doi.org/10.1007/s00521-021- 06714-z.
[26] Swaminathan, A., Varun, C., Kalaivani, S. (2021). Multiple plant leaf disease classification using DenseNet-121 architecture. International Journal of Electrical Engineering and Technology, 12: 38-57. https://doi.org/10.34218/IJEET.12.5.2021.005
[27] Arathi, B., Dulhare, U.N. (2023). Classification of cotton leaf diseases using transfer learning-DenseNet-121. In book: Proceedings of Third International Conference on Advances in Computer Engineering and Communication Systems, 393-405. http://doi.org/10.1007/978-981-19-9228-5_33
[28] Eunice, J., Popescu, D.E., Chowdary, M.K., Hemanth, J. (2022). Deep learning-based leaf disease detection in crops using images for agricultural applications. Agronomy, 12(10): 2395. https://doi.org/10.3390/agronomy12102395
[29] Shoaib, M., Shah, B., EI-Sappagh, S., Ali, A., Ullah, A., Alenezi, F., Gechev, T., Hussain, T., Ali, F. (2023). An advanced deep learning models-based plant disease detection: A review of recent research. Frontiers in Plant Science, 14: 1158933. https://doi.org/10.3389/fpls.2023.1158933
[30] Saleem, M.H., Potgieter, J., Arif, K.M. (2019). Plant disease detection and classification by deep learning. Plants, 8: 468. https://doi.org/10.3390/plants8110468
