Weed Detection and Localization in Soybean Crops Using YOLOv4 Deep Learning Model

Soybean is a widely grown edible oil seed as it is rich protein food for human being and animals. Animal consumes it through soybean meal, and humans use it as oil. According to Soy stats, Brazil is the world’s major soybean producer and it shares around 25% of edible oil. It is needed to improve the quality and quantity of soybean by removing weeds. Weeds can compete with soybean plants for essential resources like water nutrients and sunlight, and so crop yield can be negatively impacted. Also weeds increase the risk of disease and pests, interfere the harvest and post-harvest process, thus increasing the production cost. So accurate and efficient weed detection model is needed to optimize crop yields quality, minimize herbicide usage and production costs, promoting sustainable and eco-friendly farming practices and enable precision weed management. At present, for weed management herbicides are sprayed, which causes harmful environmental effects. Locating the weed precisely and spraying the herbicides at the specific location reduces the adverse effects. Additionally, weeds and soybean are similar in color and shape, Intra- and inter-species variability of weeds in terms of its features like shape, size, color, and texture is also very less. So accurate and robust detection of weeds remains as a challenging task. To address this issue various technologies and methods have been developed for detection of weeds in soybean field. Earlier methods include visual inspection of the field by farmers, where weeds are identified by their appearance and manually removed. This is labor- intensive and time –consuming, not practical for large fields. Later feature based methods are used considering color, histograms, texture descriptors and shape features. In recent years, machine learning algorithms like Support vector machines (SVM), K-nearest neighbors (K-NN), etc. are used for classification. These methods have limitations that they may not have capability to learn and adapt to variations in lighting conditions, view point and background clutter.

Deep learning models such as convolutional neural networks (CNNs) has ability to handle complex and diverse datasets effectively. In object detection, to localize the multiple objects popular models like single shot multibox detector (SSD), region based convolutional neural network (R-CNN) and You Only Look Once (YOLO) are widely used. In order to improve the detection accuracy and to increase the robustness of the model, Contrast Limited Adaptive Histogram Equalization (CLAHE) is used for preprocessing.

The objectives of this paper are as follows:

1. Applying CLAHE for preprocessing the images in the dataset.
2. Investigating different state-of-art deep learning models viz., Darknet19, Mobilenetv2, VGG19, Resnet18, Inceptionv3 and Densenet201 for classification of weed/crop.
3. Detection and localization of weed/crop with different state-of-art deep learning models viz., YOLOv4, R-CNN and SSD networks to propose an accurate model for precision agriculture, particularly for soybean crop.

This paper is organized into five chapters. Chapter2 is for literature work on earlier traditional feature based algorithms and deep learning algorithms used in weed/crop detection. Chapter3 presents the Data Acquisition, labeling, and implementation of YOLOv4 with Loss function. Chapter4 presents the simulation results along with performance analysis of different pre-defined networks, R-CNN, SSD and YOLOv4. Finally, chapter5 concludes with some multi- directions for future work.

3.1 Data set

a.png

(a)

b.png

(b)

Figure 1. Sample images in dataset

For this work the data set used is a publicly available dataset from Kaggle website. Santos Ferreira used soybean images that are captured using drone, contains soybean images, grass images, broadleaf images and soil images. From this we considered grass and broad leaf as weed and soybean as crop. The data set is split into three parts. Sixty percent is for training, ten percentages for validation and remaining for testing. Example sample input images shown in Figure 1.

3.2 YOLOv4 architecture

YOLOv4 algorithm is used for accurate and efficient object detection tasks shown in Figure 2. It has three parts viz., backbone, neck and head network. The backbone is based on CSPDarknet53 (Cross Stage Partial Darknt53) to extract the hierarchical features from the input image. The cross-stage part represents connecting information across different stages of layers of the network, where as partial network implies that not all the layers or stages are used. CSPDarknet53 represents combination of Darknet framework and CSP architecture, using 53 layers in the network. Neck uses path aggregation network (PANet) and Spatial Attention Module (SAM). PANet helps the model to aggregate features from different layers and ability to detect objects at different scales and aspect ratios. SAM focus on relevant regions to enhance feature fusion. Head network is responsible for predicting bounding boxes and class probabilities, it has three detection sub heads, which are designed for corresponding objects at different scales. The prediction includes bounding box coordinates (x, y, width, and height), object score and class probabilities. FPN is for combining multi-scale features. It enhances the model’s ability to detect objects of various sizes and maintain good accuracy. SSP can improve the networks ability to detect objects at different spatial resolutions. It also allows the model to focus on both small and large objects in the image. SSP kernels size is 1×1, 5×5, 9×9 and 13×13 for max pooling, the stride is considered as 1.

3.3 YOLOv4 loss function

In YOLOv4, object detection model, the loss function is composed of three components viz., classification loss, localization loss and confidence loss. These are used to train the model to accurately detect objects in an image. Classification loss is used to determine how well the model classifies objects within each grid cell. If an object is present in the grid cell then classification loss is computed. If there is no object present in the cell, the loss is calculated based on confidence score, which should be close to zero. The Localization loss or Regression loss is used to measure how well the model predicts the bounding box coordinates for each objects in the image. Confidence loss on the other hand measures how well the model predicts the confidence score, which indicates the likelihood that an object exists within a grid cell. These three loss functions are combined to form the final loss function used for training YOLOv4.

Classification loss can be evaluated as given by Eq. (1):

$\sum_{i=0}^{S^2} 1_i^{\text {obj }} \sum_{c \in \text { classes }}\left(p_i(c)-p_i^{\wedge}(c)\right)^2$    (1)

Here, $1_i^{o b j}=1$ means the object present in the cell or else it is zero, and $p_i^{\wedge}(c)$ is class 'c' conditional class probability. Localization loss evaluated using Eq. (2):

$\begin{gathered}\lambda_{\text {coord }} \sum_{i=0}^{S^2} \sum_{j=0}^B 1_{i j}^{o b j}\left[\left(x_i-x_i^{\wedge}\right)^2+\left(y_i-y_i^{\wedge}\right)^2\right]+ \\ \lambda_{\text {coord }} \sum_{i=0}^{S^2} \sum_{j=0}^B 1_{i j}^{o b j}\left[\left(\sqrt{w_i}-\sqrt{w_i{ }^{\wedge}}\right)^2+\left(\sqrt{h_i}-\right.\right. \\ \left.\left.\sqrt{h_i{ }^{\wedge}}\right)^2\right]\end{gathered}$     （2）

Here, $1_{i j}^{o b j}=1 $ if the $\mathrm{j}^{\text {th }}$ bounding box of cell 'i' is accountable for object detection, or else it is 0.

2.png

Figure 2. Architecture of YOLOv4

Confidence loss is obtained by Eq. (3):

$\sum_{i=0}^{S^2} \sum_{j=0}^B 1_{i j}^{o b j}\left(c_i-c_i^{\wedge}\right)^2$     （3）

$c_i^{\wedge}$ gives the confidence scores of the box $\mathrm{j}$ in cell  'i'.

$1_{i j}^{o b j}=1$ if object is present  in $j^{\text {th }}$ bounding box of cell 'i' or else it is 0 .

No object is detected means confidence loss is obtained using Eq. (4).

$\lambda_{\text {noobj }} \sum_{i=0}^{S^2} \sum_{j=0}^B 1_{i j}^{\text {noobj }}\left(c_i-c_i^{\wedge}\right)^2$    （4）

$1_{i j}^{\text {noobj }}$ is complement of $1_{i j}^{o b j}$.

$c_i^{\wedge}$ is $\mathrm{j}^{\text {th }}$ box of cell 'i' confidence score.