Thresholding Value for Contour Segmentation Model in the Detection of Infected Plants with Drone Acquisition

Rice plants are a source of raw materials for rice, which is a staple food consumed by the wider community. The largest rice production in the world is still dominated by China, which is the largest rice producer in the world, followed by India, which is the second largest rice producer. Indonesia is ranked fourth as a rice-producing country globally with 6% of world rice production [1]. Thus, rice production requires special attention because rice is a staple food throughout the world and a touch of technology is needed to maintain rice production, especially during rice planting. Rice plants in the planting process are not free from pests and diseases, both pests and diseases in the category of light or heavy attacks [2]. The development of technology can be used to distinguish between rice plants infected with disease and healthy rice plants based on the image of rice plant leaves. This paper aims to detect infected rice plants based on leaf images acquired with a drone camera. The image of the infected rice plant is detected using digital image processing with the image contour segmentation method. In predicting infected plants, image segmentation aims to separate healthy plant areas and areas that show symptoms of infection.

Image segmentation involves dividing or splitting an image into segments with similar characteristics. The purpose of segmentation is to identify specific objects, boundaries, or features in an image, which will be used for further analysis in image processing. Segmentation begins by identifying the values of pixels that have certain characteristics such as color intensity, texture, and shape. Image segmentation of the contour method involves edge detection, one of which is edge detection canny by combining with some morphological operations and deletion noise [3]. Removal noise is used to process image color spaces and textures in the final classification process as per the selected detection [4]. This process is a technique for finding static threshold values to convert the RGB color space of the cooled image into the Hue, Saturation, Value (HSV) color space. Convert RGB to HSV by adding color features and geometric features to locate objects in the input imagery [5]. Image segmentation has been widely used in the formation of features in object detection in plants, including segmentation in determining the quality of oil palm [3], classification of tomato diseases [4], detection of diseases in apple plants [6], leaf diseases on plants [7, 8], detection of leaf diseases of biofuel plants [9] and also the health sector [10-12].

Segmentation in image processing can also be combined with several other methods in development for the early detection, recognition, and classification of specific objects. Segmentation often encounters obstacles when detecting objects caused by several things, including blurred image boundaries, unclear background colors, varying object sizes, color variations to small object sizes. To overcome this, the research conducted by Zhang and Zhang [13] modified U-Net for segmentation by introducing block residues (Resblock) and residue paths (Respath). Resblocks are inserted on the U-Net to overcome gradient and explosion loss and 2 Respaths are used in place of 2 connections to improve feature transformation.

Accurate and precise segmentation is essential for health especially since the segmented object has various angles, shapes, image brightness, and several factors that disrupt segmentation. There are 2 stages proposed to overcome this ha, namely by segmentation processing involving Convolutional Neural Network (CNN) [14] and secondly by using an attractive segmentation method [15]. Image segmentation can also be applied in determining lung scan images in covid-19 patients. The lungs in covid-19 patients have gray values similar to arteries, veins and bronchi. Each period's COVID-19 lesions vary in size and shape. A crucial first step in identifying and diagnosing COVID-19 is accurate lung segmentation on CT scans. With an average segmentation accuracy rate of 97.4%, the pulmonary parenchyma's segmentation accuracy has increased in both the contrast clarity of CT images and the consistency of the pulmonary parenchyma's regional features [11]. Threshold-based values in order to better understand the green hue of the leaves, the color index is employed to split the foreground and background of plant photos. Two fixed threshold techniques are suggested for the color index to differentiate between the background (soil) and the foreground (greenery). The experimental findings demonstrate that the proposed approach performs better than alternative algorithms when applied to plant images, achieving a classification ratio of 1.93 ± 0.05 and a segmentation error of 6.62 ± 5.85% [8].

The digital image acquisition process is the steps to obtain a digital image from its original source, such as through a digital camera, scanner, or other sensor. One of the simple digital image models is the binary image model or black-and-white imagery. This model uses only two-pixel values, namely 0 (black) and 1 (white). Binary imagery is suitable for describing images that have only two levels of intensity. The sensors in the imaging system are arranged to form a 2-dimensional plane with a magnitude of f(x,y) proportional to the energy emitted by the light source. Consequently, the magnitude f(x,y) cannot be zero and must be finite.

$0<f(x, y)<\infty$    (1)

The function f(x,y) can be separated into 2 parts, namely:

1. The amount of light coming from its source and symbolized by i(x,y) has a value between 0 and ∞.
2. The degree of the object's ability to reflect light r(x,y) has a value between 0 and 1.

In the context of digital image processing, the neighbors of a pixel are a group of pixels that are around a particular pixel. The term is often used in image processing operations such as spatial filters and edge detection techniques. The neighbors of a pixel are defined by the size of an area that is around that pixel. This area can be square or circular depending on the definition of the desired neighbor. This area size is referred to as the "kernel size" or "window size" (kernel/window size). The use of neighbors in image processing is very important because it allows for operations such as filtering, convolution, edge detection, and more.

Some operations require information from the pixel neighbor to generate Output which is right. A pixel p at coordinates (x,y) has 4 horizontal and vertical neighbors with coordinates (x+1,y), (x-1,y), (x,y+1), (x,y-1). This set of pixels is called the 4-neighbor of p, notated with N4(p). As for the 4 neighbors diagonally from p, ND(p) has coordinates (x+1,y+1), (x+1,y-1), (x-1,y+1), (x-1,y-1). On Picture 2.3 indicated pixel accuracy p. If ND(p) combined with N4(p) hence it is called the 8-neighbor of p, notated N8(p) [16].

1.png

Figure 1. Area and perimeter object shapes

A digital image that is composed of pixels to form an object and from these pixels will form an area or Area (A). The area can reflect the size or weight of the actual object on a solid object of almost the same shape, but this is not the case for non-solid or hollow objects. In addition to the term Area which indicates the area of the area, there is also the term Perimeter (P) to find out the length of the object's boundary or it can also be called the circumference (Figure 1). To calculate the perimeter size is calculated by calculating the number of pixels from the object constraint. Then there is one variable, namely compactness (C), which is an object measured through the analysis of dimensionless form factors with Eq. (2).

$\mathrm{C}=\frac{\mathrm{P}^2}{\mathrm{~A}}$     (2)

with:

C= Cohesiveness (Compactness)

P= Perimeter

A= Area

Compactness is used to identify the same shape and size of objects but with the fit of different pixel edges. For an object with flat edges, it will give a value of C of 4π, and if an object with uneven edges, the value of C will be even smaller. Another dimensionless form factor is the roundness of an object R, which is the comparison between the area of an object and the area of a circle that has the same perimeter value. For the calculation of the area of the plantation, use Eq. (3). Figure 1 shows the difference between the area and perimeter of a digital image.

 $R=\frac{4 \cdot \pi A}{P^2}$    (3)

with:

R= Roundness (kebundaran)

A= Area

P= Perimeter

Active contours in the image segmentation method are segmentations that apply a closed curve model and then will move wide or narrow according to following the desired object. The beginning of active contour segmentation was discovered by Kass et al. [17]. Active contour segmentation is also called snakes. Widening or narrowing the curve by minimizing the energy of the image using external forces and can be affected by the characteristics of the object in question such as the lines or edges of the object. The external energy in question can be formulated as in Eq. (4).

$E=\int_2^1 E_{\text {int }}(\vec{Y}(s)) d s+\int_2^1 E_{e x t}(\vec{Y}(s)) d s$     (4)

With:

E= energy that affects the active contour segmentation

Eint= external energy is influenced by the shape of the object

Eext= external energy that affects the curve in a widening and narrowing direction towards the desired object.

$\vec{Y}(s)$ = curve in two-dimensional space

Whereas Eq. (5) shows the formula for the internal energy search [18].

$Eint=\left\{\alpha(s)\left|\vec{Y}_{\mathrm{s}}(s)\right| 2+\beta(s)\left|\vec{Y}_{\mathrm{SS}}(s)\right| 2\right\} / 2$  (5)

Value α(s) and β(s)is a function that determines how a curve moves in space. While external energy is presented in Eq. (6).

${Eext}=|\nabla \mathrm{G}(Y(s) 2)|$   (6)

The G variable is the image to be segmented and this contour segmentation is interconnected and sequential points one point to another and controlled by a straight line. The determination of the points must be predictable as the contours of the initials and close to the desired shape of the objects. The contour of the initials will approach the object either wide or narrow due to the influence of external energy until it touches the outline of the object.

As a dataset used in this study, it is the result of the acquisition of drone cameras. Drone technology has often been used as a tool to solve several problems in life. Drone is also called the term unmanned aerial vehicles (UAV), unmanned aerial systems (UAS), unmanned aircrafts (UA) or remotely piloted aircrafts (RPA) [19]. With the advancement of new technologies, UAV platforms have become an ideal tool for monitoring crops with more precision due to their flexibility and low cost [20]. On drones for agriculture, sensors are installed that are usually directed towards rice fields to detect light reflected from the surface leading to the sensor. The intensity of the light captured by the sensor depends on the exposure from the sun to the surface and the efficiency of the light reflecting off the sensor surface [21]. Color sensor Red, Green, Blue (RGB) is the most common type of sensor used in drones. The sensor is able to capture visible light (wavelength 400-700nm) in the channel Red, Green, Blue (RGB) that are stacked on top of each other similar to human vision. Three channels are used to calculate the value of each pixel in the image produced by the sensor. Data from light sensors is often easier to identify qualitatively because hidden information is more familiar.

Images of rice plants produced by drones will be used as a dataset as input in determining the location or points of infected plants. Methods for detecting infected plants using molecular diagnostic methods such as polymerase chain reaction (PCR), rapid fragment length polymorphisms (RFLP), real-time PCR, loop- mediatedisothermal amplification (LAMP), recombinase polymerase amplification (RPA), and point-of-care diagnostic methods [22].

The technology that is developing today is the existence of the term Smart Farming make it possible to integrate information technology communication in traditional agriculture [23]. Technology has penetrated into various fields, including agriculture, for example. Internet of Things (IoT) [24, 25], Big Data analysis Remote Sensing [26], Machine Learning [27] and also Unmanned Aerial Vehicle (UAV) [21, 28, 29]. Use of UAV or drones in agriculture are able to reduce observation time so that they are more efficient [30]. Drones are able to assist farmers in observing plant conditions from the air which includes irrigation systems, soil varieties, pests and fungal attacks [31]. The resulting image of the drone has information in the infrared range and Visual Spectrum and images can be extracted to get certain information that cannot be known by looking with the eye.

The stage of calculating the area of infected plants based on contour segmentation starts from the image of the YOLO detection result and then converts it into a gray image and a binary image. In this binary image, the segmentation process is carried out because it will be easier to calculate the pixels with binary values of 0 and 1. Contour segmentation is an image processing technique used to determine the constraints of image objects. The results of this contour segmentation are represented in the form of a matrix to be converted from pixel shape to meters so that the area can be known based on the shape of the object.

The way contour segmentation works in determining the initial in segmentation process of infected rice plant objects is carried out by randomly generating the initial mask initials. The illustration of the initial calculation in this contour segmentation is to determine the initial mask point at random that surrounds the object. Figure 6 shows an object that has been converted to a binary image with the object represented by a binary color 0 (black). The binary image is an object that will be segmented by contour segmentation. After the initial points of this mask are raised, they will then be shifted closer to the edge of the object so that segmentation will be formed according to the desired object.

6.png

Figure 6. Contour mask initials dots

Table 1 shows the recapitulation of the coordinates (x,y) of the initials mask at the beginning of the image to be segmented. The shift of the initial point from the initial determination to the approach of the object to be segmented based on external and internal energy according to Eq. (5). Internal energy is obtained from the elastic energy between the point to be calculated with its adjoining point as well as the energy of the curve. As for external energy, it is obtained from the influence of the image to be segmented [18].

Table 1. Position of the initial point of the mask

In accordance with the energy equation in contour segmentation which states that energy comes from the addition of internal energy and external energy. The calculation of internal energy assuming that the value a_{of i} is 0.1 and constant for all i and also applies to b_i, then the internal energy value for the position of the 1st initial point can be known in the following way:

$\begin{aligned} & E_{\text {Int }}=\alpha_i\left(\left|P x_i-P x_i\right|^2+\left|P y_i-P y_{i-1}\right|^2\right)+\beta_i\left|P_{i-1}-P_i+P_{i+1}\right|^2 \\ & =0.1 \times\left(|1-0|^2+|1-8|^2+0.1 \times|0-1+0| 2+|8-1+6| 2\right) \\ & =5.0+17.9=22.9\end{aligned}$

Internal energy is already known, and then external energy is calculated, namely energy that is influenced by the image to be segmented. The search for external energy here is by calculating the distance between the first initial point and the nearest point of the segmented image using the Euclidean equation. In the case of the image in Figure 6, it is known that the initial point of the 1st mask has the 2 closest distances to the image pixel to be segmented, namely pixels (6.4) and (4.6), therefore the calculation of the distance between the 2 pixels is as follows:

$\begin{aligned} & \operatorname{Eeks}(\mathrm{Pl}(6,4))=\sqrt{(0-6)^2+(0-4)^2}=\sqrt{36+16}=\sqrt{52}=7,211 \\ & \operatorname{Eeks}(\mathrm{Pl}(4,6))=\sqrt{(0-4)^2+(0-6)^2}=\sqrt{16+36}=\sqrt{52}=7,211\end{aligned}$

The external energy from the two nearest points will be taken by one of the smallest values (7.211). Once both energies are known, the next step is to sum the internal and external energy values to find out the total energy value.

$E=E_{\text {int }}+E_{e k s t}=22,9+7,211=30,111$

The calculation to find out the energy of other initial points can use the same method, namely calculating the external and internal energy values. Thus, the illustration of the energy calculation method that has been carried out above will be used to calculate the segmentation of the contour in the image of infected rice plants. Figure 7(a) shows the original image of the infected rice plant which is an input dataset as a segmented image. Meanwhile, Figure 7(b) is the initial point of the mask as the basis for determining the energy in the contour segmentation process. Using Matlab 2020a software, the boundaries of the edges of the segmented object image are calculated. Segmentation here is carried out to find out the location of the infected rice plant area, therefore the initial points consist of several points due to the size of the infected rice plant area.

7.png

Figure 7. (a) Original image of the rice plant, (b) Image of initial mask

Active contour segmentation is a method in image processing that aims to identify and isolate certain objects in an image using adaptive mathematical curves. This method is also known as "Snake" or "Curve Evolution". Initialization of masks in image segmentation is the first step in determining the initial segmented area or region. Masks serve as a guide or initial clue for segmentation algorithms to begin the process of identifying and isolating the object in question from the background of the image. Once the mask is initialized, the segmentation algorithm can then be applied to improve or refine the results by identifying and isolating objects based on those guidelines. Figure 8(a) is a contour segmentation image in a binary imagery version. Binary color 0 (black) is the area of healthy rice plants while the image of binary color 1 (white) is the area of infected rice plants. Meanwhile, Figure 8(b) is the final result of the image of rice plants that have been segmented into contours. Segmentation images are the result of a merging operation between segmented binary images and native images.

8.png

Figure 8. (a) Segmented binary image, (b) Segmented image

The choice of mask initialization method depends on the characteristics of the image and the purpose of segmentation. The initial mask is combined with the original image to figure out the desired object, be it an RGB image or a binary image, using a subtract operation. The subtract operation itself is an operation to merge 2 images by subtracting or adding pixel values from the two images. Table 2 shows some of the segmentation results in infected plants.

Table 2. Results of segmentation of images of infected rice plants

Image	Initialization Mask	Binary Image	Image Segmentation
1a.png	1b.png	1c.png	1d.png

Once the limitations of the object are known, the next step is to represent the segmented binary image into a matrix. The binary pixel values of the segmented image are used to calculate the area of the segmented area. The pixel value of a binary image is easier to calculate than a color image. Next, the area and parameters are calculated to find out the circumference and area of the segmented object. However, in this case, the circumference and area of the object are still in pixels and are used as the first step in determining the area of the object in meters. The overall area and circumference results of segmented infected rice plants are shown in Table 3.

Table 3. Calculation of the area of segmentation results

In Figure 9, a graph is shown of an area of the infected plant image. From the graph, it can be known that the value of the infected area from all images of rice plants. The image of infected rice plants can be known by applying contour segmentation, but there are some images of rice plants that produce inappropriate area values or do not have infected areas only. This is because there are restrictions on healthy and infected areas that are not limited by a clear image. Based on the validation of the segmentation results by the agricultural office related to the object of research, the best results were obtained from the segmentation of infected images with a value of 97.02% and an inappropriate value of 2.80%.

9.png

Figure 9. Area of infected plant image

In the paper, we propose a new segmentation model called CTFFNet that combines Convolutional Neural Network (CNN) and Transformer for weed segmentation in remote sensing images of rice fields. This is based on a comparison of the contour segmentation results of infected rice plant detection, specifically by comparing it with research conducted by Guo et al. [36] Using Transformer and Residual CNN, we create two feature extraction modules that generate both local visual and global semantic features. In the feature fusion step, we employ an attention mechanism for dynamic feature selection and integration to merge these complementary traits efficiently. This aids in the model's automatic learning of the significance of various features for the adaptive fusion of local and global data. The segmentation performance is improved by the Intersection of Union (IoU) segmentation results, which yield a value of 72.8 [36].