Remote Sensing-Based Estimation of Seedling Density in Nursery Gardens Using YOLOv4 Deep Learning Algorithm

Forestry's integration with information technology has emerged as a pivotal driver for the industry's advancement, marking a critical juncture for high-quality and sustainable growth within China's forestry sector. A paradigm referred to as Smart Forestry heralds a new era for the Chinese forestry industry, representing a significant milestone in the trajectory of contemporary forestry [1, 2]. In large and medium-sized nursery gardens, where vast expanses are populated with a tremendous number of seedlings, manual counting methods have proven both time-intensive and laborious. Traditional statistics-based fuzzy reasoning methods are marred by significant inaccuracies and intricate processes. Furthermore, the conventional quadrat sampling and survey methodologies are constrained by the criteria for quadrat selection [3, 4]. Consequently, the accuracy of these statistical outcomes has been found inadequate for the exacting standards of contemporary agricultural and forestry sectors. Within this framework, the fusion of UAV aerial photography with digital image processing technology is recognized as an essential tool for modern agroforestry surveys in this evolving context.

In the realm of image segmentation based on color attributes, Wang et al. [3] introduced a method capitalizing on various techniques, encompassing the transformation from RGB to Lab channels, separation of color channels, the 2D OTSU segmentation, and mathematical approaches for morphological rectifications. Qi [4], on the other hand, utilized a statistical approach, merging elevated angle images captured by drones with a cascade classifier rooted in Histogram of Oriented Gradient (HOG) features, tailored for enumerating spruce trees in nursery foundations. However, such conventional image processing methods are notably vulnerable to meteorological conditions, often compromising the accuracy and precision of recognition. In a distinct approach, Zhang et al. [5] presented a tree enumeration method employing the deep learning-based YOLOv4 target detection framework. Despite its capacity for accurate tree quantification, its computational intensity resulted in prolonged processing times.

A focus of the current investigation is the enumeration of Korean pines in nursery gardens using a deep learning-based target detection framework. Given the considerable variations in the size, color, and morphology of Korean pines, compounded by occlusions encountered during intensive detection phases, a lightweight Ghost module was integrated. This module, built upon the YOLOv4 target detection algorithm, facilitated dimension adjustments on features of the backbone network [6]. Subsequently, a lightweight attention module was devised, aiming to diminish background interference while augmenting the detection capabilities for dense and diminutive targets [6]. Finally, comparative analyses were conducted against the YOLOv3 model, the Faster R-CNN model, and the multi-scale segmentation method, reinforcing the proposed model's superiority [7].

3.1 Backbone network module

The YOLOv4 model contains many basic building blocks, and due to the large amount of convolution module computations, there are several feature maps, which increase the computational resources, leading to an increase in the model computation [8-10]. In the assembly of the feature pyramid, YOLOv4's primary feature extraction network, termed CSPDarknet53, channels three pivotal feature layers into the augmented feature extraction networks, specifically SPP and PANet [11, 12]. Given the extensive convolution calculations necessitated in this procedure, the operational speed is inherently compromised. To counteract this, GhostNet, which employs the Ghost module capable of generating analogous features with diminished parameters, was opted as a replacement for CSPDarknet53 in this research [12].

A delineation of GhostNet's architecture is provided in Table 1. This structure encompasses 11 sequences and spans across 21 network layers. Central to this design is the G-bneck, a residual module, with the Ghostmodule shaping the primary construct. It is discerned that the architecture of GhostNet predominantly consists of the G-bneck series [11]. From this configuration, three distinct effective feature layers were extracted: the inaugural effective feature layer (522×40), the secondary effective feature layer (262×112), and the tertiary effective feature layer (132×160). Upon the enhancement of this model, a notable reduction in its storage requirement was observed. Consequently, with GhostNet serving as the principal feature extraction network, a leaner model was realized, culminating in heightened detection efficiency.

Table 1. Architectural overview of GhostNet

3.2 The Dynamic Region-aware Convolution (DRConv) module

In the pursuit of accurately estimating seedling numbers through remote sensing image analysis and understanding the evolutionary trajectory of sown areas, an elevated precision in the detection model becomes imperative. Recognized for its capabilities, the DRConv is noted to automatically allocate filters, thereby enhancing detection precision. Concurrently, this mechanism is known to transpose channel filters into spatial dimensions, fortifying the representational capacity of convolution without incurring additional computational overhead.

For the objectives of this research, DRConv was integrated with GhostNet. Its inclusion was driven by two facets: necessity and superiority. On the necessity front, DRConv aptly satisfied the dual demands of amplifying seedling detection accuracy while judiciously managing computational expenditures. As for its superiority, not only did it bolster detection precision without compromising speed, but its inherent self-adaptive attributes, when synergized with GhostNet, showed a heightened adaptability specifically tailored for seedling detection within the realm of remote sensing imagery.

3.2.1 Overall structure

The structure of DRConv is given in Figure 2. Assuming: Z∈E^I×C×V represents the input of standard convolution; I, T, and V respectively represent height, width, and channel; A∈R^I×C represents spatial dimension, T∈R^I×C×P represents output, Q∈R^V represents standard convolution filter, * represents 2D convolution operation, then the feature mapping of the p-th output feature can be expressed as:

$T_{i, c, p}=\sum_{v=1}^v Z_{i, c, p} * Q_v^{(0)}(i, c) \in A$               (1)

In basic local convolution, assuming: Q∈R^I×C×P represents filters that are not shared in spatial dimensions, Q⁽⁰⁾_i,c,p represents a single non-sharing filter at the position of pixel (i,c) that is different from the standard convolution, then the following formula gives the expression of feature mapping of the p-th output feature:

$T_{i, c, p}=\sum_{v=1}^v Z_{i, c, p} * Q_{i, c, p}^{(0)}(i, c) \in A$               (2)

Based on above formula, the guided mask L={A₀,...,A_l-1} that can represent region division in spatial dimensions could be defined, wherein only one filer can be shared in each region, denoted as y∈ [0, l-1]. Let filter Q_y∈R^V correspond to region A_y, it can be assumed that the filter of the region is represented by L={Q₀,...,Q_l-1}, the v-th channel of Q⁽⁰⁾_y is represented by Q⁽⁰⁾_y,v, and any point in region A_y is denoted as (i, c), then the p-th channel of the output feature map of this layer can be written as:

$T_{i, c, h}=\sum_{v=1}^v Z_{i, c, v} * Q_{y, v}^{(0)}(i, c) \in A_y$               (3)

The implementation of DRConv has two main steps, including the setting of the learnable guided mask module and the filter generation module, and the functions of the two modules will be introduced in detail in the following text.

21-2.png

Figure 2. Structure of DRConv

3.2.2 The learnable guided mask module

Utilized within the architecture, the learnable guided mask module employs guided masks to segment spatial features into several distinct regions across spatial dimensions. This segmentation process has been observed to permit differentiated processing across these spatial regions, subsequently augmenting the model's adaptability and fortifying its competency in discerning intricate spatial configurations. Furthermore, it is through the guided mask that the filter distribution across spatial dimensions is determined. Filters are systematically allocated to their pertinent regions and are refined via the loss function. This procedural design enables filters to autonomously recalibrate their distribution in response to varying input spatial data, proving instrumental in the accurate capture of local image features and amplifying the model's proficiency in isolating and differentiating features within specific areas.

Specifically speaking, if a DRConv of size j×j contains l shared regions, then guided features with l channels could be constructed based on a standard convolution with a kernel size of j×j. Assuming: D∈R^I×C×l represents guided feature, L∈R^I×C represents guided mask, D_i,c represents the guided feature vector at position (i,c), then for each position (i,c) in the spatial region, there is:

$L_{i, c}=\arg \max \left(\hat{D}_{i, c}^0, \hat{D}_{i, c}^1, \cdots, \hat{D}_{i, c}^{l-1}\right)$          (4)

For filters [Q₀,...,Q_l-1], Q_Li,c was generated by H(·), assuming L_i,c represents the index describing the maximum channel size of guided feature D at position (i,c), then, the filter Q^{^}_i,c corresponding to position (i,c) can be attained:

$\hat{Q}_{i, c}=Q_{L_{i, c}}, L_{i, c} \in[0, l-1]=Q * L_{i, c}$              (5)

Through above operations, all positions established connections with filters, in the meantime, contextual similarity of pixels using the same filters reached the ideal state.

Then, a one-hot-form substitution D^{^} of the guided mask was introduced in the process of back propagation, then there is:

$\hat{D}_{i, c}^k=\frac{e^{D\,_{i, c}^k}}{\sum_{b=0}^{l-1} \,\,e^{D\,_{i, c}^b}} k \in[0, l-1]$              (6)

Through the processing of above formula, D^{^k}_i,c was made as close as possible to 0 or 1. Besides, Q^{^}_i,c can be regarded as a one-hot-form of filters [Q₀,...,Q_4l-1] multiplied by L_i,c and can be approximated as [D^{^0}_i,c,...,D^{^l-1}_i,c], further, the gradient of D^{^k}_i,c can be attained based on the following formula:

$\nabla \hat{D}_{i, c}^k M=\nabla_{\hat{Q}_{i, c}}, \cdots, \hat{D}_{i, c}^{l-1} k \in[0, l-1]$         (7)

Assuming: <  > represents the dot product, $\odot$ represents the element-by-element multiplication, ∇M represents the gradient of tensor with respect to the loss function, then the approximate back propagation is given by Eq. (5):

$\nabla D_{i, c} M=\hat{D}_{i, c} \Phi\left(\nabla_{\hat{D}_{i, c}} M-1\left\langle\hat{D}_{i, c}, \nabla_{\hat{D}_{i, c}} M\right\rangle\right)$             (8)

3.2.3 The filter generation module

The primary role of the filter generation module has been identified as the creation of tailored features, enabling the model to concentrate more accurately on distinct attributes of images. Such a design has been shown to augment both the detection precision and computational efficiency. Within each designated region, custom filters are produced by the filter generator to facilitate standard 2D convolution operations. Consequently, the model has been enabled to generate optimal filters tailored to the distinctive characteristics and demands of each specific region.

Figure 3 gives the structure of the filter generation module. Assuming: Z∈R^I×C×V represents input, Q=[Q₀,...,Q_l-1] represents l filters, by setting the downsampling of sample Z as j × j, l filters could be generated, and the size of their kernels was all j × j. Next, two consecutive 1 × 1 convolutional layers employing sigmoid(·) as the activation function and without activation function were adopted. By generating custom features, the model can focus more precisely on the specific features of each image. This not only enhances the model’s adaptability to complex structures and different types of images, but also improves its detection precision.

21-3.png

Figure 3. Structure of the filter generation module

3.3 Model optimization for enhanced lightweight implementation

In contrast to traditional convolutional methods, DRConv has been recognized for its capability to efficiently extract feature information, maintaining computational complexity and translational properties unchanged. Enhanced convolutional representation in spatial dimensions is believed to augment DRConv's comprehension and feature capture capabilities within images. With smaller models exhibiting diminished representational prowess due to limited parameters, DRConv has been devised to augment spatial information capture without a surge in computational complexity. Consequently, an enhancement in the performance of smaller models was observed. Empirical results further substantiated the efficacy of DRConv for these models, highlighting a notable precision enhancement. Furthermore, the capability of DRConv to autonomously assign multiple filters to spatial regions exhibiting analogous representations has been recognized. Such a design is speculated to fortify the adaptability of smaller models to diverse spatial features, amplifying their efficacy with intricate structures and diverse image types.

Given the distinct growth phases of Korean pines observed in nursery gardens and the homogeneity of the target background within captured images, the integration of an attention mechanism within deep networks was found to significantly influence vital image information. Emphasis was placed more discerningly upon the seedlings, facilitating superior feature extraction. Although various attention mechanisms exist, such as the Squeeze-and-Excitation (SE) [13], it was discerned that SE predominantly contemplates the intrinsic channel data, overlooking the pivotal role of locational information [14]. Based on this observation, the Convolutional Block Attention Module (CBAM) was posited to exhibit superior performance [15].

Motivated by experimental insights, a fusion of DRConv with CBAM was suggested, aiming at precision enhancement. Such a combined approach is posited to considerably elevate detection precision without compromising on processing speed. CBAM, having gained traction for target detection tasks, represents a composite attention mechanism. It facilitates feature map interconnections across channels and space. However, the CAM and SAM calculations intrinsic to CBAM are believed to introduce substantial redundancies [6]. Thus, CBAM was incorporated within one of the two pivotal feature layers emanating from the backbone network, hypothesized to influence target features without inducing excessive redundancy. CBAM encompasses two distinct modules: the channel attention module and the spatial attention module, as elucidated in Figure 4.

21-4.png

Figure 4. The CBAM attention mechanism

Within this figure, 'Conv' is indicative of convolution, 'GAP' symbolizes global average pooling, and 'GMP' denotes global maximum pooling.

3.4 Optimization of the loss function

Optimizing the loss function is essential to the advancement of machine learning models, especially when precision and adaptability are imperative. The loss function's role in attenuating the Loss signal through adjusted weight values is established [16]. Such adjustments enable the provision of feedback when mispredictions occur and subsequently enhance the model's convergence speed. In recent studies, the GIoU loss function was employed as a metric for location regression loss assessment [17], while other researches [18, 19] incorporated the GIoU loss as the primary loss function.

For a nuanced capture of both spatial and temporal variations and growth characteristics inherent in seedlings, alongside the enhancement of the model's detection precision, convergence speed, and generalization capabilities, a distinctive approach was undertaken in this analysis. The evolution value of the seedling area over time, as determined by the Gaussian plume model, was utilized as the corrected value. Subsequently, a YOLOv4 confidence loss function was crafted. Recognizing the disparate morphological and distributional nuances seedlings manifest across growth stages, the employment of the Gaussian plume model's evolution value for loss function correction was hypothesized to bolster the model's adaptability to varied seedling growth phases and ambient conditions.

Leveraging the Gaussian plume model, the density V(z,t,x,y) of the Korean pine forest within the nursery garden at any given temporal instance can be represented as:

$V(z, t, x, y)=S(z) r^{-s\left[z-i\left(y-y_u\right)\right]\,^2} r^{-n t^2} r^{-v G^2}$            (9)

The variance equations are:

$\left\{\begin{array}{l}\delta_z^2=\frac{\int_0^{\infty}\left[z-i\left(y-y_u\right)\right]\,^2 V d z d y}{\int_0^{\infty} V d z d y} \\ \delta_t^2=\frac{\int_0^{\infty} t^2 V d y}{\int_0^{\infty} V d y} \\ \delta_x^2=\frac{\int_0^{\infty} G^2 V d z}{\int_0^{\infty} V d z}\end{array}\right.$              (10)

Formula of the evolution source is:

$W=\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} i V d z d t d x d y$                (11)

Under the condition that the influence of time is not considered, constant terms s, n, and v can be calculated by the following equations:

$V\left\{\begin{array}{l}s=\frac{1}{2 \delta_z^2} \\ n=\frac{1}{2 \delta_t^2} \\ v=\frac{1}{2 \delta_x^2}\end{array}\right.$              (12)

By combining Eq. (9) with Eq. (11) and Eq. (12), the calculation formula of S(z) can be attained:

$S(z)=\frac{W}{2 \pi i \delta_z \delta_t \delta_x}$             (13)

Assuming: under an evolution source, z represents precipitation, t represents the supply amount of wastes, x represents sunlight intensity, y represents sunlight duration, then the density of seedlings can be written as V(z,t,x,y); S(z) represents the transition function, G represents the growth rate of seedlings; $\delta_z$, $\delta_y$, $\delta_x$ are evolution parameters in different directions; i represents the wind speed in the environment; W represents the degree of human intervention, then by combining Eq. (9) with Eq. (12) and Eq. (13), there is:

$V(z, t, x, y)=\frac{W}{(2 \pi)^{\frac{3}{2}} i \delta_z \delta_t \delta_x} \,\,e^{-\frac{\left[z-i\left(y-y_u\right)\right]\,^2}{2 \delta_z^2}} \,\,e^{-\frac{t^2}{2 \delta_t^2}} \,\,e^{-\frac{G\,^2}{2 \delta_x^2}}$              (14)

The modelling process in this paper only considered the evolution of seedling area over time, then the above formula can be simplified to:

$V(y)=\frac{W}{2 \pi i \delta_z \delta_t \delta_x} e^{-\frac{\left[i\left(y-y_u\right)\right]\,^2}{2 \delta_z^2}}$              (15)

From the analyzed video capturing the evolution of the seedling area, it was observed that the sown area for seedling targets rapidly attained a peak value, underscoring the significance of "primary seedling targets" within a specified temporal frame. For simulating the evolution trajectory of the seedling area, the Gaussian plume model was employed. Within the stipulated interval [1, 2], a corrected value was generated by the Gaussian function, reflecting the seedling area's temporal progression. This derived value was instrumental in amplifying the confidence score of prediction boxes encompassing targets identified proximate to the peak time. In essence, if targets were detected around or at the zenith of primary seedling target incidence, a bolstered confidence score was ascribed. Continuous iterative training ensured heightened model sensitivity towards the delineated "primary seedling targets". The integration of this corrected value fostered model refinement in a specific trajectory, culminating in a refined model adept at pinpointing primary seedling targets with augmented precision and confidence during real-time detection.

The confidence degree loss function is articulated as follows:

$\begin{aligned} & C O D_{L O(O B)} =\left\{\sum_{u=0}^{J \times J} \sum_{k=0}^L U_{u k}^{p n k}\left[V_u \log \left(\hat{V}_m\right)+\left(1-V_u\right) \log \left(1-\hat{V}_m\right)\right]\right\} * V \\ & +\sum_{u=0}^{J \times J} \sum_{k=0}^L U_{u k}^{N O}\left[V u \log \left(\hat{V}_m\right)+\left(1-V_u\right) \log \left(1-\hat{V}_m\right)\right]\end{aligned}$               (16)

In the present investigation, the enumeration of Korean pine seedlings was meticulously performed using a refined YOLOv4 model. Rapid identification of seedling targets was achieved, and their count was determined with precision. An impressive recall rate of 88.47 was observed for the model, underscoring its superiority compared to other prevalent target detection models. Within the scope of the experimentation, only test set images were subjected to predictions, and the dimensions of the predicted images remained congruent with the input dimensions. Comprehensive analysis involving the entirety of the sample set was not executed; therefore, ensuing studies are anticipated to delve into large-scale and high-resolution plant enumeration methodologies, potentially incorporating strategies like cropping, predictive analysis, and splicing of expansive regions.