A Hybrid Attention and Transformer-Based Deep Network with Multi-Scale Feature Fusion for Black Pepper Leaf Disease Detection

Agriculture is backbone of many economies across the world, and crop health plays a critical role in sustaining food security and farmer’s livelihoods. However, plant diseases are responsible for significant global crop losses every year, leading to a substantial decrease in yield, reduced food supply, and economic strain on farming communities [1, 2]. According to the Food and Agriculture Organization (FAO), nearly 20% to 40% of global crop production is lost annually due to pests and diseases, highlighting critical need for early and accurate disease detection mechanisms [3]. These plant diseases are primarily caused by a combination of biotic factors such as fungi, bacteria, viruses, pests, and abiotic stressors like nutrient deficiency, poor irrigation, and unsuitable soil conditions [4, 5]. Among these, fungal and bacterial pathogens remain the most common and destructive causes of plant health deterioration [6, 7]. Moreover, traditionally, farmers have relied on manual and experience-based methods to identify and predict plant diseases. This involves visual inspection of leaves, stems, and fruits, often guided by past farming knowledge or informal consultations with local experts. While this method is time-tested, it is highly subjective and prone to human error. In many cases, the disease symptoms appear similar, leading to misidentification. As a result, farmers may apply incorrect chemicals or pesticides, which not only fail to resolve the issue but also degrade soil health and contribute to environmental pollution. The lack of timely and accurate diagnosis leads to progressive crop damage, reduced productivity, and significant economic losses.

Among the various high-value crops, black pepper holds prominent position, particularly in India, which is one of the largest producers and exporters of this spice globally. Often referred to as the “King of Spices”, black pepper is widely used in culinary, medicinal, and industrial applications due to its pungent flavor and therapeutic properties [8, 9]. The spice has been cultivated in India for centuries, with Kerala, Karnataka, Tamil Nadu, and parts of the Northeastern states serving as the major cultivation zones. The crop thrives in tropical climates with high humidity and well-drained soil, but its growth and yield are heavily influenced by environmental and management conditions [10]. Despite its importance, black pepper cultivation faces severe threat from various diseases affecting its leaves, which are first visible indicators of underlying plant health issues. The diversity in soil types, use of different fertilizers, pesticides, and varying irrigation practices across regions contributes to a range of leaf diseases. Some of the most common ones include algal leaf spot, anthracnose, bacterial blight, chlorosis (due to iron or magnesium deficiency), nitrogen deficiency, slow wilt, quick wilt, and pest-induced damage [11, 12]. These diseases not only affect the quality of the leaves but also hinder photosynthesis and plant development, leading to reduced yield, poor spice quality, and sometimes total crop failure.

In recent years, the advancement of Deep Learning (DL) techniques has opened new avenues for automated and precise plant disease identification. Convolutional Neural Networks (CNNs) have become a popular tool for image-based disease classification due to their powerful feature extraction capabilities [13, 14]. Several works have utilized CNNs for identifying diseases in crops like rice, tomato, grapevine, and apple [15-17]. However, very limited research has focused specifically on black pepper leaf disease classification. The few existing studies primarily aim at disease classification, rather than early prediction or detailed feature analysis, which is critical for timely intervention and disease management [18-24].

Despite these advances, existing CNN- and transformer-based approaches still face important technical limitations in black pepper leaf disease detection. Most current models rely on a single-stream learning strategy that extracts features either from raw RGB images or from deep convolutional layers, without explicitly modeling disease boundary structures and fine-grained lesion morphology. Moreover, transformer-based models, while effective in capturing global context, often lack mechanisms to emphasize local structural cues such as vein distortions, edge irregularities, and lesion contours that are crucial for differentiating visually similar black pepper diseases. In addition, multi-scale symptom variations and background clutter remain insufficiently addressed by conventional architectures, leading to reduced robustness under real-field conditions. These unresolved challenges highlight the need for a unified framework that can jointly exploit edge-based structural information, attention-guided contextual features, and global dependency modeling for more reliable black pepper leaf disease identification.

To address these limitations, the present work introduces a novel model, Deep Attention-Based Transformer Network (DATNet) for accurate identification of black pepper leaf diseases. Unlike traditional CNN-based models, DATNet incorporates a dual-path architecture that not only captures low-level features like edges and textures through Sobel edge detection, but also learns high-level contextual information using Data-efficient Image Transformer (DeiT). Furthermore, it integrates Atrous Spatial Pyramid Pooling (ASPP) for multi-scale feature learning and Convolutional Block Attention Module (CBAM) to enhance the attention on disease-relevant features. By fusing features from both paths, DATNet provides a more comprehensive understanding of disease patterns, leading to improved performance in identifying diverse and complex disease types affecting black pepper leaves. This approach thus offers a robust, scalable, and efficient solution for early detection and effective management of black pepper diseases, ultimately supporting farmers with smarter and more sustainable agricultural practices.

The main novelty of this work lies in the proposed DATNet framework, which introduces a dual-stream learning strategy that explicitly separates structural and contextual feature representations for black pepper disease detection. Unlike conventional single-stream CNN or transformer-based models, DATNet integrates Sobel-based edge enhancement with transformer-based global modeling in one stream, while simultaneously employing ASPP and CBAM to extract multi-scale and attention-refined contextual features in a parallel stream. These complementary representations are fused to form a disease-aware embedding that captures both lesion boundaries and texture-level symptoms. This pipeline-level orchestration establishes a task-driven inductive bias for agricultural disease imaging and demonstrates how edge priors, attention refinement, and transformer self-attention can be synergistically combined under limited data conditions. The proposed model achieves superior performance over several state-of-the-art deep learning baselines, confirming the effectiveness of this integrated design.

The contributions of DATNet are presented below.

•A novel model DATNet is proposed for accurate identification of black pepper leaf diseases.

•A dataset has been collected, which consists of over 800 high-resolution images of both healthy and diseased black pepper leaves, captured from real-world farms in India.

•The model integrates both CNN and Transformer-based architectures using a dual-path approach to enhance disease feature learning.

•Techniques like Sobel edge detection, ASPP, CBAM and DeiT are utilized to capture both low-level and high-level features.

•The DATNet model significantly outperforms traditional DL models such as CNN, ResNet, DenseNet, Inception, and VGG16 in terms of accuracy, precision, recall, and F1-score.

•The work emphasizes not just classification but also early prediction of diseases, addressing a key limitation in previous research.

The manuscript is organized in the following manner. In Section II, the literature survey is discussed, where different DL-based approaches presented for black pepper are discussed. In Section III, the DATNet model is discussed in detail. Section IV discusses results of DATNet model compared with existing DL approaches. Finally, Section V presents conclusion and future work.

This section presents DATNet approach for black pepper disease detection which integrates preprocessing, edge detection for segmenting black pepper leaf, i.e., removing background and only considering leaf, feature extraction using attention-mechanism and transformers and finally prediction. For understanding complete process of DATNet, this work first presents architecture of DATNet, then the dataset collection process for evaluation of DATNet is discussed, then preprocessing steps, feature extraction process and prediction approach is discussed in detail.

3.1 Architecture

The architecture of DATNet is designed for accurate black pepper leaf disease detection. The model begins with a custom dataset comprising both healthy and unhealthy leaf images. Preprocessing steps include image resizing, RGB conversion, and pixel normalization. The architecture follows a dual-path structure as shown in the Figure 1. In the first stream, Sobel edge detection is applied for segmentation and for enhancing boundary features before feeding into DeiT for deep feature extraction. The second stream uses original image, which passes through an ASPP block to capture multi-scale features, followed by CBAM and DeiT for contextual refinement. Features from both streams are then concatenated. The combined features are passed through FCL dense layers with dropout to prevent overfitting. The final classification is achieved using a softmax layer, and performance is evaluated using the following metrics, accuracy, precision, recall, and F1-score.

Figure 1. Architecture of the proposed Deep Attention-Based Transformer Network (DATNet) model

Unlike conventional single-stream CNN or transformer pipelines, DATNet introduces a dual-path learning strategy that explicitly separates structural (edge-based) and contextual (appearance-based) representations before feature fusion. This architectural connectivity ensures that disease boundary cues and texture-based disease patterns are learned independently and later combined in a complementary manner. The novelty of DATNet lies not in the isolated use of Sobel, ASPP, CBAM, or DeiT, but in their coordinated integration as parallel feature abstractions guided by domain-specific disease characteristics of black pepper leaves, enabling robust discrimination under background clutter and illumination variations.

3.2 Dataset collection

In this work, for evaluation of DATNet, a dataset was collected from Kerala Agricultural University, Wayanad in India. The dataset comprised over 800 high-resolution images of black pepper leaves, captured using high-defining camera under natural-lightning conditions. The collected data included two classes, i.e., healthy and unhealthy black pepper leaves. Sample images from both classes are presented in Figure 2 and Figure 3.

The healthy leaves were free from any visible symptoms or signs of infection, representing ideal conditions. In contrast, the unhealthy samples exhibited a wide range of disease conditions, including quick wilt, algal rust, red rust, slow wilt, anthracnose, bacterial infections, chlorotic symptoms, magnesium deficiency, marginal gall, nitrogen deficiency, leaf patches, and various pest attacks. To ensure accurate labeling, all leaf images were first annotated by the authors and then verified by agricultural experts, Ms. Julie I. Elizabeth from Kerala Agricultural University, Wayanad, India. These experts carefully examined each image to confirm the presence and type of disease, ensuring the reliability and authenticity of the dataset. The dataset featured significant variability in terms of lighting, leaf orientation, background clutter, and contrast levels, thus posing realistic challenges for the detection model. This diversity ensured that trained model can generalize well to different environmental conditions and leaf appearances, thereby improving its robustness and practical applicability in field scenarios.

2.png

Figure 2. Healthy black pepper leaf images

3.png

Figure 3. Diseased black pepper leaf images

3.3 Preprocessing

The DATNet model has been trained using the image dataset, consisting of two classes, normal and diseased black pepper leaf images, which have been discussed in detail in above section. Every image underwent preprocessing, for ensuring compatibility with different DL approaches. The initial step in preprocessing included resizing the input image. Hence, all input images were resized to uniform spatial-dimension of 224 × 224 pixels. This dimension was chosen as it is optimal for transformer-based architecture, i.e., DeiT which has been used in this work, which expects fixed-size inputs. As the images are in color, 3-channel input, i.e., Red-Green-Blue (RGB) channel was considered, which helps in capturing more discriminative color features, also important for distinguishing diseased and healthy images. For standardizing input range and providing better gradients-flow during training, the pixel values of every image were normalized to [0,1] using Eq. (1).

$x^{\prime}=\frac{x}{255}$           (1)

In Eq. (1), $x$ denotes original pixel-value in range [0,255] and $x^{\prime}$ denotes normalized pixel value. This normalization process helps in accelerating convergence and prevents vanishing gradients during backpropagation. After preprocessing, the image is passed through an edge detection layer, which consists of a modified Sobel-filter for segmenting the leaf from background. The edge detection layer also enhances identification of contours of infected regions. In Sobel-filter, two convolutional kernels $G_x$ and $G_y$ are used for computing horizontal and vertical gradients respectively. The $G_x$ and $G_y$ is denoted as pixels as presented in Eq. (2) and Eq. (3) respectively.

$G_x=\left[\begin{array}{lll}-1 & 0 & 1 \\ -1 & 0 & 1 \\ -1 & 0 & 1\end{array}\right]$           (2)

$G_y=\left[\begin{array}{ccc}-1 & -1 & -1 \\ 0 & 0 & 0 \\ 1 & 1 & 1\end{array}\right]$           (3)

The convolution kernels for segmenting image and for edge detection is evaluated using Eq. (4).

$S(i, j)=\sum_{n=-1}^1 \sum_{n=-1}^1 I(i+m, j+n) \cdot G(m, n)$          (4)

In Eq. (4), $S(i, j)$ denotes new pixel-matrix after applying convolution I to G , where $G(m, n)$ denotes original pixelmatrix and $I(i, j)$ denotes pixel at position $(i, j)$. Using this operation, the DATNet approach enhances linear features like leaf vein and diseased area boundaries, helping in downstream feature extraction. The core of DATNet is based on DeiT model. Unlike CNNs, DeiT employs self-attention and operates on flattened image patches. Hence after edge detection, the image is passed to DeiT where image is split to non-overlapping patches and each patch is linearly projected. The patches are created using Eq. (5).

$P=$ Flatten $(\operatorname{Conv2D}(S(i, j))$          (5)

In Eq. (5), Conv$2 D$ layer has stride equivalent to patch-size which simulates patch extraction and projection. Further, each patch embedding processed considering self-attention using Eq. (6).

Attention $=\sigma\left(\frac{Q K^T}{\sqrt{d_k}}\right) V$          (6)

In Eq, (6), $\sigma$ denotes softmax process which is evaluated using Eq. (7) ensuring attention weights sum to $1, Q$ denotes query matrix, $K$ denotes key matrix and $V$ denotes value matrix, $d_k$ denotes dimension of key-vectors.

$\sigma(\vec{z})_i=\frac{e^{z_i}}{\sum_{j=1}^K e^{z_j}}$           (7)

In Eq. (7), $\sigma$ denotes Softmax, $\vec{z}$ denotes input-vector, $e^{z_i}$ denotes input-vector exponential-function, $e^{z_j}$ denotes output-vector exponential-function and $K$ denotes class, i.e., two in this work. After edge detection and extracting edge-based features using DeiT, a feature-map is achieved which has edge-based features. For extracting more features, the segmented image is passed on to next layer, which is discussed in detail in next section. 

This preprocessing stage establishes a strong structural prior by emphasizing disease boundaries and venation patterns before transformer-based representation learning. By converting raw images into edge-enhanced representations prior to patch embedding, DATNet ensures that the self-attention mechanism operates on diagnostically meaningful contours rather than background noise. This connectivity between edge extraction and transformer tokenization enables the model to encode spatial disease morphology more effectively than standard end-to-end RGB learning.

3.4 Feature extraction

For capturing features at multiple scales, especially for different kind of black pepper disease in unhealthy images, this work has utilized ASPP, which is semantic-segmentation approach for resampling given feature-layer at multiple dilation-rates before convolution. The ASPP using convolutions with varying dilation-rates $d_i$ is evaluated using Eq. (8).

$y_i=\operatorname{ReLU}\left(B N\left(\operatorname{Conv}\left(x, d_i\right)\right)\right)$          (8)

In Eq. (8), ReLU denotes Rectified Linear-Unit which is evaluated as $f(x)=\max (0, x), x$ denotes input feature map, $\operatorname{Conv}$ denotes convolutional-operation and $d_i$ denotes dilation rate $(1,6,12,18)$.In ASPP, this work has considered $d_i$ has been set as dynamic, such that it provides larger dilation rates, as it increases receptive-fields without additional parameters, making ASSP to detect context at multiple-scales. In this work, for efficient feature extraction, i.e., what and where to focus, this work has integrated CBAM in DATNet approach, which refines features using attention mechanism. The CBAM has two attention modules for capturing features, i.e., channel-attention module and spatial-attention module. The channel-attention module focusses on which feature-maps are important, by evaluating Eq. (9).

$\begin{gathered}M_c(F)=\rho(\operatorname{MLP}(\operatorname{AvgPool}(F)+\operatorname{MLP}(\operatorname{MaxPool}(F))))\end{gathered}$          (9)

In Eq. (9), $M_c$ denotes feature-maps extracted using channel-attention module, $F$ denotes feature-maps, $\rho$ denotes sigmoid function, MLP denotes Multi-Layer Perceptron, AvgPool denotes average-pooling operation and MaxPool denotes max-pooling operation. Similar to channel-attention module, the spatial-attention model focuses on which spatial-region are important, by evaluating Eq. (10).

$M_s(F)=\rho\left(f^{7 \times 7}([\operatorname{AvgPool}(F) ; \operatorname{MaxPool}(F)])\right)$            (10)

In Eq. (10), f denotes filter-size, where $7 \times 7$ filter-size has been set in this work. The final output achieved using CBAM is as presented in Eq. (11) and Eq. (12).

$F^{\prime}=M_c(F) \cdot F$           (11)

$F^{\prime \prime}=M_s\left(F^{\prime}\right) \cdot F^{\prime}$           (12)

Using Eq. (12), the ASPP-CBAM extracts feature maps, which then goes through DeiT process as presented in Eq. (5) and Eq. (6), such that the features can be fused, i.e., edge-based feature-map and ASPP-CBAM feature-map. Hence, in this work, the edge-based feature-map achieved using Eq. (6) is denoted as $F_1$ and the ASPP-CBAM feature-map achieved using Eq. (12) and goes through process of DeiT using Eq. (5) and Eq. (6) is denoted as $F_2$. From this the fused feature-map output is denoted using Eq. (13). 

$F=\left[F_1 ; F_2\right]$           (13)

In Eq. (13), [;] denotes concatenation of feature-maps. This feature fusion captures both edge-enhanced and context-rich features, improving prediction accuracy. Further, after feature extraction, the fused features are passed on to prediction layer, which is discussed in the next section.

The ASPP-CBAM-DeiT cascade enables DATNet to jointly model multi-scale contextual cues and attention-driven feature prioritization. While ASPP captures variations in lesion size and spread patterns, CBAM enforces selective emphasis on disease-relevant channels and spatial regions. The subsequent DeiT encoding preserves long-range dependencies between these refined features. This sequential connectivity transforms raw convolutional responses into structured disease-aware embeddings, providing a principled mechanism for feature refinement beyond simple concatenation.

3.5 Prediction

In this layer, the fused features are passed through FCL, as denoted using Eq. (14), Eq. (15) and Eq. (16).

$z_1=\operatorname{ReLU}\left(W_1 F+b_1\right), z_1 \xrightarrow{\text { Dropout }}$            (14)

$z_2=\operatorname{ReLU}\left(W_2 z_1+b_2\right), z_2 \xrightarrow{\text { Dropout }}$           (15)

$\hat{y}=\sigma\left(W_3 z_2+b_3\right)$           (16)

In Eq. (14) and Eq. (15), $z_1$ denotes first FCL layer, $z_2$ denotes second FCL layer, $W_1, W_2, W_3$ denotes weights and $b_1, b_2, b_3$ denotes bias. In Eq. (16), $\hat{y}$ denotes final output achieved using FCL layer. The $\xrightarrow{\text { Dropout }})$ helps in preventing overfitting. For training DATNet, Sparse Categorical Crossentropy loss was utilized which was evaluated using Eq. (17).

$L=-(y \log (p)+(1-y) \log (1-p))$          (17)

In Eq. (17), $y \in\{0,1\}$, which denotes ground truth label and $p$ denotes predicted probability of classes. The loss-function presented in Eq. (17) penalizes incorrect prediction more when model is correct, effectively optimizing performance of the DATNet approach.

The fully connected prediction layer operates on fused representations that encode both structural edges and contextual disease patterns. This fusion-based prediction differs from conventional single-stream classifiers by exploiting complementary modalities of visual evidence, thereby improving generalization under limited training samples.

3.6 Pipeline model

The complete processing pipeline of the proposed DATNet framework is illustrated in Figure 1, Figure 4, and Figure 5. The pipeline begins with preprocessing and Sobel-based edge extraction, which isolates leaf boundaries and disease contours by suppressing irrelevant background pixels. This step converts raw images into structural representations that are passed into the first DeiT stream (edge-stream), enabling the transformer to learn long-range dependencies among disease boundaries and venation patterns.

Figure 4. Processing of the Sobel edge feature map to DeiT framework

Figure 5. Process of feature extraction framework

In parallel, the original RGB image is processed through the ASPP module to capture disease patterns at multiple receptive field scales. This is followed by CBAM, which applies channel attention to determine what disease features are important and spatial attention to determine where they occur on the leaf surface. These attention maps, visualized in Figure 5, highlight infected regions such as chlorotic zones, necrotic patches, and rust-affected areas, thereby providing interpretability to the model’s decision-making process. Similarly, Figure 4 visualizes how Sobel-enhanced feature maps are tokenized and processed through DeiT self-attention, demonstrating the model’s focus on disease boundaries rather than background clutter.

The outputs of both streams are transformed into patch embeddings and encoded using self-attention, producing two complementary feature sets: edge-driven features ($F_1$) and context-driven features ($F_2$). These are fused through concatenation, forming a unified disease representation that integrates geometric structure with semantic texture cues. This pipeline design ensures that DATNet does not rely on a single representation modality, but instead exploits cross-domain feature synergy.

Given the relatively small dataset size of 800 images, overfitting risks are addressed through multiple mechanisms: (i) Sobel-based segmentation reduces background variability, (ii) ASPP increases receptive field diversity without increasing parameters, (iii) CBAM enforces selective feature suppression, (iv) dropout layers are applied in the fully connected stages, and (v) transformer attention enables parameter sharing across patches rather than pixel-wise memorization. Together, these design choices act as implicit regularization strategies.

Although DATNet integrates multiple known components, their pipeline-level orchestration constitutes a novel learning paradigm tailored for agricultural disease imaging. The proposed pipeline demonstrates how structural priors (edges), contextual encoding (ASPP), attention-guided refinement (CBAM), and global dependency modeling (DeiT) can be unified into a single coherent disease-detection framework. This architectural synergy justifies the effectiveness of DATNet beyond incremental improvements and establishes a transferable design principle for low-data agricultural vision problems.

In this work, a novel deep learning framework termed DATNet was introduced for automated identification and classification of black pepper leaf diseases. Beyond achieving high classification performance, the proposed approach demonstrates an important methodological insight: the explicit integration of structural edge information with attention-guided contextual representations significantly enhances the discriminative capability of vision-based disease detection systems. By combining edge-enhanced preprocessing, multi-scale feature extraction, channel–spatial attention refinement, and transformer-based global dependency modeling within a unified pipeline, DATNet effectively captures complementary visual cues related to lesion boundaries, texture variations, and disease spread patterns. The experimental evaluation on a real-world dataset collected from Indian black pepper farms confirms that this integrated learning strategy is well suited for agricultural imaging scenarios characterized by complex backgrounds and subtle inter-class variations. Rather than relying solely on deeper networks or increased parameter counts, DATNet introduces a task-driven inductive bias that guides the learning process toward diagnostically meaningful regions of the leaf surface. This highlights the practical importance of incorporating domain knowledge, such as disease morphology and boundary structures, into modern attention-based architectures. From a broader perspective, the findings of this study suggest that hybrid architectures combining convolutional feature extraction with transformer-based self-attention offer a promising direction for plant disease diagnosis, particularly under moderate data availability. The proposed design can be generalized to other crops and disease categories, making it a transferable framework for precision agriculture applications. Future work will focus on extending DATNet toward a hybrid disease classification and progression prediction system, enabling early warning and timely intervention in crop management. In addition, further validation using larger and more diverse datasets, along with statistical reliability analysis and field deployment studies, will be pursued to strengthen the robustness and real-world applicability of the proposed approach.