Lightweight Brain Tumor Detection in MRI Based on YOLO11 with Efficient Multi-Scale Attention and Wise-IoU Optimization

In recent years, with the continuous improvement of medical intelligence, brain tumor, as a neurological disease that seriously threatens human health, its early accurate diagnosis has become a key link in clinical diagnosis and treatment [1]. Medical image detection is the core means of brain tumor diagnosis, but traditional detection methods rely on manual image reading, which not only consumes a lot of medical resources, but also has problems such as poor real-time performance and diagnosis accuracy easily affected by doctors' experience, making it difficult to meet the needs of efficient preliminary screening in primary hospitals and accurate clinical diagnosis and treatment [2, 3]. At the same time, brain tumor image detection also faces multiple technical challenges: significant differences in tumor scales, overlapping image features of different types of tumors, and sample category imbalance in datasets, leading to traditional algorithms prone to missed detection, blurred boundary localization, and confused type discrimination.

In addition, the existing deep learning-based detection models usually have a large number of parameters and high computational costs, making it difficult to adapt to the deployment requirements of mobile devices and primary medical institutions. Therefore, developing an intelligent system integrating data balance processing, multi-scale tumor accurate detection and lightweight deployment is of great practical significance for improving the efficiency of brain tumor diagnosis and optimizing the allocation of medical resources.

In the field of medical image detection, target detection technology has evolved from traditional machine learning to the stage dominated by deep learning. Although traditional image processing methods have low computational overhead, their detection accuracy is insufficient under the interference of complex tumor morphology and image noise; deep learning-based algorithms such as the YOLO series and CNN have significantly improved feature extraction capabilities, but they still have problems such as insufficient capture of multi-scale features and impaired localization performance when facing the multi-scale characteristics and low-quality samples of brain tumors [4, 5]. In terms of model deployment, the existing algorithms generally have the problems of large number of parameters and high computational complexity, making it difficult to meet the operation requirements of edge devices in primary hospitals; at the same time, the traditional loss function has unreasonable gradient allocation for low-quality samples, which is easy to lead to the decline of model localization performance. Therefore, how to realize accurate detection of multi-scale tumors, robust fitting of low-quality samples under the condition of limited hardware resources, and complete model lightweight optimization to adapt to clinical deployment has become the core problem that this research needs to focus on solving.

To address the above problems, this paper carries out multi-dimensional technological innovation and algorithm optimization around the clinical needs of brain tumor medical image detection, and the main innovations are as follows:

(1) Propose an adaptive data processing scheme, solve the sample imbalance problem through category balance strategy and combined data augmentation technology, expand sample diversity by combining Mosaic and MixUp technologies, and improve the model's detection ability and robustness for tiny lesions [6];

(2) Introduce an Efficient Multi-scale Attention (EMA) module, which accurately captures multi-scale tumor features without dimensionality reduction through feature grouping [7], 1 × 1 and 3 × 3 parallel sub-networks and cross-spatial learning mechanism, establishes the dependency between local details and global morphology, and reduces the loss of tumor detailed information;

(3) Adopt the Wise-IoU loss function, reasonably allocate gradient gain through a dynamic non-monotonic focusing mechanism, weaken the harmful effects of low-quality samples on bounding box regression, and optimize the model's localization performance;

(4) Introduce the ShuffleNetV2 lightweight scheme [8], which greatly reduces the number of model parameters and computation within a controllable accuracy loss range through grouped convolution and depthwise separable convolution technologies, adapting to edge device deployment.

3.1 YOLO11 algorithm

YOLO11 is a target detection algorithm developed by Ultralytics. Compared with the previous generation YOLO models, YOLO11 has a prominent balance between lightweight and clinical practicality in brain tumor detection, which can meet the frame rate requirements of clinical real-time diagnosis. At the same time, through module optimization, its detection accuracy is significantly better than that of the same level previous generation models when dealing with complex backgrounds such as MRI image artifact interference and irregular tumor morphology, especially the recall rate of tiny lesions is significantly improved.

Nevertheless, YOLO11 still has problems in the small target detection of brain tumors, accuracy improvement and lightweight deployment. Therefore, this study improves YOLO11 to enhance its feature extraction and lightweight deployment capabilities, which can accurately identify tumor regions, boundaries and subtle features from brain tumor medical images [9]. Even in challenging scenarios with poor image quality and limited equipment, it can more accurately capture the complex features of tumors, meeting the needs of clinical auxiliary diagnosis. The network structure of YOLO11 mainly includes three parts: Backbone, Neck and Head, and the overall network structure is shown in Figure 2.

Backbone is a key component of the YOLO architecture, responsible for extracting features from input images of multiple scales. This process involves stacking convolutional layers and dedicated blocks, such as residual blocks and bottleneck blocks. These structures can effectively extract the deep semantic information of images, and generate feature maps of various resolutions through different step sizes and convolution kernel sizes, laying a foundation for subsequent multi-scale target detection and ensuring that the model can capture target features from tiny to large sizes.

Figure 2. Network framework of YOLO11

Neck fuses features of different scales and transmits them to the Head for prediction. This process usually involves upsampling and concatenation of feature maps from different levels, enabling the model to effectively capture multi-scale information. In this way, Neck can fuse shallow detailed features and deep semantic features, so that the subsequent Head can not only accurately locate the target but also accurately identify the target category when making predictions, improving the overall detection performance.

Head is responsible for generating the final predictions of target detection and classification. It processes the feature maps transmitted from the Neck, and accurately parses the target information contained in the feature maps through well-designed convolution operations and prediction branches. Finally, the Head outputs the bounding box coordinates of objects in the image, determines the category labels of the targets, and also gives the confidence score for the prediction results, providing a quantitative reference for the reliability of the detection results, thus realizing the accurate detection and classification of targets such as brain tumors.

3.2 Efficient Multi-scale Attention module

Accurate detection of brain tumors is the core link of neuroimaging diagnosis and clinical treatment decision-making, and its detection accuracy directly depends on the effective extraction and focusing of tumor region features. As a new generation of one-stage target detection algorithm, YOLO11 shows excellent performance in general target detection scenarios, but still has limitations when facing the complex features of brain tumor medical images: the traditional feature extraction module of YOLO11 is difficult to accurately model the diversity of tumor features, and is prone to feature focusing deviation.

As a core means to enhance the feature representation ability, the attention mechanism can focus on key regions and suppress redundant information through adaptive weight allocation, providing an effective idea for solving the feature modeling problem of brain tumor image detection. Different from the traditional static attention, the EMA module [10] can not only capture the global contextual features of tumor regions, but also finely depict local detailed features such as tumor edges and textures, effectively adapting to the diversity and variability of brain tumor features.

Before the proposal of the EMA attention mechanism, a variety of attention mechanisms have been tried in medical image detection tasks, but all have obvious limitations for brain tumor detection scenarios: the SE attention mechanism [11] only focuses on channel dimension dependency, lacks accurate capture of the spatial position information of lesions in medical images, and is difficult to adapt to the spatial heterogeneous features of lesions such as brain tumors; CBAM [12] combines channel and spatial attention, but has insufficient spatial feature capture ability in brain tumor MRI detection, and the localization accuracy of small lesions or lesions with blurred boundaries is limited. The EMA attention mechanism conducts refined learning of tumor multi-semantic features through feature grouping, the parallel sub-networks take into account channel correlation and multi-scale spatial feature extraction, and cross-spatial learning realizes feature fusion from global to local. It not only makes up for the deficiencies of traditional mechanisms in a single dimension such as spatial localization, fine-grained feature depiction or long-range dependency modeling, but also adapts to the lightweight characteristics of YOLO11, achieving a balance between accuracy and efficiency in brain tumor medical image detection.

3.2.1 Efficient Multi-scale Attention module

The EMA attention mechanism reconstructs the feature extraction and fusion module of the algorithm (Figure 3). In the EMA module, the 1 × 1 convolution of the CA module is selected as a shared component [13], and a 1 × 1 branch is constructed to accurately capture the correlation features between tumor channels; at the same time, the 3 × 3 convolution kernel is set in parallel with the 1 × 1 branch to form a 3 × 3 branch to quickly aggregate the multi-scale spatial structure information of tumors; combined with feature grouping and multi-scale structure design, the short-range and long-range dependency of tumor features is efficiently established. This improved scheme aims to fully combine the detection efficiency advantages of YOLO11, the feature modeling advantages of EMA attention and the computational efficiency advantages of parallel sub-structures, and ultimately achieve accurate focusing and efficient detection of brain tumor regions, providing more reliable algorithm support for clinical brain tumor auxiliary diagnosis.

Figure 3. Structure of Efficient Multi-scale Attention (EMA) module

(1) Feature grouping

There are different semantic features such as tumor core area and edema area in brain tumor images, and single-dimensional feature extraction is difficult to distinguish various semantic information. For this reason, the EMA mechanism divides the input feature map into G sub-feature groups in the channel dimension. Each sub-feature group focuses on learning a class of brain tumor semantic features, avoiding mutual interference between different semantic features, realizing refined modeling of multi-dimensional semantics of brain tumors, and laying a foundation for subsequent attention weight allocation.

(2) Parallel sub-networks

A large local receptive field of neurons is the key to capturing multi-scale spatial information of brain tumors. The EMA constructs a sub-network containing three parallel paths to extract the attention weight descriptor of the grouped feature map: two of the paths belong to the 1 × 1 branch, and the third path is the 3 × 3 branch, which optimizes the cross-channel information interaction mode combined with the characteristics of brain tumor image features (Figure 4).

Figure 4. Logical framework of parallel sub-networks

The 1 × 1 branch focuses on capturing the dependency between brain tumor channels and controls the computational cost at the same time. This branch adopts two 1D global average pooling operations to encode channel features along the two spatial directions of height and width respectively, and shares the 1 × 1 convolution kernel without dimensionality reduction to realize differentiated learning of different cross-channel interaction features and accurately depict the correlation intensity between tumor channels.

The 3 × 3 branch is aimed at the local spatial features of brain tumors, such as the local texture of tumor boundaries and the spatial distribution of small lesions, and only stacks a single 3 × 3 convolution kernel to capture multi-scale feature representations, expanding the feature space while avoiding excessive increase in computation.

(3) Cross-spatial Learning

Accurate detection of brain tumors needs to capture both pixel-level local features and global contextual dependencies, such as the spatial relationship between tumors and surrounding brain tissues. EMA draws on the cross-spatial learning idea of PSA attention [14] and designs a cross-spatial information aggregation method to realize the fusion of brain tumor features from global to local.

Firstly, for the output of the 1 × 1 branch, global spatial information is encoded through 2D global average pooling, as shown in Eq. (1).

$z_c=\frac{1}{H \times W} \sum_j^H \sum_i^W x_c(i, j)$              (1)

On this basis, the long-range dependency of brain tumors is established; secondly, the output of the 3 × 3 branch is also encoded by 2D global average pooling, and after converting the output of the 1 × 1 branch to matching dimensions, a second spatial attention map retaining accurate spatial positions is generated to capture the pixel-level pairwise relationship of tumors. Finally, in each feature group, the weight values of the two spatial attention maps are aggregated and activated by the Sigmoid function to highlight the global contextual information of tumor regions and suppress the interference of background such as normal brain tissues.

Finally, the EMA outputs a feature map with the same size as the input feature map, which can be directly embedded into the feature extraction module of YOLO11 without additional adjustment of the network structure dimension. This design not only realizes the accurate modeling of brain tumor channel features, multi-scale spatial features and global contextual features, but also maintains lightweight characteristics, effectively improving the focusing ability and detection efficiency of YOLO11 for brain tumor regions.

3.3 Wise-IoU

Optimizing the loss function can significantly improve the fitting ability and accuracy of the model. However, most studies are usually based on a premise—that the training samples are of high quality. This assumption leads to overfitting of the BBR loss function. In the field of target detection, the definition of the loss function for Bounding Box Regression (BBR) plays a decisive role in the improvement of model performance, especially on low-quality samples. If the BBR loss of these samples is improved indiscriminately, it may have a negative impact on the localization ability of the model. Focal-EIoU v1 [15] was proposed to solve this problem, but its focusing mechanism is static and does not fully tap the potential of the non-monotonic focusing mechanism.

Based on this view, Tong Z proposed a dynamic non-monotonic focusing mechanism (FM) and designed an IoU-based loss named Wise-IoU (WIoU) [16]. This project introduces the WIoU loss function, which reduces the competitiveness of high-quality annotated data and the harmful gradient generated by low-quality data, making the model focus more on brain tumor data of ordinary level.

3.3.1 Dynamic non-monotonic focusing mechanism

Aiming at the clinical practical problem of low-quality samples (such as image artifacts, annotation deviations, and samples with blurred tumor boundaries) in brain tumor medical image detection, the traditional bounding box regression loss function adopts a static gradient allocation strategy for all samples, which is easy to generate harmful gradients due to excessive attention to low-quality samples, or insufficient optimization for ordinary quality samples, leading to the decline of the model's localization accuracy for brain tumor regions. For this reason, Wise-IoU introduces a dynamic non-monotonic FM, abandons the fixed threshold setting of the static mechanism, and realizes dynamic quantitative evaluation of brain tumor sample quality by defining the outlier degree $\beta$, as shown in Eq. (2).

$\beta=\frac{\bar{L}_{I o U}^*}{\bar{L}_{I o U}}$                 (2)

This ratio $\beta$ reflects the deviation degree of the current anchor box relative to the average level. A smaller $\beta$ indicates a higher quality of the anchor box, and vice versa.

Combined with the sample distribution characteristics of brain tumor detection, FM designs a non-monotonic gradient gain allocation strategy based on the outlier degree $\beta$: assign a small gradient gain to high-quality brain tumor samples with small $\beta$ values to avoid overfitting of the model to high-quality samples; also assign a small gradient gain to low-quality brain tumor samples with large $\beta$ values to effectively weaken the harmful interference of low-quality samples such as artifacts and annotation deviations on bounding box regression; assign the maximum gradient gain to ordinary quality brain tumor samples with $\beta$ values in the middle interval, which account for the highest proportion and are the most clinically representative, making the model training focus on the clinical common brain tumor image features and improving the adaptability of the algorithm to real clinical scenarios.

Since low-quality samples are inevitably present in the training dataset, these samples may be over-penalized if processed by traditional geometric metrics, thus affecting the generalization ability of the model. An ideal loss function should reduce the dependence on these metrics when the anchor box and target box are well aligned, so as to reduce unnecessary intervention in the training process and further enhance the generalization of the model. On this basis, WIoU v1 with two-layer attention mechanism is designed, as shown in Eq. (3).

$L_{W I o U v l}=R_{W I o U} L_{I o U}, \quad R_{W I o U}=\exp \left(\frac{\left(x-x_{g t}\right)^2+\left(y-y_{g t}\right)^2}{\left(W_g^2+H_g^2\right)^*}\right)$                   (3)

Among them, $R_{ {WIoU }}$ is the distance attention factor, which is the square of the center point distance divided by the area of the minimum enclosing box. When the quality of the anchor box is general, $R_{ {WIoU }}$ will be large, amplifying the contribution of $L_{ {IoU }}$; when the quality of the anchor box is very good, $L_{ {IoU }}$ itself is small, the amplification effect of $R_{ {WIoU }}$ will be weakened, and the model will focus more on the center point distance factor.

The dynamic non-monotonic focusing mechanism WIoU v3 introduces a non-monotonic focusing coefficient $r$ on the basis of WIoU v1, as shown in Eq. (4).

$L_{W I o U v 3}=r L_{W I o U v 1}, r=\frac{\beta}{\delta \alpha^{\beta-\delta}}$                     (4)

Among them, $r$ is controlled by two hyperparameters $\alpha$ and $\delta$. It can be seen from the formula that $r$ is not a monotonically increasing or decreasing function. By changing the values of $\alpha$  and $\delta$, there can be an optimal value of $\beta$, denoted as C, so that the gradient gain $r$ reaches the maximum. For those samples whose outlier degree $\beta$ is exactly near C, the model will give the maximum attention. Since $L_{ {IoU }}$ is dynamic, the quality division standard of the anchor box is also dynamic, which enables WIoU v3 to make the most suitable gradient gain allocation strategy according to the current situation at every moment.

3.4 ShuffleNetV2 lightweight module

The clinical application scenarios of brain tumor medical image detection require the algorithm to have both detection accuracy and deployment flexibility. Edge-end hardware such as primary medical institutions and mobile diagnostic equipment has the problems of limited computing resources and insufficient storage capacity, while the original YOLO11 model has a high number of parameters and computation, making it difficult to meet the clinical lightweight deployment requirements. For this reason, this study introduces the ShuffleNetV2 lightweight architecture, reconstructs the Backbone and Neck parts of YOLO11 in a lightweight way combined with the feature extraction requirements of brain tumor images, and greatly reduces the number of model parameters and computation within a controllable accuracy loss range through core technologies such as channel splitting, channel shuffling and depthwise separable convolution, realizing the edge-end adaptation of the brain tumor detection algorithm.

ShuffleNetV2 [17] is an efficient and accurate lightweight convolutional neural network. ShuffleNetV2 inherits the Channel Shuffle operation of ShuffleNetV1 [18], which breaks the problem of information isolation between channels caused by grouped convolution, ensures that the multi-scale features of brain tumor images can fully interact, and achieves better model accuracy at the same time. In addition, it also introduces a Channel Split strategy, which divides the input channels into two parts: one part is directly transmitted, and the other part is processed through a series of convolutions, which can increase the diversity between features and improve the capture efficiency of the model for subtle features of brain tumors.

The first core network module of ShuffleNetV2 is shown in the left figure of Figure 5 which divides the input feature map in the channel dimension. The left branch performs the same mapping operation, while the right branch includes three consecutive convolutional layers with the same number of input and output channels. This design realizes feature reuse to improve the performance of the model. The second core module is the downsampling module, as shown in the right figure of Figure 5. This module does not use the channel splitting strategy, but directly inputs the feature map into two channels, and uses depthwise separable convolution with a step size of 2 for downsampling in each branch [19].

Figure 5. Structure of ShuffleNetV2

Among them, depthwise separable convolution is the core technology to realize lightweight, which splits the standard convolution into two independent steps: depthwise convolution and pointwise convolution. Depthwise convolution performs convolution operations on the brain tumor feature map of each channel separately to capture local spatial features; pointwise convolution completes information fusion between channels through 1 × 1 convolution to generate the final feature representation. Compared with standard convolution, this method can greatly reduce the number of model parameters and computation without significantly losing the key feature information of brain tumor images. The final improved model is shown in Figure 6.

Figure 6. Network framework of YOLO11-ESW

Focusing on the clinical pain points and technical bottlenecks of brain tumor medical image detection, this paper constructs a complete solution integrating data preprocessing, feature enhancement, localization optimization and lightweight deployment. Through multi-dimensional innovative improvements to the YOLO11 algorithm, it effectively solves the core problems such as sample imbalance, inaccurate detection of multi-scale tumors, interference of low-quality samples and difficult model deployment.

The research systematically solves the pain points of uneven sample distribution and difficult detection of tiny lesions in the brain tumor dataset through category balance strategy and combined data augmentation technology, providing a more diverse and representative data source for model training; the introduced EMA module realizes the accurate capture of multi-scale tumor features through feature grouping, parallel sub-networks and cross-spatial learning mechanism, greatly improving the model's ability to identify tumors with complex morphology and subtle lesions; the adopted Wise-IoU loss function optimizes gradient allocation through a dynamic non-monotonic focusing mechanism, significantly weakens the interference of low-quality samples and image artifacts, and improves the accuracy and stability of tumor boundary localization; finally, the improvement combined with the ShuffleNetV2 lightweight architecture realizes a significant reduction in the number of parameters and computation on the premise of controlling accuracy loss, successfully breaking through the deployment limitations of edge devices in primary medical institutions.

The results of four groups of targeted comparative experiments fully verify the effectiveness of each improved module. The improved YOLO11_EWS algorithm achieves a precision of 96.3% and a recall of 95.4%, with a significant improvement in detection accuracy compared with the original YOLO11. Meanwhile, the number of parameters is reduced by 41% and the computation is reduced by 43%, and its comprehensive performance is better than mainstream models such as YOLOv5 and YOLOv8. This algorithm can not only provide efficient technical support for early accurate screening of brain tumors and real-time intraoperative localization, but also adapt to resource-constrained scenarios such as mobile terminals and embedded devices, providing a feasible path for primary hospitals to optimize medical resource allocation and improve diagnosis efficiency, and has important clinical application value and promotion prospects.

Future research can further expand in three directions: first, explore the integrated application of weakly supervised and semi-supervised learning technologies to reduce the dependence on large-scale annotated datasets and adapt to the real scenario of high clinical data annotation costs; second, deepen the collaborative optimization of attention mechanism and lightweight architecture to further compress the model volume while improving the detection robustness of rare tumor types and lesions with extreme morphology; third, construct a multi-modal data fusion detection framework, integrate multi-source medical image information such as MRI, CT and PET, realize the integrated output of tumor benign and malignant identification and staging diagnosis, and provide more comprehensive support for clinical treatment decision-making.