Weakly Supervised Semantic Segmentation of Transmission-Line Insulators Using GrabCut Pseudo-Labels from Bounding Boxes and a Lightweight TransAtt-UNet

2.1 Research progress of insulator image segmentation

As an important research direction in power vision intelligence, insulator image segmentation has received extensive attention in recent years. Early studies mostly relied on traditional image processing methods, such as threshold segmentation, edge detection and morphological operations. Such methods are highly dependent on the background, with unstable performance under conditions of illumination changes and complex texture interference, and are thus difficult to adapt to UAV inspection scenarios [11].

With the development of deep learning, the FCN realized end-to-end pixel-level prediction for the first time, laying a foundation for semantic segmentation [12]. The U-Net structure proposed by Ronneberger et al. [13] effectively fuses shallow detailed features and deep semantic information through the symmetric encoder-decoder design and skip connection mechanism, achieving remarkable results in medical image segmentation. Due to its excellent multi-scale representation capability, this structure has been widely applied in industrial inspection and power equipment segmentation tasks.

In the field of power inspection, researchers have made various improvements to U-Net, such as introducing residual structures to enhance feature extraction capability, or combining Feature Pyramid Network (FPN) to improve small target recognition performance. In addition, Mask-RCNN has been applied to insulator instance segmentation, realizing joint optimization of target localization and segmentation through the region proposal mechanism. However, these methods mostly rely on a large number of pixel-level annotated data, resulting in high annotation costs in practical engineering applications.

The DeepLab series models have shown excellent performance in the field of semantic segmentation. DeepLabv3+ fuses features of different receptive fields through the Atrous Spatial Pyramid Pooling (ASPP) module, significantly enhancing multi-scale semantic representation capability [14]. Under complex background conditions, DeepLabv3+ has a stronger context modeling capability than U-Net, and is therefore widely used in power image segmentation tasks [15]. Nevertheless, such methods take convolution as the core, mainly focusing on local receptive field information, and have limited capability in modeling long-range dependencies [16].

2.2 Application of attention mechanism in segmentation tasks

The attention mechanism effectively improves the model representation capability by assigning higher weights to important features. In visual tasks, the attention mechanism is mainly divided into two categories: channel attention and spatial attention [17].

SENet enhances the capability of modeling the relationship between feature channels through the channel attention mechanism, but does not consider spatial dimension information [18]. Subsequently, CBAM (Convolutional Block Attention Module) proposes to combine channel attention and spatial attention simultaneously to realize a dual enhancement structure of channel-spatial. CBAM recalibrates the feature map through the sequential attention mechanism, which can effectively suppress background noise and strengthen target boundaries.

In target segmentation tasks under complex backgrounds, the attention mechanism can significantly improve the quality of boundary expression. Especially in UAV inspection images, insulators are often visually confused with structures such as conductors and towers, and the introduction of the attention mechanism helps the model focus on the target region and improve segmentation accuracy [19].

2.3 Application of transformer in visual segmentation

Transformer was initially applied in the field of natural language processing, with its core being the self-attention mechanism. Dosovitskiy et al. proposed Vision Transformer (ViT), which divides an image into fixed-size patches and inputs them into the Transformer encoder to realize global feature modeling [20]. This method breaks through the limitation of the local receptive field of convolutional networks and exhibits excellent performance on large-scale datasets.

However, ViT has high computational complexity. To improve efficiency, Liu et al. proposed Swin Transformer, which realizes self-attention calculation within local windows by introducing the sliding window mechanism and hierarchical structure, and performs window shifting between different layers. This not only ensures computational efficiency but also enhances global modeling capability [21].

In recent years, the fusion of CNN and Transformer has become a mainstream trend. One type of method introduces the Transformer module in the encoder bottleneck layer to supplement global semantic information; the other adopts a parallel branch structure, using the Transformer branch to extract high-resolution global features to complement the convolutional branch. Experiments show that the fusion structure can significantly improve target connectivity and structural integrity [22].

In the insulator segmentation task, targets usually present slender structures with repeated texture features. Relying only on local convolution is prone to segmentation discontinuity, and the introduction of Transformer can effectively enhance the overall coherence.

2.4 Research status of weakly supervised segmentation

Semantic segmentation is highly dependent on pixel-level annotations, while high-quality annotations are costly. In actual power inspection scenarios, the common annotation forms are bounding boxes or coarse-grained annotations, making weakly supervised segmentation an important research direction [23].

GrabCut is a classic graph cut algorithm that separates the foreground by constructing Gaussian mixture models for the foreground and background and based on the energy minimization strategy [24]. This method can generate relatively accurate foreground masks under the constraint of bounding boxes, and has high practical value in industrial scenarios.

In recent years, weakly supervised methods based on Class Activation Mapping (CAM) and self-training strategies have also emerged, but they usually rely on additional classification networks or complex training strategies, leading to high difficulty in engineering deployment [25]. In contrast, the method of generating pseudo-labels based on GrabCut and combining with deep segmentation network training features a simple structure and high reproducibility, making it more suitable for engineering promotion.

4.1 Model introduction

U-Net and its improved models are widely used in segmentation tasks such as medical imaging and industrial inspection. Their encoder-decoder structure can extract semantic information layer by layer and restore spatial resolution in the decoding stage, while skip connections fuse shallow edge textures with deep semantic features, thus balancing localization accuracy and semantic discrimination capability. Precisely because this structure shows stable performance in small-sample and fine-grained boundary scenarios, U-Net is often used as the basic framework for engineering segmentation tasks. However, in UAV inspection images, insulator targets have obvious scene characteristics: first, insulators usually present slender chain or string structures with repeated textures and regular arrangement inside the targets; second, variable inspection angles lead to significant target scale differences, with local occlusion, motion blur and strong light reflection being common; third, structures in the background such as tower components, conductors and insulator fittings are similar to insulators in shape or texture, which is prone to foreground-background confusion. Under these conditions, modeling relying solely on the local receptive field of convolutional networks is often difficult to stably capture the overall connectivity and long-range geometric relationships of targets, manifested as discontinuous segmentation results, boundary adhesion or background false detection, especially prominent in complex backgrounds and small target conditions.

Based on the above problems, this paper introduces Transformer and attention mechanism on the U-Net framework and proposes TransAtt-UNet to improve the segmentation robustness in complex inspection scenarios in the way of lightweight feature extraction + global relationship modeling + key detail enhancement. The specific structural design is as follows.

4.2 Network architecture

(1) Lightweight encoder: MobileNetV2 is adopted to replace the traditional ResNet-based encoders [10]. MobileNetV2 splits standard convolution into depthwise spatial convolution and 1 × 1 pointwise convolution by using depthwise separable convolution, greatly reducing the number of parameters and computational load; at the same time, it reduces information loss while ensuring representation capability through the inverted residual structure and linear bottleneck. For engineering applications such as UAV inspection, the lightweight encoder not only helps improve training and inference efficiency but also is more conducive to subsequent deployment on edge devices or inspection platforms.

(2) Transformer bottleneck: A Transformer Encoder module is introduced on the low-resolution and high-semantic features output by the encoder [13]. Since the computational complexity of low-resolution features is more controllable, it is suitable for carrying the global interaction of self-attention. Transformer establishes associations between arbitrary positions in the feature sequence through the multi-head self-attention mechanism, which can capture long-range dependency relationships and overall structural information. For insulator targets with the characteristics of slenderness, repetition and easy discontinuity, global modeling can enhance the ability to identify target connectivity and geometric consistency, and reduce false detection caused by similar local textures. Meanwhile, the multi-head mechanism allows the model to focus on different relationship patterns in different subspaces—for example, part of the attention focuses on the continuity of chain structures, and another part focuses on the differences from background conductors and towers, thus improving the discrimination robustness in complex backgrounds.

(3) Attention skip fusion: A channel-spatial attention module (e.g., CBAM) is introduced at the skip connections to recalibrate the shallow detailed features from the encoder [12]. The skip connections of the traditional U-Net are usually simple concatenation or addition, which easily introduce background noise into the decoder, leading to misclassification at boundaries or texture interference. CBAM first measures the importance of different feature channels through channel attention, and then emphasizes the response of key regions through spatial attention, which can suppress background structures similar to insulators (such as fittings, conductors and tower material textures) and enhance the response intensity of boundary and slender structure regions. In this way, skip connections no longer transmit details indiscriminately but selectively transmit effective details, improving the quality of boundary restoration and anti-interference capability in mechanism.

(4) Decoder: The spatial resolution is restored through progressive upsampling and convolution fusion. In each stage of the decoding phase, the upsampled high-level semantic features are fused with the shallow detailed features filtered by attention, and multi-scale information is further integrated through convolution to gradually refine the target boundary and restore the complete shape of the insulator, finally outputting a binary segmentation mask. This decoding strategy retains the advantages of U-Net in boundary restoration, and with the supplement of global semantics by Transformer and detail screening by attention, the model is more likely to obtain continuous, complete and low-noise segmentation results in complex inspection backgrounds.

In general, the design idea of TransAtt-UNet is as follows: the encoder uses a lightweight structure to ensure efficiency; the bottleneck layer uses Transformer to supplement the global relationship modeling capability; the skip connections use the attention mechanism to control noise transmission and detail enhancement; the decoder restores the high-resolution mask in multi-scale fusion. This combination not only faces the complexity and engineering deployability of UAV inspection scenarios but also provides a more robust network foundation for high-precision segmentation under weakly supervised conditions.

4.3 Loss function

To address the problem of pixel imbalance between the foreground (insulator) and background, this paper uses a weighted combination of BCE loss and Dice loss: L = λ·L_BCE + (1-λ)·L_Dice. The Dice loss can be defined as: L_Dice = 1 - (2·|P∩G| + ε) / (|P| + |G| + ε), where P is the predicted foreground set, G is the pseudo-label foreground set, and ε is the smoothing term.

Based on the experimental scheme designed in Chapter 5, this section compares the performance of three classic semantic segmentation baseline models on the insulator pseudo-label dataset, and conducts analysis combined with qualitative visualization results.

6.1 Quantitative comparison results

Table 2 presents the mIoU and Dice indicators of U-Net, SegNet and DeepLabv3+ on the validation set. All results are obtained under the same data division and training configuration.

Table 2. Quantitative comparison of different methods on the validation set

It can be seen from Table 2 that the three classic semantic segmentation networks can all learn a certain degree of foreground-background discrimination capability under the current weakly supervised setting, with mIoU and Dice indicators stably falling in the range of 40% ~ 70%. Specifically: The mIoU of U-Net is 51.50% and the Dice is 65.99%. Under the conditions of only a small number of training epochs and GrabCut pseudo-masks as the label source, U-Net can already give relatively usable insulator segmentation results; the mIoU and Dice of SegNet are 47.20% and 62.50% respectively, which are slightly lower than U-Net overall, consistent with the network structural characteristics of SegNet that mainly relies on pooling indices in the upsampling stage and has limited capability in restoring slender structures; DeepLabv3+ (ResNet-18) achieves the optimal values in both indicators in the current experiments, with an mIoU of 53.80% and a Dice of 68.50%. Compared with U-Net and SegNet, DeepLabv3+ integrates multi-scale context information more fully through atrous convolution and the ASPP module, and has stronger adaptability to inspection scenarios with complex backgrounds and obvious target scale variations.

6.2 Analysis of model performance

The experimental results of the three baseline models on weakly supervised pseudo-labels show that:

The GrabCut pseudo-label scheme is usable: even without introducing complex weakly supervised training techniques, classic CNN segmentation networks can already learn insulator segmentation results with practical engineering significance on this basis;

The network structure has a significant impact on weakly supervised performance: structures capable of modeling multi-scale context and global information (e.g., DeepLabv3+) are more robust under weakly supervised conditions, while structures relying on local textures are more susceptible to interference when pseudo-labels contain noise;

This baseline comparison also indirectly confirms the design motivation of the subsequently proposed TransAtt-UNet: by introducing the Transformer encoder and attention skip connections, it is expected to further strengthen global relationship modeling and key detail expression while maintaining lightweight, making up for the shortcomings of pure CNN architectures in weakly supervised scenarios.

6.3 Result discussion

Two empirical conclusions can be drawn from the comparison and ablation results. First, in the case of noise in pseudo-labels, the network is significantly sensitive to the quality of supervision: structures that can utilize multi-scale context or have stronger feature screening capabilities are usually more robust to pseudo-label errors. Second, insulator targets have the characteristics of slenderness and repeated textures, and the boundary regions are often the most vulnerable to background interference. Therefore, introducing the attention mechanism at the skip connections to control the transmission of detailed features, combined with the global relationship modeling of the bottleneck layer, helps improve the overall segmentation quality while maintaining lightweight. These observations also support the design motivation of TransAtt-UNet: under the constraints of engineering deployability, the combination of lightweight encoding + global modeling + attention detail enhancement is used to improve the usability of weakly supervised scenarios.