Enhanced Detection of Electric Power Facilities Utilizing a Re-Parameterized Convolutional Network

In recent years, the management and planning of electrical grids have increasingly relied on digital twin technology, marking a significant advancement in modern power network infrastructure. The role of electric power facilities is central to the functioning of electrical grids, with the smart management of these facilities being crucial for the development of digital and intelligent grid systems. Research in computer vision has primarily focused on the real-time detection, identification, and categorization of objects in images. Utilizing UAVs equipped with geolocation modules and advanced edge computing models enables the integration of geographical and equipment data management, enhancing digital mapping and visual services within digital electricity networks. Despite this, there remains a scarcity of research focused on the detection of objects within common power facility environments. Implementing object detection algorithms can greatly benefit digital electricity networks, including intelligent detection of facility faults and digitalized management of data collection systems. Figure 1 illustrates how data from on-site measurements, digital mapping, scene management, and digital twins are integrated to form a comprehensive, modern, digitally intelligent power network system.

Conventional methods for object detection typically depend on pre-defined templates [1] or rely on geometrical shapes and prior knowledge [2, 3], while algorithms for object-based image analysis [4, 5] focus on segmenting and classifying objects. These traditional techniques often require manual data extraction, which complicates achieving precise detection results. Recent advancements in machine learning have led to the development of enhanced techniques for feature extraction from images [6, 7], including support vector machines (SVM), AdaBoost, and decision trees. These approaches are capable of identifying statistical characteristics such as the gray-level run length matrix (GLRLM) [8], histogram of oriented gradients (HOG) [9], bag of words (BOW) [10], local binary pattern [11], DCT coefficients [12], scale-invariant feature transform (SIFT) [13], and the deformable part model (DPM) [14]. The advent of deep learning algorithms, especially those based on CNN [15], has significantly advanced object detection, enabling the analysis of large datasets without the need for pre-existing templates and facilitating the recognition of complex patterns.

Deep learning's efficacy has been underscored by the success of CNN in tasks such as image classification, detection, and recognition, leading to the proposal of several high-performance deep network models like AlexNet [16], GoogleNet [17], and VGGNet [18], known for their straightforward yet deep structures. ResNet [19] introduced a novel residual structure, addressing the learning degradation issue present in very deep networks by simplifying the model's internal framework. Subsequent developments have introduced complex neural networks featuring skip connections and multi-branch architectures, which excel at feature extraction and are widely applied in computer vision. However, these increasingly complex structures can lead to unnecessary computational demands and reduced efficiency and interpretability. RepVGG [20], a simpler model that forgoes complex connections for a streamlined approach, matches ResNet's performance with faster inference times through the Rep-Conv-Block method, reflecting a shift towards models that balance accuracy with lightweight, rapid processing.

1.png

Figure 1. The relationship between on-site measurement data, digital mapping, scene management, and digital twin bases

The advancement of deep learning frameworks has significantly contributed to the evolution of methods for target detection. These algorithms are broadly categorized into one-stage and two-stage models. In one-stage models, object detection is achieved through direct regression of the target box, with default boxes pre-set based on the feature map. This category includes models like YOLO [21], SSD [22], and subsequent enhancements such as YOLOv3 [23] with its upsampling layer and layer-skipping concatenation, as well as YOLOv4 [24] and YOLOv5 [25], which incorporate various advanced techniques like expanding the receptive field, e.g., SPP-Net [26], RFB-Net [27], and attention mechanisms [28, 29], and YOLOx [30] that refine the prediction branch and candidate box approach. The two-stage models, initiated by R-CNN [31], pre-generate candidate regions through Selective Search before performing feature extraction. Building on this, Fast R-CNN [32] and Faster R-CNN [33] were developed to address the slow detection speed of candidate boxes, high training costs, and inefficiencies in the R-CNN by employing the RPN [34] and Feature Pyramid Network (FPN) [35] for enhanced feature fusion across different layers. Mask R-CNN [36] introduced a distinct binary mask branch for separate prediction, and improvements in alignment issues were made in Faster R-CNN with the adoption of RoIAlign over RoIPooling.

Mask R-CNN has demonstrated remarkable success in object detection, leading to the development of numerous enhanced versions. Innovations include the use of a non-quantization rounding pooling layer [37] to tackle issues related to the fixed count of interpolated pixels, and the creation of an improved ResNet-FPN [38] designed for preprocessing synthetically altered images. Beyond modifications to the architecture, further enhancements [39] have been made to optimize Mask R-CNN's loss function, refine the dimensional output of RoIs [40], and incorporate transfer learning strategies to improve performance on datasets with limited samples. Additionally, Mask R-CNN has been adapted to include RoIWarping [41] for better feature mapping, and a Light-Head subnet has been introduced to reduce the model's complexity. The development of Cascade R-CNN [42] employs a multi-stage cascading architecture and revises the IoU thresholding approach to enhance feature utilization.

This paper explores the identification of electric energy facilities by tailoring the Mask R-CNN model to a manually curated facility dataset for multi-scale object detection. Our contributions include creating a comprehensive dataset of electric energy facilities, segmenting this dataset by facility scale to enhance detection across different sizes, and implementing a Rep-Mask R-CNN model with innovations in model structure, such as adaptive anchors in RPN, a recursive FPN structure, and a re-parameterized backbone. Additionally, we employ transfer learning to address small dataset convergence challenges, achieving effective training outcomes by leveraging weights from larger datasets for objects of varying sizes and numbers.

In this study, we delve into the unique challenges of detecting electric energy facilities, characterized by their strong class homogeneity, significant variations across scenes, and size disparities among targets. To address these challenges, we enhance and tailor our dataset model accordingly. To ensure robust generalization capabilities, we compile a comprehensive dataset of electrical power facilities using web scraping and on-site photography. These images are meticulously labeled by hand, with each being classified and annotated with accurate bounding boxes and masks according to scale. This meticulous preparation enhances the model's ability to detect facilities of varying sizes with greater precision, especially when identifying smaller objects within large-scale images. To better match the anchor sizes to the actual dimensions of the targets, instead of defaulting to the conventional anchor ratios of 1:1, 1:2, and 2:1 for bounding box regression, we adapt the MS clustering algorithm. The process of our modified MS clustering algorithm is as follows: (1) Take the ratio of length to width of all ground truth boxes as the basis for clustering to initialize the sample space; (2) Initially determine a central point, and calculate vectors from all points to this central point within a specified step distance; (3) Compute the mean of all vectors within the space to find a mean offset value; (4) Shift the central point to this mean offset location; (5) Continue shifting the central point until it meets specific criteria.

The above process can be summarized in a mathematical form as follows:

$M(p)=\frac{1}{K} \sum_{p_i \in S_h}\left(p_i-p\right)$,      (1)

$p^{t+1}=M\left(p^t\right)+p^t$,      (2)

$p_i^*=\arg _p \min M\left(p_i\right)$,      (3)

where, $p_i$ represents all points in the sample space, $M$ represents the mean error, after several iterations, the result $p^*$ is the clustering result, which is the final choice of anchor ratio in RPN that best fits our current dataset.

A key enhancement within Mask R-CNN is the integration of the FPN. Opting for ResNet as its foundational architecture, FPN incorporates residual elements and skip connections, leveraging the strengths of ResNet's design. However, given the inherently complex nature of this two-stage model, we opted for a re-parameterized approach to simplify the architecture. This strategy avoids the need for the intricate skip connections and multi-branch configurations found in models like ResNet or the Inception modules, despite their proven efficacy in boosting model performance.

2.png

Figure 2. The method for combining a 3×3 convolution layer with a BN layer and calculating a tensor that performs like an identity operation to achieve similar fusion

In our deployment, we employ a model reconstruction strategy using re-parameterized layers. This approach differs from RepVGG's architecture, which typically includes a parallel arrangement of a 3×3 convolution layer and a 1×1 convolution layer, combined with the original data through summation. Instead, our design incorporates two parallel 3×3 convolution layers along with an identity path that includes a BN layer. These three paths are then summed together, ultimately integrating them into a single block for streamlined processing. We achieved this design via three steps, as follows:

First, we decouple the triple frame group and combine them as one block. We fuse the 3x3 convolution layer and BN layer as follows:

$r e p(x)=\gamma(W(x)+(x-\mu)) \sigma^{-1}+$ bias,      (4)

where, tensor x is the input, and the fusion result is calculated with convolution weight W and other convolution layer parameters. In Figure 2, the methodology for integrating the 3x3 convolution layer with the Batch Normalization (BN) layer is depicted, along with the approach for calculating a tensor that performs an equivalent identity operation to facilitate a similar fusion.

Subsequent to the vertical and horizontal merging processes, zero-padding is utilized to adjust the 1x1 convolution layer and the identity layer to a 3×3 format. These adjusted layers are then amalgamated with our redesigned 3×3 layer, resulting in two distinct module sequences: the 3×3-1×1 combined block and the 3×3-identity combined block.

$M^{(3,1,0)}=rep\left(M^{(i)} *\left(W^{(3)}+W^{(1)}+1\right), \mu, \sigma, \gamma, \beta\right)$,      (5)

And our re-parameterized convolution block replaces the 1×1 layer with 3×3 layers as follows:

$M^{(3,3,0)}=rep\left(M^{(i)} *\left(W^{(3)_1}+W^{(3)_2}+1\right), \mu, \sigma, \gamma, \beta\right)$,      (6)

where, $\mu, \sigma, \gamma, \beta$ represent the parameters in convolution layers, $M^{(i)}$ denotes the input tensor, $M^{(3,3,0)}$ and $M^{(3,1,0)}$ denote the outputs of the Rep Convolution block layer and the original Rep block, the weights of 3×3 layers, $W^{(3)_1}$ is differ from $W^{(3)_2}$ to get different results from the 3×3 convolution operation. Following the layering of modules with analogous structures, an adaptive average pooling layer is introduced to adjust the output size based on the input parameters. Consequently, the output from our layered feature mapping operation, facilitated by Rep-RPANet, serves as the input for the subsequent module.

Ultimately, Figure 3 illustrates the entire re-parameterization procedure of the Rep-Convolution-block, transitioning from a three-branch architecture to a Conv-BN setup, and thereafter converting to a unified layer structure, culminating in the model's reconstruction.

3.png

Figure 3. The re-parameterization of the Rep-Conv block involves transitioning from a three-branch architecture to a Conv-BN setup, eventually consolidating into a single-layer structure

The widespread adoption of the FPN framework is attributed to its superior capability in extracting features from objects across multiple scales [44]. Enhancements to the original FPN, such as cascade FPN, up-sample FPN, and recursive FPN [45], have been developed to further refine its performance. Despite the ability of a traditional FPN to extract features at multiple levels, it typically relies on the output from the k-th layer alone for RoI computations. The k is obtained as follows:

$k=\left|k_0+\log _2(\sqrt{w h})-\log _2 H\right|$,      (7)

where, w and h denote the width and height of the input image, respectively, with H typically set to 224. This is because large-scale RoIs are better extracted from low-resolution feature maps to enhance the detection of larger targets, whereas small-scale RoIs benefit from high-resolution feature maps for improved detection of smaller targets. Addressing the challenge of potentially losing low-level feature information with this distribution approach, we integrate low-level features with high-level features. Specifically, we introduce the RPANet by incorporating side connections within the FPN framework. This enhances the integration of multi-scale features at various levels. Furthermore, a two-level recursive structure is added to facilitate a second cycle of feature fusion. In this model, let n represent the number of recursion epochs, $C_i^{(n)}$ the output of the i-th stage of the model backbone (e.g., our Rep-blocks), $P_i^{(n)}$ the output of each layer in the top-bottom feature fusion path employing down-sampling to match the feature map sizes, and $N_i^{(n)}$ the output of each layer in the bottom-top feature fusion path using up-sampling to equalize the feature map sizes. The sum of these outputs serves as the input for the next recursion epoch or as the final outputs of RPANet. Should the n-th epoch be the final one, the complete model structure of RPANet is illustrated in Figure 4.

4.png

Figure 4. The structure of RPANet

The primary focus of this study is on the intelligent detection and identification of specific electric power facilities, crucial for on-site data measurement and management. In our research, we analyze the pros and cons of various architectural models for detection and introduce a Rep-Mask RCNN model equipped with RPANet, aimed at identifying electrical power facilities. In our proposed model, we utilize Rep-Conv blocks as the backbone with varied convolutional layer structures, replace the traditional FPN with RPANet, and strike a balance between computational efficiency and detection precision. Additionally, we constructed a dataset of electric power facilities from scratch, conducted anchor clustering, segmented the data according to the scale of target boxes, and applied transfer learning techniques to optimize detection outcomes across different target sizes. The experimental outcomes demonstrate that our Rep-Mask RCNN model not only achieves a high inference speed but also maintains robust capabilities in box detection and classification. Looking ahead, our future endeavors will focus on two main areas: firstly, expanding our dataset with more images captured under various conditions such as different lighting, weather, and shooting angles to enhance the model's generalizability; and secondly, addressing challenges associated with the Rep-Conv blocks backbone, such as the initial training issue of substantial loss, despite its effective performance enhancements. Our forthcoming work will explore refining the model structure, potentially incorporating attention mechanisms and transformer architectures, to further elevate detection performance and accelerate inference speeds.