Weakly Supervised Product Defect Image Segmentation and Visual Association Recognition for Quality Traceability

In this section, the correlation-driven weakly supervised defect parsing network, designated as ADW-Net, is systematically elaborated for quality traceability. Most existing weakly supervised defect segmentation studies have focused primarily on optimization at the loss function level. In contrast, a complete architectural design is conducted in this work from four dimensions: feature flow, data structure, geometric operators, and inference engine. The forward propagation of the network follows the feature transmission path of the input image, sequentially accomplishing multi-scale feature extraction via the backbone network, defect seed region generation, pixel-level feature rearrangement, and visual-semantic association embedding encoding, which further supports the construction of an offline retrieval database and the online inference pipeline. The entire network is trained relying solely on image-level category labels, collaboratively achieving pixel-level defect segmentation and quality traceability association recognition. Each module sequentially performs initial defect region mining, pseudo-mask refinement optimization, and cross-batch feature domain alignment, thereby facilitating the establishment of stable mapping relationships between defect visual representations and production traceability information within a shared feature space.

2.1 Overall architecture and feature forward propagation path

The overall architecture of ADW-Net and its multi-level feature flow diagram are illustrated in Figure 1. The encoding backbone of ADW-Net is implemented with a ResNet-50 with its classification layer removed, into which a hybrid dilated convolution strategy is incorporated to enlarge the effective receptive field while preserving the detail fidelity of high-resolution feature maps without increasing parameter count. Following forward propagation of the input image $\boldsymbol{X} \in \mathrm{R}^{3 \times 448 \times 448}$ through the backbone, feature maps $F_l \in \mathrm{R}^{C_l \times H_l \times W_l} \mid l=1,2,3,4$ are produced at four hierarchical levels, with spatial resolutions of 1/4, 1/8, 1/16, and 1/32 of the input image, respectively, and corresponding channel dimensions $C_l$ of 256, 512, 1024, and 2048. The low-level feature $F_2$ is rich in edge and texture details, whereas the high-level feature $F_4$ encodes semantic discriminative information. A natural response misalignment exists between these two levels when defect scales vary drastically, and direct interpolation-based fusion would disrupt the texture continuity in distorted regions. To address this issue, within the cross-scale bridging module, explicit spatial resampling of $F_4$ is performed toward $F_3$ alignment using deformable convolution v2, for which the offsets $\Delta p$ are regressed by a lightweight sub-network from the channel-wise cross-correlation response maps between $F_4$ and bilinearly downsampled $F_3$, with a convolution kernel size of 3 × 3 and the number of offset groups set to 4 to balance modeling capacity and computational cost. The aligned high-level features and the shallow features are each passed through independent 1 × 1 convolutions to compress the channel dimensions uniformly to C' = 256, and are subsequently fed into the gated fusion unit. Rather than employing a fixed weighted averaging scheme, this unit adaptively determines the contribution ratio of deep and shallow features according to defect scale, computed as:

$\widetilde{F}_{{fuse}}=\sigma\left(W_g \cdot\left[{up}\left(F_2\right), {up}\left(F_3\right), F_4\right]\right) \odot {up}\left(F_2\right)+\left(1-\sigma\left(W_g \cdot\left[{up}\left(F_2\right), {up}\left(F_3\right), F_4\right]\right)\right) \odot F_4$                  (1)

where, up( ) denotes bicubic interpolation, which unifies the spatial resolution while preserving high-frequency edge information; $W_g \in \mathrm{R}^{C' \times 3 C'}$ is a learnable channel-wise weight matrix; and σ( ) is the sigmoid function, which maps the gating values to the interval (0,1). Through this gating mechanism, a soft switching between detail-dominated and semantics-dominated modes is enabled within the network: F₂ contributes texture primitives for fine boundary localization, while $F_4$ contributes class-discriminative semantic context, with the fusion ratio automatically optimized via backpropagation according to the inherent scale of the defects.

1.png

Figure 1. Overall architecture of ADW-Net and multi-level feature flow diagram

The segmentation decoder is implemented with a modified DeepLabv3+ architecture as the upsampling backbone. The inherent grid effect of parallel atrous convolutions in the standard atrous spatial pyramid pooling module introduces periodic artifacts, which are particularly detrimental to the recovery of fine linear defects. In this work, the standard atrous spatial pyramid pooling is replaced with a densely connected atrous spatial pyramid pooling module: four atrous convolutional layers with dilation rates of 3, 6, 12, and 18 are stacked in a dense skip-connection manner, where the output of each layer is concatenated along the channel dimension and fed into the subsequent layer, enabling the receptive field to be continuously expanded from 33 × 33 to 129 × 129, thereby covering multi-scale context ranging from local textures to global shapes. Information isolation among parallel branches is avoided through this densely connected structure, while aliasing artifacts induced by multiple dilation rates are eliminated via a terminal 1 × 1 channel fusion layer. A sub-pixel convolution module is embedded in the upsampling chain with an upsampling factor of 2, through which the spatial resolution is enlarged while channel-dimensional information is reorganized into the spatial dimension to preserve high-frequency details. This branch is combined with a bilinear interpolation branch via element-wise addition, achieving a balance between overall contour smoothness and local edge sharpness. Following the processing through the aforementioned decoder pipeline, the final segmentation prediction $\widehat{M} \in \mathrm{R}^{1 \times 448 \times 448}$ is output, with spatial resolution strictly consistent with that of the input image. Through this overall forward propagation path, an end-to-end differentiable mapping from the raw image to pixel-level prediction is established, providing a multi-scale feature foundation for the subsequent seed region generation and contrastive refinement stages.

2.2 Multi-scale progressive activation seed generation

Under the supervision of image-level category labels y only, this stage is designed to extract reliable defect seed regions from multi-scale feature maps, thereby providing initial localization priors for the subsequent contrastive refinement. The mechanism of multi-scale progressive activation and seed generation is illustrated in Figure 2. For each scale $l \in 2,3,4$, an independent classification branch is established with global average pooling and fully connected layers, and the classification weight is denoted as $w_l^c \in \mathrm{R}^{C_l}$, where $c$ corresponds to the defect category. To eliminate numerical discrepancies in feature magnitudes across different scales, L2 normalization is introduced into the computation of the initial activation map:

$A_l^c=\operatorname{ReLU}\left(\frac{\sum_{k=1}^{C_l} w_{l, k}^c \cdot F_{l, k}}{\left\|w_l^c\right\|_2+\epsilon}\right)$                   (2)

where, $\epsilon=10^{-8}$ is used to prevent division by zero, and ReLU is applied to suppress negative responses. Unlike existing methods that employ a fixed Top-k percentage threshold, a dynamic decay mechanism is introduced in this work: the positive region selection ratio $k_t$ is linearly decayed from 30% to 10% over the course of training epochs, with the threshold $\theta_{{pos}}^l={mean}\left({Top}_{k_t}\left(A_l^c\right)\right)$, such that the seed regions are progressively tightened as the model discriminative capacity improves. Concurrently, a variance suppression term ${Var}\left(A_l^c\right)$ is introduced for the non-Top-k regions, through which penalties are imposed on isolated pixels with drastic response fluctuations within local neighborhoods, thereby preventing spurious high responses caused by single-point noise. Through this dynamic strategy, it is ensured that the seeds maintain sufficient coverage in the early training stages to guide contrastive learning, while high-precision positive samples are leveraged in the later stages to drive boundary refinement.

The deep activation map $A_4^c$ encodes sufficient semantic discriminative information; however, its spatial resolution is merely 1/32 of the input image, causing defect boundaries to become smoothed and blurred. Although the shallow activation map $A_2^c$ achieves precise boundary localization, it tends to misclassify background textures as defect regions. To unify semantic correctness and boundary completeness, a bidirectional structural consistency distillation loss is designed in this work, through which the deep and shallow activation maps are constrained to converge toward each other. Unlike pixel-wise constraints that rely solely on mean squared error, this distillation incorporates two complementary criteria. The first is a structural similarity constraint, in which the structural similarity index measure between $A_2^c$ and the upsampled $A_4^c$ is computed within 11 × 11 local windows, compelling the spatial layout of shallow activations to align with deep semantics. The second is an edge gradient magnitude consistency constraint, in which the spatial gradient magnitudes of the activation maps are computed and the upsampled deep results are constrained to produce gradient responses at defect boundaries that are as sharp as those from the shallow layer. This bidirectional constraint is formalized as:

$L_{ {cons }}=\left(1-\operatorname{SSIM}\left(A_2^c,{up}\left(A_4^c\right)\right)\right)+\left\|\nabla A_2^c-\nabla u p\left(A_4^c\right)\right\|_1$                (3)

where, ∇ denotes the spatial gradient computed via the Sobel operator. Through backpropagation of this loss, the network parameters generating both $A_2^c$ and $A_4^c$ are simultaneously updated, such that the deep features receive detail injection derived from the shallow layer at boundaries, while the shallow features receive category guidance derived from the deep layer in semantic regions, thereby forming a mutually beneficial evolutionary collaborative mechanism.

After spatial resolution unification, the three-scale activation maps are fused as:

$\widetilde{A}={Avg}\left(u p\left(A_2^c\right), u p\left(A_3^c\right), A_4^c\right)$                (4)

where, up( ) denotes bicubic interpolation, through which $A_2^c$ and $A_3^c$ are upsampled to the same spatial dimensions as $A_4^c$. The fused result is fed into a fully connected conditional random field for refined edge correction, in which the pairwise potential employs a contrast-sensitive kernel, with Red–Green–Blue (RGB) color difference $\theta_{r g b}$ = 10 and spatial Euclidean distance $\theta_{x y}$ = 3 taken into consideration, the compatibility matrix is implemented with the Potts model, and iterative optimization is performed for 5 iterations. In the confidence map after conditional random field refinement, pixels with confidence above 0.7 are assigned to the positive seed region $P_{ {seed }}^{ {pos }}$, pixels below 0.3 are assigned to the reliable negative background $P_{ {seed }}^{ {neg }}$, and ambiguous pixels between these two thresholds are explicitly ignored in the subsequent contrastive loss computation. Through this ternary strategy, the noise introduced by low-quality pseudo-labels is effectively suppressed, providing reliable spatial priors for the contrastive refinement in Section 2.3.

Figure 2. Mechanism diagram of multi-scale progressive activation and seed generation

2.3 Dynamic contrastive representation refinement and pixel-level feature rearrangement

Although the seed region S provides the approximate spatial extent of defects, its internal composition is inherently plagued by holes and boundary irregularities due to the discriminative localization nature of classification networks, rendering it unsuitable for direct use as the final segmentation mask. At this stage, pixel-level contrastive learning is introduced, through which pixel representations of the same category are pulled closer together and those of different categories are pushed apart in the feature space, thereby refining the coarse seeds into pseudo-masks $M_{ {pseudo }}$ with filled interiors and clear boundaries. The scale and quality of negative samples are crucial for contrastive learning, as a sufficiently large number of negative examples enables the complete manifold of the background distribution to be characterized, compelling defect features to aggregate toward the positive prototype. To this end, a two-level storage architecture is constructed for the effective management of large-scale negative sample queues. The hierarchical cascade storage and contrastive representation refinement are illustrated in Figure 3. The L1-graphics processing unit resident buffer stores the projected features of all pixels within the current mini-batch, with a batch size of 16, a feature dimension of 2048, and a feature map spatial size of 16 × 16, totaling approximately 32,768 feature vectors, which are directly involved in the current loss computation. The L2-central processing unit circular buffer maintains an additional 32,768 historical features in host memory, with asynchronous direct memory access transfers overlapping with the graphics processing unit computation pipeline to prefetch the next batch of data during backpropagation, thereby keeping graphics processing unit memory consumption within acceptable limits while ensuring a total negative sample queue length $N_{{neg }}$ of 65,536. Upon enqueueing of new features, a diversity-prioritized strategy is adopted, in which the maximum distance between the new sample and all existing features in the queue is computed, and enqueueing is permitted only when this distance exceeds a dynamic threshold $\delta_{ {queue }}=0.5 \cdot {Var}(Q)$, where ${Var}(Q)$ denotes the variance estimate of the current queue features. Through this mechanism, redundant similar samples are eliminated, and the effective representational capacity of the queue is improved by approximately 40% compared with a simple first-in, first-out strategy.

Figure 3. Diagram of hierarchical cascade storage and contrastive representation refinement

The contrastive refinement relies on a positive prototype to provide a stable clustering center. The positive prototype $c_{ {pos }} \in \mathrm{R}^{2048}$ is updated via exponential moving average with momentum:

$c_{{pos }}^{(t)}=m \cdot c_{ {pos }}^{(t-1)}+(1-m) \cdot \bar{z}^{(t)}$                 (5)

where, $\bar{z}^{(t)}=1 /\left|p^{ {pos }}{ }_{ {seed }}\right|$ is the mean of the projected features of all pixels within the positive seed region in the current mini-batch, and the momentum coefficient m = 0.999 controls the update smoothness. In the early training stages, extreme samples may dominate the mean direction, leading to prototype oscillation. To address this issue, second-order momentum correction is introduced, through which the prototype update magnitude during the first 50 epochs is reduced by a factor of $1 / \sqrt{1-m^t}$, thereby effectively suppressing early random fluctuations. A hard example mining strategy is focused on pixels within a three-pixel bandwidth around the seed region boundaries, where the feature entropy is highest and discriminative uncertainty is maximal. The contrastive loss is assigned adaptive weights $\omega_i=1+exp \left(-\gamma \cdot {entropy}\left(z_i\right)\right)$, where γ = 2.0 controls the weight scaling range. Integrating the above designs, the pixel-level contrastive loss function is formulated as:

$L_{{cont }}=-\frac{1}{\left|P_{ {seed }}^{ {pos }}\right|} \sum_{{i} \in P_{{seed }}^{ {pos }}} \omega_i \cdot \log \frac{exp \left(z_i \cdot c_{ {pos }} / \tau\right)}{exp \left(z_i \cdot c_{ {pos }} / \tau\right)+{\sum}_{j=1}^{N_{ {neg }}} exp \left(z_i \cdot n_j / \tau\right)}$                 (6)

where, $n_j$ denotes the j-th background pixel feature in the negative sample queue, and τ = 0.07 is the temperature coefficient that controls the sharpness of the softmax distribution.

The generation of pseudo-masks is performed via a hard update strategy rather than online soft updating. Specifically, at intervals of T = 5 training epochs, the latest segmentation predictions on the validation features from the current model are utilized, and a complete new set of pseudo-masks $M_{ {pseudo }}$ is regenerated through conditional random field refinement and confidence threshold filtering, replacing the old labels from the previous stage for subsequent training. Through this hard update mechanism, the continuous gradient coupling between pseudo-labels and model parameters is severed, effectively preventing the cumulative amplification of minor boundary errors over prolonged iterations. If online soft updating were adopted, only local corrections to the previous round of pseudo-masks would be made each time, allowing errors to slowly propagate along the temporal dimension. In contrast, the complete regeneration relies on the current model's overall perception of the global feature distribution, endowing it with a stronger self-correction capability. The updated pseudo-masks are used for supervising the computation of $L_{ {seg }}$, which is formulated as a weighted combination of cross-entropy and soft Dice coefficient: $L_{{seg }}=L_{C E}+0.5 \cdot L_{ {Dice }}$. Notably, the pseudo-mask generation process is performed with gradient backpropagation completely blocked, serving solely as a fixed supervisory target to drive the parameter updates of the segmentation decoder, thereby avoiding the degenerate collapse problem commonly encountered in self-training. Through the collaborative iteration of the aforementioned contrastive refinement and hard update strategy, the seed regions are progressively evolved into high-quality pseudo-masks with fully filled interiors and precisely adhered boundaries, providing reliable pixel-level spatial support for the subsequent association embedding.

2.4 Geometry-aware visual-semantic association encoder

The segmentation mask $\widehat{M}$ provides the spatial extent of defects; however, the quality traceability task requires the system to infer the production batch and process conditions from the visual appearance of defects, which necessitates the encoding of pixel-level segmentation results into discriminative compact vectors. The visual representation of defects encompasses two complementary dimensions: first, the texture statistics within the defect interior, which reflect material characteristics and the physical imprints of the generative process; and second, the geometric topology of the defect contour, which characterizes the mechanical boundary conditions during the morphological formation process. The interior texture feature g is obtained via masked average pooling over the deep features $F_{ {aspp }} \in \mathrm{R}^{256 \times 112 \times 112}$ for all pixels within the segmented region: $g=1 /|\Omega| \Sigma_{i \in \Omega} F^{(i)}{ }_{ {aspp }}$, where Ω denotes the set of defect pixel coordinates in $\widehat{M}$. The extraction of geometric features is performed with greater sophistication. First, the binary contour of $\widehat{M}$ is traced using the Suzuki-Abe border-following algorithm, yielding an 8-connected directed boundary point sequence $B=\left\{p_1, p_2, \ldots, p_K\right\}$, with which the contour extraction is accomplished at O(K) linear complexity and natural support for hole topologies is provided. For each boundary point $p_i$, bilinear interpolation sampling is performed at two pixel positions inward and two pixels outward along the normal direction within its spatial neighborhood, forming a strip feature tensor $T \in \mathrm{R}^{K \times 5 \times 256}$, which encodes the feature gradient patterns on both sides of the defect edge and is sensitive to discriminative texture transitions at boundaries. To eliminate the rotational ambiguity introduced by the selection of the contour starting point, circular shift alignment is applied to the strip features: the minimum bounding rectangle of the defect instance is computed, its principal direction is determined via principal component analysis, and the starting point of the contour sequence is aligned to the extreme position along this principal direction, thereby ensuring that the geometric representation remains stable against rotational variations of the instance. Subsequently, max pooling and average pooling are performed along the contour point dimension, and the two results are concatenated to obtain a 256-dimensional shape representation s. The geometry-aware visual-semantic association and domain adaptation are illustrated in Figure 4.

Figure 4. Diagram of geometry-aware visual-semantic association and domain adaptation