Automated Extraction and Classification of Product Design Elements Based on Image Recognition

With the rapid development of AI technology, computer vision and deep learning methods have been widely applied in various fields, especially in product design, where image recognition technology provides strong support for product innovation and optimization [1-4]. In traditional product design processes, designers rely on experience and intuition to select and arrange elements, but this process is often limited by personal perspective, time constraints, and efficiency issues [5-7]. With the accumulation of large amounts of product image data, how to automatically extract effective design elements from design images, analyze and optimize design solutions, has become a key issue that needs to be addressed in the current product design field. Therefore, the research on automated extraction and classification of product design elements based on image recognition technology has gradually become an important research direction in both academia and industry.

The application of image recognition technology in product design can help designers quickly obtain structural features and layout information of design elements, provide data support for product innovation, and improve design efficiency and quality through automation [8-11]. By automatically analyzing and classifying product design images, manual intervention can be reduced, design consistency and accuracy can be improved, and the product design process can be optimized. In addition, this research can help enterprises reduce product development cycles, lower costs, and improve market responsiveness and competitiveness [12-15]. Therefore, developing image recognition-based automated product design element extraction and classification technology has important academic value and broad application prospects.

Although existing research has made some progress in the image recognition and classification of product design elements, there are still some shortcomings in current methods [16-19]. Traditional image feature extraction methods often rely on shallow, handcrafted features, lacking the ability to accurately identify deeper, more complex design elements. Furthermore, existing classification methods often face issues of low accuracy and insufficient robustness when dealing with complex design patterns [20-25]. Especially when processing design images with diversity, complex backgrounds, or partial occlusions, traditional algorithms often perform unsatisfactorily. Therefore, how to maintain high accuracy while improving the robustness of the method has become a significant challenge in current research.

This paper addresses the shortcomings of existing methods and proposes a deep learning-based image recognition technology aimed at improving the automated extraction and classification of product design elements. The main content of the study includes two aspects: first, the residual network for deep feature extraction of product design elements, which effectively captures the complex design features in images through a multi-level deep learning model; second, the design element classification method based on the spatial pyramid attention mechanism, which enables efficient classification and localization of design elements at different scales and perspectives. Through the combination of these two technologies, this paper not only improves the accuracy of design element recognition but also enhances the adaptability and robustness of the method. The research results are expected to provide an efficient and accurate automated tool for the product design industry, promote the intelligentization of product design, and offer valuable references for related research fields.

2.png

Figure 2. Overall network architecture of product design element classification method

3.png

Figure 3. Structure of SPCS mechanism

4.png

Figure 4. Structure of the SPCE mechanism

(1) SPCS Mechanism

The SPCS mechanism is designed to improve the accurate extraction of local and global information in the product design element classification task. Figure 3 shows the structure of this mechanism. The main goal of this mechanism is to enhance the network’s perception of spatial structures and details in complex design images by aggregating spatial features from different scales. The mechanism first performs multi-scale pooling operations on the input feature map. During this process, the feature map undergoes one-dimensional global pooling along the vertical and horizontal directions, generating two merged feature vectors, EDQ and EDG. These two pooling vectors effectively capture long-range interactions in the image, thereby enhancing the expression of global information. This pooling method enables the mechanism to preserve long-range spatial dependencies in the image while maintaining efficiency. Furthermore, the mechanism performs two-dimensional adaptive average pooling to further optimize the extraction of local region information, generating multiple pooled feature maps ED1, ED2, and ED3, corresponding to feature regions of sizes 3×3, 6×6, and 8×8, respectively. Through these multi-scale pooling operations, the mechanism captures multi-level spatial information in design images, from detailed to global, significantly enhancing the multi-scale perception ability of product design elements.

Furthermore, to ensure that the multi-scale local and global spatial features are effectively input into the subsequent spectral attention mechanism, the mechanism reshapes and flattens the pooled feature maps to generate reshaped and flattened feature maps, ED. These feature maps include feature vectors EDQ, EDG, and aggregated features ED1, ED2, and ED3 at multiple scales. Each feature map corresponds to the number of image blocks at different scales, such as 9×, 36×, and 64×, representing feature regions from small to large scales. These feature maps are concatenated along the spatial dimension to form a large-scale spatial feature set TD. Specifically, let features EDQ, EDG, ED1, ED2, and ED3 be represented as d^q_e$\in$R^Z×G×1, d^g_e$\in$R^Z×G×1, d¹_e$\in$R^Z×9×1, d²_e$\in$R^Z×36×1, and d³_e$\in$R^Z×64×1, where the height and width of feature DL are denoted by G and Q. The pooled feature maps ODQ, ODG, OD1, OD2, and OD3 in the z-th channel are represented by d^g_oz(g), d^q_oz(g), d¹_oz, d²_oz, and d³_oz. Adaptive AvgPool is denoted by AA(*). Finally, the feature TD is generated and denoted as d_t$\in$R^{Z×(G+Q+T)×1}, and there are:

$\left\{\begin{array}{l}d_{o z}^g(g)=\frac{1}{Q} \sum_{q=1}^Q d_z(g, q) \\ d_{o z}^q(g)=\frac{1}{G} \sum_{g=1}^G d_z(g, q) \\ d_{o z}^1=A A(3) \\ d_{o z}^2=A A(6) \\ d_{o z}^3=A A(8)\end{array}\right.$        (1)

Thus, assuming the output feature in the z-th channel after the SPCS mechanism is d_tz, the spatial dimension concatenation operation is denoted by [ ]. The reshaped features EDQ, EDG, ED1, ED2, and ED3 in the z-th channel are represented as d^g_ez, s^q_ez, d¹_ez, d²_ez, and d³_ez. The concatenation of feature DL in the z-th channel along the spatial dimension can be represented by the following formula:

$d_{t z}=\left[d_{e z}^g, d_{e z}^q, d_{e z}^1, d_{e z}^2, d_{e z}^3\right]$       (2)

(2) SPCE Mechanism

The main function of the SPCE mechanism is to help the network focus more precisely on key areas in the image by combining spectral and spatial features, thereby improving classification accuracy. Figure 4 shows the structure of this mechanism. The basic principle of this mechanism is to use the precise spatial information contained in the feature map, generating a series of attention maps to enhance the model’s ability to perceive different spatial regions and scales. Specifically, this mechanism first encodes the feature maps with multi-scale information and precise location information into two main attention maps. These two attention maps can capture spatial dependencies along specific directions, effectively retaining important spatial information in the image in the generated spectral attention map. This operation ensures that the SPCE mechanism can focus on key information at different positions and scales in the product design element classification task, allowing the model to automatically adjust its attention to local and global structures in the image and avoid overfocusing on irrelevant areas.

In the SPCE mechanism, the generation process of the spectral attention map is achieved through a two-step operation. The mechanism first reduces the number of channels in the spatial feature map to a certain proportion of the original size using convolution operations, thereby reducing the computational complexity and optimizing feature representation. Then, the compressed feature map is split from the spatial dimension into two groups and undergoes channel expansion to generate channel weights with different spatial features. These channel weights allow the model to dynamically adjust and weigh features at different spatial positions, effectively capturing spectral features that integrate various spatial information. In the product design element classification task, this mechanism can adaptively enhance attention to critical local areas in the design image, while suppressing the impact of unimportant regions, helping the model understand the structure and features of design elements from global to local. The specific steps are as follows:

1) Feature Processing and Channel Dimensionality Reduction

Process the aggregated spatial features. Through a 1×1 convolution operation, the model reduces the channel number of the input features to 1/e of the original size. After the convolution operation, the feature map is standardized using batch normalization to ensure the model maintains a stable learning rate during training and accelerates convergence. Let the Mish activation function be denoted as σ. The convolution operation is denoted as D₁. The kernel size is denoted as Z/e×C×1×1, with the number of output and input channels being Z/e and Z, respectively, and the reduction ratio being e. The expression for the feature d_t obtained after feature processing and channel dimensionality reduction is:

$d_l=\sigma\left(D_1\left(d_t\right)\right)$     (3)

After the above operation, the size of d_t becomes Z/e×T×1.

2) Spatial Dimension Splitting and Reshaping

After the channel reduction and batch normalization, the SPCE mechanism splits and reshapes the feature map from the spatial dimension into two groups. One group contains the feature maps with precise location information, denoted as row features LDG and column features LDQ. These feature maps can capture dependencies along the horizontal and vertical directions in the image, further emphasizing the importance of spatial structure in the image. The other group contains feature maps with multi-scale information, denoted as LD1, LD2, and LD3, with sizes of 3×3, 6×6, and 8×8, respectively. Through this multi-scale splitting and reshaping, the mechanism can effectively retain spatial information at different scales, helping the model focus on different levels of spatial details when processing product design elements. Let the convolution operation with kernel size Z×Z/e×1×1 be denoted as D_g and D_q. LDG and LDQ are denoted as d^g_l and d^q_l, thus:

$\left\{\begin{array}{l}h^g=\sigma\left(D_g\left(d_l^g\right)\right) \\ h^q=\sigma\left(D_q\left(d_l^q\right)\right)\end{array}\right.$          (4)

3) Weighted Summation and Focus on Spectral Information

After the spatial dimension splitting, the SPCE mechanism performs weighted summation on the row features, column features, and multi-scale features LD1, LD2, and LD3. This weighted operation allows the SPCE mechanism to fuse information at different levels of global and local features, ensuring that important spatial regions receive more attention. By applying weighted processing to the feature map, the mechanism enhances the model’s focus on spectral information for precise location regions, especially for parts of the image with special structures or design elements. Let the feature DL in the z-th channel be denoted as d_z(g,q), and the coordinate feature in the z-th channel be denoted as d^zp_E(g,q). After applying h^g and h^q to the feature map DL, the weighted summation of h^g, h^q, and DL is:

$d_E^P(g, q)=d_z(g, q) \times h_z^g(g) \times h_z^q(q)$          (5)

4) Convolution Operation and Spectral Attention Feature Extraction

To extract richer spectral attention features from the multi-scale features, the SPCE mechanism convolves the split multi-scale features with convolution kernels. Each scale’s feature is convolved with convolution kernels of different sizes. Specifically, LD1, LD2, and LD3 are convolved with convolution kernels of sizes Z×Z/e×3×3, Z×Z/e×6×6, and Z×Z/e×8×8, respectively, resulting in spectral attention features h¹, h², and h³ at different scales. Let LD1, LD2, and LD3 be denoted as d¹_l, d²_l, and d³_l, and the sum of h¹, h², and h³ be denoted as h^t$\in$R^Z×1×1, thus:

$\left\{\begin{array}{l}h^1=D_1\left(d_l^1\right) \\ h^2=D_2\left(d_l^2\right) \\ h^3=D_3\left(d_l^3\right)\end{array}\right.$          (6)

Let the feature in the z-th channel be denoted as d^to_E(g,q), and the multi-scale feature is denoted with the superscript to.

5) Feature Integration and Spectral Attention Map Generation

The spatial pyramid coordinate attention feature in the z-th channel can be represented as:

$d_E(g, q)=d_z(g, q)+d_E^{z p}(g, q)+d_E^{t o}(g, q)$        (7)

By convolving the multi-scale features and extracting spectral attention features, the SPCE mechanism generates spatial pyramid coordinate attention features for each channel. These attention features can adaptively focus on important regions in the image and further optimize the spatial-spectral joint feature representation through fusion with other features.

6) Feature Fusion and Final Output

Finally, the SPCE mechanism fuses the original feature map with the generated spectral attention features via skip connections. This fusion operation, by connecting the feature map with the spectral attention map, further enhances the model’s spatial-spectral feature representation. With the combination of batch normalization, 1×1 convolution, Sigmoid activation function, and concatenation operation, the SPCE mechanism generates the final spatial pyramid coordinate attention features. These features not only encompass information from different spatial scales but also adapt to changes in complex design elements in the image.

The deep learning-based design element extraction and classification method proposed in this study holds significant potential in various practical application scenarios. In the field of industrial design, this method can automatically extract and classify design elements, assisting designers in quickly iterating design solutions and significantly reducing the workload of manual annotation. By automating the recognition of design elements, designers can focus more on creativity and innovation, enhancing design efficiency. In consumer product development, especially in industries such as home appliances, automobiles, and electronics, the automatic extraction and classification of design elements enable companies to swiftly identify market trends, optimize product designs, and shorten product development cycles. Furthermore, the model can help developers select the most promising design elements in the early stages of the design process, improving product market competitiveness. In the field of smart manufacturing, with the continuous advancement of intelligent production processes, this method can closely integrate the extraction of design elements with the manufacturing process. By integrating with CAD systems and Manufacturing Execution Systems (MES), it can achieve seamless linkage between design and production, greatly enhancing production efficiency. However, practical applications still face some challenges. Firstly, the diversity of design data and the scarcity of annotated data limit the model's generalization ability. To address this, we will employ data augmentation techniques and utilize Generative Adversarial Networks (GANs) to supplement the data in future work.