An Image Classification and Retrieval Algorithm for Product Display in E-Commerce Transactions

The percentage of e-commerce in all transaction modes is gradually rising as Internet technology develops [1-7]. E-commerce has unmatched benefits over conventional consumption techniques, but it also restricts the amount of contact that consumers have with the things they are purchasing [8-16]. Therefore, the buyer can only rely on the information provided by the seller in the e-commerce transaction product display in order to understand the transaction product. The informational display of e-commerce transaction products has a direct impact on the buyer's perception of value [17-22]. The first item in the transaction product information flow, which is particularly significant for product sales, is the efficient transaction product display image.

Taxonomies are frequently used on e-commerce websites to better categorize the products. A novel neural product categorization model was put forth by Chen et al. [23] to extract fine-grained classes from product content. By simultaneously recognizing classes from the product content and projecting classes from a predefined class vocabulary, the model classifies products. Extensive tests on actual e-commerce platform datasets show how well the suggested methodology works. Hidden Markov Model (HMM) performance in e-commerce product classification was examined by Mathivanan et al. [24], who also provided a parameter estimation method for HMM evaluation. In order to portray the material colors and textures of products in general, Yamashita and Mujibiya [25] presented a way to assist users in capturing and creating product visualizations. Users can see the product through the auto-discovered view path or manually travel through the automatically detected camera spots. Users may be able to recognize materials and gain a better understanding of product quality by visualizing objects from various perspectives that reveal information about light reflection and refraction. E-commerce systems often use images to communicate qualities that are challenging to describe in plain text. Wroblewska et al. [26] made a contribution to demonstrating how to offer quantifiable measures of product image business quality by highlighting the benefits of deep learning approaches over more conventional methods for identifying information in images.

Insufficient attention is paid to the e-commerce transaction product display image, a special transaction product information, and a lack of in-depth study, as shown by the findings of the current studies conducted both domestically and overseas. As the number of commodity classes rises, there is also a lack of scientific guidance for the buyer's efficient retrieval of transaction products because it has not been established that the emotions present in the current transaction product display image have any effect on the buyer's purchase intention. In order to do this, this work investigates image classification and retrieval methods for product presentation in e-commerce transactions. The network architecture for e-commerce transaction product display and image categorization is designed in section 2. The model primarily consists of three components: the core module for extracting image polarity emotion features, the polarity emotion intensity perception module, and the classification module for emotion feature fusion. The model developed in this paper makes use of the network learning concepts of weak supervision and distributed label smoothing to address the issue that e-commerce transaction product display images are susceptible to, namely the issue that the intensity of polar emotions cannot be accurately reflected. In section 3, the network training method is improved. Section 4 completes the design of the e-commerce transaction product display image similarity retrieval algorithm, which improves the buyer's retrieval efficiency of transaction product display images. The experimental results verify the validity of our classification model and retrieval method.

The polarity emotion classification model for e-commerce transaction product display images developed in this study learns the results of two parallel branches based on class labels, including positive and negative emotion attention maps, emotion intensity, and the final polar emotion classification results of e-commerce transaction product display images. Since image-level tags are typically used to identify product display images for e-commerce transactions, there is a risk that the intensity of polar emotions will not be adequately conveyed. To address this issue, the model developed in this paper, which refers to the network learning concepts of distributed label smoothing and weak supervision, innovates the network training method.

Let $\{A, B\}=\left\{\left(A_i, B_i\right)\right\}^{M_{i=1}}$ be a training set containing $\mathrm{M}$ transaction product display image samples, the image sample is represented by $A_i$, and the emotion class label of $A_i$ is represented by $B_i \in\{0,1\}$. Assume that the weighting coefficients of the corresponding item loss function are represented by $\mu_1$ and $\mu_2$, the loss function that guides the learning of the polar emotion intensity perception module is represented by $K_{P M}$, the loss function that constrains the intensity of polar emotion is represented by $K_{R E}$, and the loss function that constrains the final polar emotion classification result Represented by $K_{F N}$. The multi-task loss function of the network model can be expressed as:

$K=K_{P M}+\mu_1 K_{F N}+\mu_2 K_{R E}$                    (10)

In order to improve the detection accuracy of the two parallel branches of the polar emotion intensity perception module on the emotion regions of different polarities in the transaction product display image, the model constructed in this paper sets the loss function K_PM:

$K_{P M}(B, u)=-\frac{1}{M} \sum_{i=1}^M \sum_{j=t, m} B_i^j \log \left(u_i^j\right)$                     (11)

Let $u_i \in R^2$ be the predicted emotion polarity of transaction product display image; FD(.) be the serial connection; GAP_max(.) be the global average pooling; $G^{t^{\prime}}{ }_i$ and $G^{m^{\prime}}{ }_i$ be the active emotion feature map and passive emotion feature map, respectively. The calculation flow can be expressed as follows:

$u_i^t=G A P_{\max }\left(G A P\left(G_i^{t^t}\right)\right)$                     (12)

$u_i^m=G A P_{\max }\left(G A P\left(G_i^{m^{\prime}}\right)\right)$                     (13)

$u_i=\Gamma\left(F D\left(u_i^t, u_i^m\right)\right)$                     (14)

To ensure that the predicted value of the image polarity emotion intensity output by the network model is positively correlated with the real value of the image emotion of the ecommerce transaction product display, it is assumed that $\Psi($. $)$ is the tanh activation function; |.|1 is the $K_1$ norm; $B_i-\dot{B}_i$ is the label mapping of image $A_i ; y^{m_i-y_i^t}$ is the polar emotion intensity difference of $A_i$. Both $B_i-\dot{B}_i$ and $y^m{ }_i-y_i^t$ fall in the value range of $\{-1,1\}$. This paper introduces the polar consistent constraint loss function $K_{R E}$ :

$K_{R E}(B, y)=\frac{1}{M} \sum_{i=1}^M\left|\Psi\left(B_i-\dot{B}_i\right)-\Psi\left(y_i^m-y_i^t\right)\right|_1$                    (15)

To display the polar emotion of the e-commerce transaction product display image in the form of intensity, this paper caries out distributed label training of K_FN:

$K_{F N}\left(B, B^{\prime}\right)=\frac{1}{M} \sum_{i=1}^M \sum_{j=t, m} u_i^j \log \left(B_i^{\prime j}\right)$                    (16)

In this section, this paper compares the constructed e-commerce product display image classification network model with 11 existing advanced image classification models to verify the superiority of the constructed model. Table 1 compares the performance of different image classification models in different sample sets.

As shown in Table 1, compared with state-of-the-arts like ImageNet, AlexNet, VGGNet, GoogLeNet, ResNet, MobileNet, EfficientNet, EfficientDet, NFNet, Faster R-CNN, and CenterNet, our model achieved better indices on all six e-commerce transaction product datasets.

The model in this study divides the polar emotion sections of the image to be identified and predicts the polarity of each zone, avoiding the classification results being impacted by other algorithms that integrate global features, visual attention, and multi-scale data to predict image emotion. There are instances where significant errors are produced without respect to the impact of features. As a result, the model in this study can more accurately anticipate how consumers will feel about e-commerce transaction products, increasing the impact of those outcomes on consumers' purchase intentions. The distributed label smoothing method, which is used in this paper to learn and train the network model, enhances the network model's capacity for learning and reduces noise in the sample set. As a result, the performance of the model in this work significantly outperforms that of other image classification models on sample sets 2, 3, and 5. Additionally, compared to other image classification models, the model from this paper with the polar emotion intensity perception module configured has less network parameters.

In order to verify the effectiveness of the polar emotion intensity perception module set in this paper, and the generalization of the calculation of positive and negative emotion intensities, this paper conducts a structural ablation experiment, and Table 2 gives the corresponding experimental results.

Table 1. Performance comparison of different image classification models

Table 2. Results of structural ablation experiment

Table 3. Time costs of different retrieval models

6-1.png

1)

6-2.png

2)

Figure 6. Scatterplot of positive and negative emotion intensity predictions for transaction product images

Table 4. Performance comparison of similarity measurement methods and retrieval algorithms

Table 2 shows that the classification accuracy is lowest when only the backbone module of image polarity emotion feature extraction is used to extract the global characteristics of e-commerce transaction product images. Performance has increased greatly when ImageNet, AlexNet, and VGGNet are used in place of the created model's backbone network, proving the generalizability of the configuration for the polar emotion intensity perception module.

To verify whether the prediction of the polarity intensity of each polar region of the e-commerce transaction product display image is effective and whether the positive and negative emotion intensities show a certain positive correlation, this paper uses the form of scattered points in the two-dimensional coordinate space to analyze the polarity intensity of the polarity area. Figure 6 shows the scatterplot of positive and negative emotion intensity predictions of transaction product images corresponding to different sample sets. It can be seen that the upper left area and the lower right area of the scatter plot respectively gather the predicted values of emotion intensity of positive samples and negative samples, and the overall positive and negative emotion intensities show a strong positive correlation. The main reason is that the model in this paper introduces the loss function K_RE that constrains the intensity of polar emotion and the loss function that guides the learning of the polar emotion intensity perception module is K_PM, which further verifies the rationality of setting the polar emotion intensity perception module.

Table 3 shows the time costs of different retrieval models. It can be seen that with the increase of the number of eigenvectors, the time consumption of traditional method retrieval continues to increase linearly. However, the retrieval method based on the improved product quantization algorithm optimization takes only about 1/70 of the time-consuming of the traditional method retrieval algorithm, reflecting a huge advantage in retrieval efficiency.

Table 4 compares the performance of similarity measurement methods and retrieval algorithms. It can be seen that, when the retrieval algorithm is the same, the image retrieval accuracy of e-commerce transaction products corresponding to the similarity calculation method based on cosine distance C_CD is higher, but the algorithm time cost correspondingly increases. When the similarity measurement method is the same, compared with the traditional method retrieval algorithm, the improved product quantization algorithm optimization retrieval algorithm adopted in this paper reduces the retrieval accuracy by 2.05% and 0.74%, respectively, but increases the time cost by 1.62ms and 2.83ms. Compared with the traditional product quantization algorithm optimization retrieval algorithm, the proposed improved product quantization algorithm optimization retrieval algorithm improves the retrieval accuracy rate by 1.36% and 6.22% under the condition that the retrieval time cost is not much different, which verifies the retrieval accuracy and effectiveness of our method.