Enhanced Express Package Trademark Recognition via a Novel PTD-YOLO Algorithm

The labor-intensive nature of packaging classification, combined with the high workload and limited inspection time, renders the current methodology for packaging appearance inspection suboptimal [1]. At present, manual visual inspection forms the backbone of this process, leading to a myriad of challenges, including the potential for subjective results, inconsistencies, oversights, misidentification, and arbitrary classifications. This manual approach hampers the efficiency of detection, a concern heightened by the escalating demand for product customization. Moreover, the precision of detection holds significant implications for the evaluation of both product quality and pricing.

As production continues to evolve, there has been an amplification in the demand for a diverse range of packaging types and sizes. This diversification further compounds the difficulty of manually identifying and classifying the packaging surface. In response to these challenges, the development of an intelligent express sorting and handling system has become a pressing need, with accurate packaging trademark identification forming a critical component of this endeavor.

For automated packaging trademark inspection, innovative appearance inspection methods need to be formulated. Contemporary inspection methods are based on algorithms that identify specific areas in camera images for detection, and target detection algorithms are integral to these methods [2]. Over time, a series of robust target detection algorithms have been developed, each producing commendable recognition results. The progression of these algorithms can be broadly categorized into three stages.

The first stage was marked by object detection algorithms predominantly focusing on precise image segmentation. Angelova and Zhu [3] proposed a method that detects low-level regions for full-object segmentation, which distinguishes various types of objects through fine-grained recognition. Gould et al. [4] highlighted the intricate relationship between object detection and multi-class image segmentation, introducing a hierarchical region-based method that linked these tasks based on a coherent probabilistic model. Moreover, Carreira and Sminchisescu [5] presented a framework that formed a regular image grid based on Constrained Parametric Min-Cut problems (CPMC), thereby maximizing the category of segmentation. However, these algorithms, being task-specific, lacked the versatility required for wider application.

Following this, the focus shifted to the application of inspection technology based on machine vision. Paliwal et al. [6] integrated morphological, color, and texture features to distinguish five grain types through a neural network. This method achieved approximately 90% accuracy in some grains, despite variations in size and appearance. In another study, Meng et al. [7] designed a machine vision system to assist with mechanical inter-row weeding. They used the hue component for image processing and computed the H component for generating grayscale images to mitigate the adverse impact of light. This technique enabled the system to determine the endpoint of the images, thereby tracking the edge of weeds effectively. Additionally, Qureshi et al. [8] considered two critical fruit features, texture and shape, and utilized support vector machines as a classifier for their experiments. The method demonstrated high precision in various environments.

Despite these advancements, the field continues to grapple with several challenges, not least due to the increasing diversification of the manufacturing landscape. Consequently, the use of Convolutional Neural Networks (CNN)-based object detection algorithms, renowned for their strong representational power and high versatility, has begun to supplant conventional methods.

These CNN-based defect detection networks can be subdivided into two structures: one-stage and two-stage algorithms [9]. A series of two-stage algorithms, including Fast R-CNN [10], Mask R-CNN [11], and Faster R-CNN [12], were proposed by Girshick et al. [10-12]. While these algorithms are known for their accuracy, their relatively slow speed makes them less suited for packaging trademark detection. To address this, one-stage algorithms have been developed which transform the classification problem into a regression problem. These algorithms directly predict position information, leading to an improvement in speed [13].

A diverse array of strategies have been employed in prior research to improve the accuracy and efficiency of object detection models, though these approaches often remain ill-suited for the unique challenges presented by carton damage detection scenarios. Carton damage detection scenarios are characterized by low color contrast and variable damage sizes, making it susceptible to errors.

A compelling contribution in the domain of multi-scale targets in ship detection was proposed by Guo et al. [14]. A novel model was proposed that ameliorates the backbone module and enhances spatial adaptability via a novel loss function. Empirical results revealed that the proposed model exhibits superior multi-scale detection capability and mobile platform portability.

In a different approach, Han et al. [15] devised a lightweight model and improved the network structure employing the K-means++ clustering algorithm. The modified model has shown an enhancement in the detection performance of small-size targets.

Zhang and Wen [16] directed their efforts towards mitigating the high maintenance cost of wind turbine blades (WTB). A surface defect detection algorithm for WTB, dubbed SOD-YOLO, was developed. A fusion of the K-means algorithm and attention mechanism to YOLO v5 was implemented following the collection of a WTB dataset. This model resulted in a 7.82% mAP and 28.3% increase in detection speed.

Simultaneously, Qi et al. [17] made improvements to the YOLO network structure for tea chrysanthemum detection during different stages. The methodology combines data augmentation, transfer learning, and other methods for the flowering stage detection. With a mAP of 92.49% on the test set, it demonstrated a remarkable performance and can be trained on a single GPU.

Aiming at avian detection, Siriani et al. [18] proposed an enhanced YOLO v4 model to locate birds. The incorporation of the Kalman filter allowed tracking of chickens in low light environments. This algorithm achieved a high-performance rate in bird detection even in high noise images.

Despite these advancements, the need for an efficient algorithm tailored to packaging appearance inspection is apparent. In response to this challenge, a package appearance damage target detection algorithm, known as PTD-YOLO (Packing Trademark Detection algorithm based on YOLO), is proposed. This method reconstructs the Focus module, introduces an additional detection scale, and improves the prediction header structure, making it more compatible with the packaging appearance inspection task. Compared with existing target detection algorithms, PTD-YOLO achieved high accuracy [14-18].

Three crucial contributions are put forward:

1. Considering the limited data of packaging appearance, model reparameterization technology was introduced into the Focus module to enhance the mining of existing data features. A deeper network structure was designed through multiple branches.
2. To accommodate various sizes of trademark images, a four head prediction structure was constructed. This new prediction head collaborates with the original three heads to meet the detection needs of drastically changing target scales.
3. Based on the reconstructed Focus and four-head prediction structure, the PTD-YOLO algorithm was proposed. By introducing an attention mechanism in the head before the output of object detection, the box reflection problem was overcome. This method assists the algorithm in targeting high-latitude features more related to the target trademark.

While these contributions mark a step forward in packaging appearance inspection, further investigations are warranted to fully realize the potential of these techniques in real-world applications. The expansion of this domain of research may provide further improvements and efficiencies in the field, facilitating enhanced packaging quality, and the delivery of superior products to consumers.

3.1 Parameter configuration for training

The training of the code was expedited via CUDA, executed on an RTX 3090 24G graphics card, under the Windows 11 operating system. The deep learning framework, PyTorch, in its 11.0 version was harnessed for the creation of the model. Complementarily, the computational resources included an Intel i7-12700KF CPU, paired with a 64GB memory.

3.2 Indices of evaluation

The experiment was gauged on the grounds of precision P, recall R, average precision AP, and mean average precision mAP. AP was computed as the area bordered by the Precision-Recall (P-R) curve and the coordinate axis. For each of the n categories, True Positives (TP), False Positives (FP), and False Negatives (FN) were calculated, according to the defined formula as follows:

$P=\frac{T P}{T P+F P}$                   (1)

$R=\frac{T P}{T P+F N}$                   (2)

$A P=\int_0^1(P(R) d R)$                   (3)

$m A P=\frac{\sum(A P)}{n}$                   (4)

3.3 PTD-YOLO ablation experiment

The effects of technological advancements, such as FSRP, PHFS, and Attention on the recognition accuracy of PTD-YOLO were perceptively examined through ablation experiments as reflected in Table 1.

Following the introduction of FSRP, an enhancement of 0.8% was recorded in the mAP of the PTD-YOLO model. With the further application of a four-head detection structure, the mAP noted an incremental rise of 1.5%. The integration of an attention mechanism into the prediction header fostered a more effective extraction of features from the convolutional structure, culminating in a mAP ascent of 0.3%. The combined effects of these advancements resulted in a remarkable boost in detection accuracy. Particularly, FSRP+MFS exhibited a mAP improvement of 2.3%. When FSRP was combined with a prediction header equipped with an attention mechanism, a performance enhancement of 1.7% was observed, translating to a mAP of 97.7%. When the four-head structure was coupled with an attention-stimulating predictive head, a mAP improvement of 2.5% was recorded. Remarkably, the four-head structure excelled over the other two enhancements, with the first seven ablation experiments demonstrating that the four-head configuration resulted in superior mAPs (1.5%, 2.3%, and 2.5% higher than baseline, respectively). The convergence of all three enhancements exhibited the most profound effect, signifying a 3.1% mAP increase, thus, affirming the efficacy of the proposed algorithm.

Table 1. Comparative analysis of experimental results from various modules of PTD-YOLO

3.4 Performance comparative experimen

In order to assess the superiority of the PTD-YOLO, it was compared with several conventional algorithms including the Two-Stage algorithms, the well-known SSD algorithm and a series of YOLO algorithms under One-Stage detectors. For the task of carton logo detection, YOLO v2, YOLO v3, YOLO v3-tiny, YOLO v4, YOLO v4-tiny, YOLO v5, and YOLO v5-tiny were chosen for comparison with PTD-YOLO. The Mean Average Precision (mAP) was employed as the evaluation index in this analysis. Identical hyperparameters were employed in the training and testing of each model. The outcomes of the detection are delineated in Table 2.

Table 2. Detection performance comparison among different algorithms

On scrutinizing the results depicted in Table 2, it can be discerned that the Two-Stage detection algorithms, Fast R-CNN and Faster R-CNN, despite exceeding 60% in detection accuracy, fall short of meeting the research objectives and industrial standards for carton logo detection, possibly due to their inadequacy in detecting small-scale trademarks. The SSD algorithm likewise fails to attain high accuracy. Conversely, the YOLO series algorithms demonstrate substantial progress, with mAP values surpassing 80%. YOLO v4 and v5 algorithms have even attained mAP values over 90%, signaling their potential for practical implementation. Out of the eleven algorithms evaluated, PTD-YOLO achieves superior performance, marking a maximum improvement of 36.4% in detection accuracy compared to the Two-Stage algorithm. Even in comparison with the improved YOLO v2, v3, v4, and v5, this algorithm displays improvements varying from 3.1% to 18.1% in detection accuracy, suggesting a more precise estimation of trademark position and size distribution. In sum, PTD-YOLO surpasses in both detection accuracy and real-time express packaging surface logo detection, offering superior comprehensive performance and practical utility. Figure 7 showcases an exemplary application of PTD-YOLO.

7.png

Figure 7. Illustrative Diagram of a Practical Application of PTD-YOLO

3.5 Constraints of PTD-YOLO

Despite the commendable accuracy of the proposed PTD-YOLO algorithm in tasks of carton trademark recognition, thus paving the path for potential future applications, certain limitations are also observed.

(1): The introduction of the FSRP module, while enhancing the accuracy of carton trademark detection, necessitates the incorporation of a new detector to broaden the detection range and distribution. However, akin to conventional object detection models, the implementation of any new modules demands robust terminal device support to ensure successful deployment of the model in practical production tasks, thereby escalating the economic costs.

(2): Furthermore, stability of the algorithm becomes a paramount consideration for the successful application of object detection algorithms in the realm of carton trademark detection. Given that these algorithms are primarily devised and evaluated in experimental environments, performance comparisons are often conducted within limited theoretical frameworks. Hence, to effectuate the application of these object detection algorithms in real-world scenarios, their stability must be ascertained under conditions of continuous operation.