Real Time Classification of Retail Theft Utilizing YOLO Algorithm

Shoplifting, also known as retail theft, poses a significant problem for businesses and owners, resulting in security challenges and economic losses [1]. This theft takes various forms, such as concealing an item under clothing, placing it in a personal bag, or concealing it in a pocket [2]. The primary function of camera systems within shopping environments is to serve as a preventative and theft-reducing tool. However, despite their ubiquity and availability, the number of shoplifters is on the rise, and the losses incurred by store owners are also increasing, with the number of shoplifters reaching approximately 27 million, and the losses also reaching $13 billion annually [3]. The reason for the increase in these thefts is the limited human capacity for continuous monitoring. Consequently, it is imperative to adopt an intelligent monitoring system that classifies theft incidents and analyzes individual behavior in real time [4]. Researchers have focused their attention and efforts on developing applications as a result of advances in image processing and computer vision, which have opened the door for them to address the challenges associated with developing intelligent systems in a practical and effective manner, particularly in the field of human behavior analysis [5]. This study, which used YOLO v8, seeks to automatically classify shopper behavior within shopping environments, as this will detect suspicious activity and enhance security measures early [6]. Real-time object detectors should follow an optimal trade-off between speed and accuracy. Towards this end, many works have been put forward to design efficient and effective architectures for real-time object detection [7]. Yolo version is based on a deep neural network that has been optimized to increase learning efficiency and reduce information loss across layers [8]. The image entering this model is divided into a grid. Each set of predictive boxes may identify a specific object within this grid, and each object has a probability [9]. A combination of mathematical, logical, and nonlinear steps is employed, as well as procedures to find optimal solutions, known as optimization techniques. These techniques strike a balance between correct detection and minimizing losses [10]. Accordingly, YOLOv8 will be an important tool for operating intelligent systems to monitor suspicious behavior in shopping environments, including retail stores [11]. The YOLO architecture transforms object detection by hierarchically synthesizing a single neural network that collectively performs bounding box regression, and simultaneously predicts object types contained in the images [12]. Unlike traditional approaches that separate the proposal and classification phases, YOLO offers a complete solution implemented in a single step, reducing processing time and increasing efficiency [13]. It is therefore highly suitable for real-time applications such as smart surveillance. Newer structural optimization methods are added in its latest version YOLOv8 whereby both the accuracy and speed can be improved [14]. According to the above, the YOLOv8 model was selected as the backbone detector of the proposed system for its high real-time detection and classification efficiency, as well as its balance between accuracy and inference velocity [15]. This model fits perfectly with the manually labelled data used in this work, and is used to localize people within images. and classification between normal and suspicious behavior. When the advantages of YOLOv8 (such as network architecture optimization) were utilized, the system can effectively capture fine-grained visual characteristics. Further, the model can be extended to the temporal analysis models, such as GRU or LSTM, due to its flexible architecture. This will be evaluated in the next study, which will also concentrate on the analysis of the temporal sequence of observed behaviors improving the system's accuracy by considering complex behaviors, such as product concealing and movement hesitating. For this study, the YOLOv8 model was trained on the UCF Crime Database as well as an ad-hoc set of images manually collected from various sources representing abnormal behavior within the shopping scene. The model was trained across 150 epochs with the help of a curated and labeled dataset and performed with high accuracy of classifying about 95% classes, proving its efficiency in recognizing normal and suspicious activity across static images.

Considering the automated retail surveillance application, there are ethical concerns, specifically regarding biases and implications of false positive detections. One major issue is population bias in the training data – the system could end up unfairly targeting people of a certain age, gender or ethnic profile. This bias can result in discriminatory treatment of shoppers, and it raises critical ethical and legal questions. More generally, false-positive detections are a real concern as they may lead to false incriminations of innocents and may therefore erode public confidence in the surveillance authority. Methods We propose a human-in-the-processing loop for flagged cases to minimize this risk, to make model transparent, and to regularly audit and improve the model using diverse and representative datasets. It is crucial to address these ethical issues to ensure fairness, accountability and social acceptance of surveillance systems in the retail industry.

The ethical issues on the proposed system were mitigated by a number of procedures such as: training multiple parameters with diverse dataset to mitigate demographic bias; regular model review with new data to check for drifts in the model behavior; including human feedback early in the decision process; tuning model sensitivity in order to reduce the rate of false positives; keeping track of all outcomes to ensure due process and transparency.

The YOLOv8 model was trained on UCF-Crime data to detect robberies associated with human behavior. This data was augmented with additional images from diverse cases and multiple patterns. The settings used during training were (150) epochs, a batch size of (16), and an image size of (640 × 640). To ensure the transparency and reproducibility of the proposed methodology system, the training dataset contains (15,919) images manually annotated using the Roboflow platform to ensure classification accuracy regarding human-object interactions observed in retail settings. These images are evenly distributed into normal and suspicious categories. Normal behaviors include picking up objects, looking at them, examining them, and returning them to shelves, while suspicious behaviors include picking up objects and placing them in pockets, bags, or clothing. Table 1 shows a balanced image distribution, with each category containing (7,959) images. This even distribution of categories helps reduce bias in the training process, enabling the model to effectively learn the attributes that characterize each behavior.

Table 1. Balancing distribution of training data

We also reported box loss, classification loss, and DFL on the validation set. The test results were similar to the training results, suggesting no overfitting which means no overlearn on the training data. Results from this stage are presented in Figure 2.

2.jpg

Figure 2. Loss analysis result

5.1 Loss analysis

One of the key metrics for evaluating model performance is loss. In this research, we evaluated three main types of losses:

Train / box loss: This represents the loss resulting from identifying the bounding boxes of objects. We observed a gradual decrease in this value over the epochs, indicating that the model was able to locate objects more accurately with each training cycle.

Train / classification loss: The results suggest the improvement in hits rate (i.e., accuracy) over epochs.

Train / distribution focal loss: We added the DFL to assist the model in more fitting with imbalance distributed object. The loss on the less frequent object-related challenges was substantially smaller, indicating that the model is able to address these as well.

5.2 Precision and recall analysis

Precision: The accuracy of predicting relevant objects was measured. The results showed a gradual improvement in the model's accuracy with each epoch. Accuracy increased significantly, demonstrating the model's ability to accurately identify objects. Figure 3 illustrates this metric.

3.jpg

Figure 3. Precision analysis result

Recall: The recall results reflect the model's ability to find the correct objects among all relevant objects. The results showed that recall increased over the epochs, meaning the model was able to detect a greater number of relevant objects. Figure 4 illustrates this metric.

4.jpg

Figure 4. Recall analysis result

mAP50: A measure of the mean average precision at (50%) of the threshold. The results showed that the model was highly efficient at detecting objects, with a high mAP50, reflecting the model's ability to predict objects with good accuracy.

mAP50-95: A more comprehensive measure of model performance across a range of thresholds from (50%) to (95%). These values demonstrated good performance in accurately classifying objects across multiple thresholds. Table 2  shows the results of the last five training epochs. The last epoch showed good results, with an accuracy of nearly (95) percent.

Table 2. The training for last five epochs results

5.3 Confusion matrix analysis

To fully validate the results of the proposed theft detection system, not only the confusion matrix but also the classification report were used, as shown in Figure 5 and Table 3. These tools facilitate the deep investigation of the model's predictive performance beyond the aggregate accuracy, in particular, in the case of every class implicated. The approach was validated on a (726) test images belonging to two categories: normal and suspicious. As can be seen from the confusion matrix, the network could accurately identify 480 of (500) of the normal and (210) of (226) of the suspicious images at approximately (95.03) accuracy. Further validation was conducted with the use of the classification report. These results indicate that the model is robust enough to correctly identify both normal and suspicious behaviors (Figure 6), and it performs especially well in the suspicious class, which is fundamental in real scenarios of theft detection.

5.jpg

Figure 5. Confusion matrix

Table 3. Report classification

6a.jpg

6b.jpg

Figure 6. Visual results of the system

5.4 Comparison with other models

YOLOv8 was also evaluated against different models in object detection, to wit, YOLOv5, Faster R-CNN and SSD. The numbers indicated that there is a distinct difference among performance of the models, YOLOv8 being the best among the others for most of the metrics below:

YOLOv5: It achieved an mAP@0. 5 of about (90%), but (9%) lower recall than YOLOv8.

Faster R-CNN: Although it did achieve relatively good results, it needed more time in prediction, thus, it was not efficient for real-time applications.

SSD: It was the least accurate, but most successful at predicting speed and a small model size.

The final comparison between the models was shown in the Figure 7. These findings testify YOLOv8’s better performance in capturing theft-behaviors in realistic scenarios. The method offers high accuracy with execution speed and it is suitable for real time video surveillance application. Furthermore, the large differences in the other models emphasize that the choice of model depends on what is more important (speed / accuracy / size) for the target application. To compare performance consistency across different object detection models, we repeated five independent experimental runs for each model and wrote down their Precision scores in forms of As shown in the Table 4.

7.jpg

Figure 7. Model comparison

Precision scores between these models were significantly different as confirmed by a one-way (ANOVA) test (F(3,16) = 342.61, p < 0.001). Specific differences were investigated using a pairwise t-test against the baseline models and YOLOv8. Our findings revealed that YOLOv8 significantly exceed the performances of both YOLOv5 (t(8) = 12.33, p < 0.001), Faster R-CNN (t(8) = 22.33, p < 0.001), and SSD  t(8) = 34.00, p < 0.001) illustrated in Table 5. These results prove the better and statistically significant accuracy of the YOLOv8 model with respect to the other considered detection methodologies as shown in Table 6. Table 7 shows the performance numbers across various metrics of our YOLOv8 model.

Table 4. Precision score values

Table 5. Mean and standard deviation result

Table 6. Paired t-test results between YOLOv8 and other object detection

Table 7. YOLOv8 performance across various metrics