Mobile Phone Usage Warning System for Driver Focus Monitoring

This section provides a detailed explanation of the research and methodology used. This methodology is a guide to the preparation of this research with several stages consisting of literature review, establishing a test strategy, conducting tests and performing analysis, and drawing conclusions based on the test results.

2.1 Literature review

When developing a model to detect distracted driving behaviors, such as phone use while driving, it is important to carefully consider and explore the different stages. According to Ma et al. [8], the development of detection models should consider many factors, such as driver behavioral characteristics, appropriate experimental conditions, and efficient integration of algorithms such as YOLOv8 with MHSA (Multi-Head Self-Attention) attention mechanisms. This optimization aims to improve detection accuracy, ensure computational efficiency, and enable real-time implementation in edge devices such as Jetson Nano or Raspberry Pi.

In such studies, it is important to have a thorough understanding of the specific requirements for detecting distracted driver behavior, such as phone use. This includes a thorough analysis of the relevant environmental and experimental conditions, as well as the characteristics of the driver’s behavior to be detected. In addition, the algorithms used, such as YOLOv8 enhanced with attention mechanisms, must be designed to accurately and efficiently capture the dynamic nature of the detected objects in both test scenarios and real-time applications [9].

The development of the distracted driver behavior detection model consists of several main phases. The first phase involves identifying the characteristics of the driver's behavior, the nature of the distraction, and the relevant environmental conditions. Processing methods such as the integration of YOLOv8 with MHSA mechanism were chosen to improve the accuracy of behavior detection.

The next step is to implement the system in appropriate hardware, followed by validation through testing with real data. The test results compared different conditions such as phone use, fatigue, or other distractions. The integration of attention-based algorithms showed a significant improvement in the accuracy of distracted driving detection under varying driving conditions.

Identifying the behavioral characteristics of distracted drivers, such as phone use while driving, can be based on literature data and analysis of detection system specifications. Algorithms and processing methods, such as YOLOv8 with attention mechanism, should be carefully selected to ensure detection accuracy. The next process involves the design and implementation of a computer vision-based detection system that is integrated into the hardware to detect the driver’s behavior in real time and provide warnings when necessary.

Following the development phase of the distracted driving detection system, the prototype system will enter the testing and characterization phase. In this phase, experiments will be conducted to determine the accuracy of detection of driver phone use and to validate the detection range. The detection model will be validated by evaluating its ability to accurately detect distracted behavior against other detection models that are accurate, using analytical calculations and theoretical modeling to ensure its effectiveness.

Some approaches to detecting driver distraction involve the use of smartphone sensors and AI (Artificial Intelligence) systems. Smartphone-based systems can use sensors such as accelerometers, gyroscopes, and magnetometers to detect changes in position or hand movements that characterize distracted behavior, including the use of a mobile phone [10]. Use of vehicle-mounted sensors or other devices, such as smartphones, to detect changes in driver behavior that may impair their ability to respond to dangerous situations [11].

Another approach is to develop driver behavior monitoring technologies to identify and assess significant distractions caused by driver behavior, such as the use of electronic devices while driving. Technologies that use a variety of sensors and wearable devices to detect deviations in the vehicle and in the driver's head movements to warn the driver of potential hazards [12]. Use motion sensors and the IMU (Inertial Measurement Unit) to track the driver’s behavior in real time. This helps identify normal and abnormal behavior while driving [13]. Utilizing sensors to monitor the driver's physical and psychological state [14].

The implementation used in this study uses a system that detects when a phone is being held. This system works when a phone is visible in the frame and under conditions such as holding a phone or making a call. This condition can be influenced by the presence of sufficient light. That is, if there is enough light, the detection process will run smoothly and have high accuracy. This idea can be proven in several studies on this topic, such as research conducted by Peng et al. [15]. Their research introduces NLE-YOLO, a low-light targeting detection network developed based on YOLOv5 to overcome the problems of insufficient illumination and noise interference. In addition, YOLOv7 was trained and validated using the ExDark dataset to detect twelve classes in a dark environment [16]. The results showed a significant improvement in mAP (Mean Average Precision) compared to previous studies using the same dataset.

Efficiently Expanded Layer Aggregation (E-ELAN) technology developed in the YOLOv8-based detection system. In this study, the system is proposed to be optimized for low light environments. This detection model is designed to mimic real-world conditions where varying lighting levels affect the detection accuracy. To achieve the best performance, the incorporation of multi-scale features in E-ELAN is used to improve sensitivity to small objects. In contrast, the initial layers of the model, such as CSP, serve as an efficient base feature processing platform to support generalization capabilities under different lighting conditions.

The E-ELAN architecture can be seen in Figure 1, which is an explanation of the architecture.

1.png

Figure 1. The architecture of E-ELAN [16]

In another research by Yu and Choi [17], the focus of this work is to provide information about the detection results. Figure 2 shows an approach that aims to improve risk assessment in autonomous driving by providing in-depth information for each detected object. Depiction of an object detector with depth estimation using images from a monocular camera.

2.png

Figure 2. Network detection model [17]

Another research conducted is to create a framework for real-time object recognition on mobile devices through the co-design of compression and compilation by introducing YOLO bile [18], which is used to develop phone detection applications using mobile devices. This research discusses a real-time mobile phone usage monitoring system using the YOLOv5 algorithm [19]. In addition, this research was conducted to define various techniques to specifically detect objects running in real time with YOLO-LITE without using the GPU [20].

There are also other studies focusing on vehicle detection with YOLOv4 and vehicle tracking with DeepSORT, with models trained on specialized datasets including motorcycles, cars, buses, and trucks, using TensorFlow as the main platform and CUDA for computation [21]. Detection of vehicles against the direction in real time using computer vision based methods, with three main detections, namely, detection of vehicles with video recordings using the YOLOv3 algorithm to generate bounding boxes, tracking vehicles within a certain area using the centroid tracking algorithm, then identifying vehicles moving in the wrong direction by analyzing changes in centroid position in each frame, that when a vehicle is detected against the direction, the system will automatically capture its image for further documentation [22].

To detect mobile phone use in a video or image in real time, the YOLOv8 is used [19]. YOLO is a deep learning-based object recognition system that can quickly and accurately recognize and classify objects. The system works by training a model on a dataset containing images of people using a phone, allowing the model to recognize visual patterns such as hand position [23], device shape [24], and user interaction [25]. With this approach, mobile phone use can be efficiently detected under different lighting conditions and shooting angles.

The advantage of the YOLOv8 algorithm is its efficiency and accuracy in detecting objects in real time. This capability allows for fast response to changes in position and user interaction with devices such as phones. This makes YOLOv8 very suitable for applications that require fast and accurate monitoring of user activity. In addition, YOLOv8 is known for its easy integration into various computer vision-based systems and its support for various hardware, including those with limited resources [26]. Despite its significant advantages, this algorithm has limitations. One of these is the need for a large and diverse data set to achieve optimal results, as well as the challenge of detecting objects under poor lighting conditions or other environmental disturbances. Therefore, it is important to consider these elements when designing a YOLOv8-based system to ensure reliable results.

The YOLOv8 algorithm is available in several model and size configurations that can be customized to meet application-specific needs. This algorithm is suitable for use under predefined conditions. The research uses a YOLOv8 model with an architecture optimized for detecting small objects, such as mobile phones. With the ability to accurately detect objects within a wide field of view, YOLOv8 provides the flexibility to cover different user scenarios. The model is able to capture visual data from a relatively large area in a single observation frame, enabling detection of mobile devices and user interactions with better coverage than traditional object detection methods [27].

Another study [28] discussed the Automatic Passenger Counting (APC) system developed using computer vision with the Haar Cascade algorithm on the Raspberry Pi processing unit to detect passengers, including those wearing glasses and masks. The test results show an accuracy value of 60% with certain scaleFactor, minNeighbors, and minSize. The best results were obtained with the minimum values of the scaleFactor a minNeighbors variables. This system can be integrated with EDR (Event Data Recorder) to provide real-time vehicle information.

Based on Figure 2, the YOLOv8 architecture consists of three main components: Backbone, Neck-PAN, and Head. Backbone is responsible for extracting features from the input image, Neck-PAN integrates features at different scales (small, medium, and large), while the Head generates the final prediction in the form of bounding box, confidence score, class probability, and depth. In the context of mobile phone usage detection, this system acts as a receiver of visual signals. The Backbone acts as a dermis layer that captures the initial information, the Neck-PAN processes the information in a structured manner, and the Head acts as a decision layer that determines the presence, object type, and user interaction.

The system involves various key components, including performance evaluation through precision, recall, and accuracy metrics to ensure system reliability. In terms of hardware, cameras, processing units, and IoT devices are used to support the detection process. Frameworks such as TensorFlow and OpenCV are applied in data processing, while methods include YOLOv8 algorithm, data augmentation, preprocessing, transfer learning, and post-processing to improve system performance. Data management includes data collection, annotation and storage to ensure the completeness and quality of the data used. The focus of detection is to identify the presence of a phone and its use by the driver as a key element of the system. This system enables accurate, real-time detection of the user's interaction with the mobile phone. Figure 3 shows the overall mind map of the mobile phone usage detection architecture.

3.png

Figure 3. Previous studies of phone detection system

Compared to earlier detection systems based on YOLOv5 and YOLOv7, YOLOv8 offers a more efficient architecture and improved accuracy for small-object detection. YOLOv8 integrates an ELAN, which enhances feature fusion and spatial representation, allowing it to detect phone usage even when there is partial occlusion or overlapping with the hand. These improvements are crucial for accurate real-time driver behavior monitoring.

2.2 System architecture

This study utilizes two datasets: A mobile phone usage dataset and a mobile phone object dataset. The phone usage dataset includes labeled instances of drivers using or not using phones in real vehicle settings. The phone object dataset contains diverse images of mobile phones under various angles and lighting conditions. Both datasets are representative of common in-cabin driver scenarios and were selected for their relevance to the real-world detection task.

Figure 4 is the system architecture that will be developed in this research. This system will work and be installed on the dashboard of the vehicle. The system uses the YOLOv8 algorithm with images or videos from the cabin camera processed through the ME-YOLOv8 pipeline to detect distraction behavior or signs of driver fatigue. The input data is optimized by pre-processing such as resizing to a resolution of 640×640 pixels and data enhancement [29].

4.png

Figure 4. Mobile phone usage detection architecture system

This driver behavior detection system is designed to improve driving safety, with a focus on detecting phone use while driving. The system uses an in-cab camera aimed directly at the driver to monitor driver activity in real time. The camera captures visual data indicative of phone use, such as making a call or holding a phone. The data is then processed by the YOLOv8 algorithm running on the processing unit.

The system uses the YOLOv8 algorithm as the core of data processing because of its advantages in object detection speed and accuracy. YOLOv8, the latest development of the YOLO family of algorithms, is designed to detect and analyze objects in video in real time without requiring large computing resources. It provides reliable detection even under complex environmental conditions, such as lighting variations in a vehicle cabin. In addition, YOLOv8 supports multi-object identification, enabling the system to detect interactions between the driver's hands, face, and phone with high accuracy. Using YOLOv8, the system is not only fast, but also lightweight enough to run on even low-spec devices, making it a cost-effective yet effective solution for everyday in-vehicle use.

2.3 System workflow

The functionality of the mobile phone usage detection system can be illustrated in the workflow described in Figure 5.

5.png

Figure 5. Flow chart of phone usage detection

The diagram above explains the working process of the driver’s phone usage detection system, which aims to monitor and improve driving safety. The process starts with input from a camera that records the driver's condition in real-time. The visual data obtained is processed using the YOLOv8 algorithm to detect the presence of objects such as mobile phones in the driver's area. This algorithm was chosen due to its ability to accurately recognize various objects even in diverse lighting conditions.

Once the phone is detected, the next step is to evaluate the position of the object. The system checks whether the phone is in the driver's face or hand area, which indicates that the phone is in use. To ensure accuracy, the system also checks the trust score value generated by YOLOv8. This value indicates the level of confidence in the detection, and only objects with a confidence score greater than 40% are processed further. This 40% threshold was selected based on an internal ROC curve analysis, which showed an optimal trade-off between true positives and false positives at this point. Additionally, other works on YOLOv8 for real-time detection often use similar confidence levels. This step is designed to minimize detection errors, such as misidentifying other objects like phones.

If all conditions are met–the phone is detected, its position is in the relevant area, and the trust score is high enough–the system will act by alerting the driver. This warning can be an audible alarm, vibration, or visual notification, depending on the system configuration. The main objective of this process is to increase driver awareness of the dangers of using phones while driving, while helping to reduce the risk of traffic accidents due to visual or attention distractions. The system is designed as a preventive measure to improve road safety.

2.4 Data processing and augmentation

Prior to training, all images were preprocessed and augmented. Preprocessing steps included resizing to 640x640 pixels and applying auto-orientation. Augmentation involved horizontal and vertical flips, random crops (0-12% zoom), blurring (up to 1.9px), and noise addition (up to 1.96%). These parameters were selected to simulate real-world distortions such as motion blur, occlusion, and variable lighting. The augmented dataset improved the model’s robustness and generalization to diverse driving conditions.

Before conducting the test, two main datasets were prepared, namely the mobile phone usage dataset and the mobile phone dataset. The mobile phone usage dataset includes images of the driver holding a mobile phone, while the mobile phone dataset contains images of mobile phone objects in various angles and lighting conditions. The data was obtained from several sources, including real images taken directly and additional datasets uploaded through platforms such as Roboflow for augmentation. The annotation process was done manually by marking the relevant area of the object using a bounding box.

The details of the phone usage dataset and phone object dataset are presented in Table 1 and Table 2. These datasets were processed using various augmentation techniques, as listed in Table 3, to enhance robustness under different lighting and perspective conditions.

Table 1. Dataset of mobile phone usage

Table 2. Phone dataset

Table 3. Preprocessing and augmentation data

The dataset has a total of 3264 images for the mobile phone dataset and 3707 images for the mobile phone usage dataset. Using the following conditions described in Table 3 regarding preprocessing and data augmentation.

The augmentation values were selected based on prior studies and experimental validation. For example, the 12% zoom level simulates camera drift, while up to 1.9px blur reflects motion-related distortions. Noise was injected to replicate low-quality sensor conditions in real vehicles. These techniques were carefully calibrated to preserve key visual features while introducing realistic variability to enhance the detection model's robustness.

Overall, image processing is divided into three subsets: training, validation, and testing. Most of the data was allocated for model training, while the rest was used for validation and testing to evaluate the performance of the algorithm. The total number of images for each class was adjusted to keep the dataset balanced.

2.5 Hardware implementation

The system is deployed on a laptop with octa-core processor specifications, 8GB RAM, and a 1080p USB webcam. This laptop was chosen for its balance of computing power and portability, making it suitable for in-vehicle applications. The lightweight configuration of the YOLOv8 model allows for smooth performance, achieving 10-15 frames per second with real-time object detection accuracy.

In future implementations, the system can be optimized for deployment on lightweight edge devices such as the NVIDIA Jetson Nano, Raspberry Pi 4, or other ARM-based platforms. Given YOLOv8's flexibility and efficient model architecture, the detection pipeline can be quantized and pruned to reduce computational load while maintaining real-time inference. This enables the system to be integrated into vehicles with constrained processing capabilities, further supporting large-scale, cost-effective deployment for real-world driver monitoring applications.

2.6 Algorithm benchmarking

To validate the effectiveness of YOLOv8 in this study, a comparative benchmark was conducted using two previous versions: YOLOv5 and YOLOv7. All models were trained and evaluated on the same dataset configuration, with consistent input size, training epochs, and batch sizes. The comparison focused on metrics including accuracy, mAP@0.5, precision, and recall, which are critical for object detection tasks involving overlapping and small-sized targets such as mobile phones.

Table 4. Comparing different algorithm

Table 4 summarizes the results of the benchmark comparison between YOLOv5, YOLOv7, and YOLOv8.

Based on the results in Table 4, YOLOv8 significantly outperformed both YOLOv5 and YOLOv7 in all key metrics. Its superior mean average precision (mAP@0.5) of 89.5% indicates a more reliable object detection capability, particularly under complex conditions like occlusion and overlapping objects. The improved performance is largely attributed to YOLOv8’s advanced backbone and neck modules (such as ELAN and PAN-FPN), which enhance spatial feature extraction and multi-scale detection. These advantages justify the selection of YOLOv8 as the core detection engine for the proposed system.

This research successfully developed a mobile phone usage detection system using the YOLOv8 algorithm, achieving a detection accuracy of 92.5% and mAP@0.5 of 89.5%. By combining two datasets—the mobile phone usage dataset and the mobile phone objects dataset—the system demonstrated improved reliability in detecting complex interactions between users and devices under diverse conditions. The YOLOv8 algorithm, a state-of-the-art deep learning approach, enabled real-time detection with high accuracy, focusing on identifying mobile phones and user interactions in the driver’s face or hand area. The system was tested on lightweight edge devices, demonstrating potential for cost-effective and scalable deployment in real-world vehicle environments. Despite its promising performance, challenges such as a 42% false negative rate in low-light conditions and a 53% false positive rate for phone-like objects like wallets remain. Future work should focus on expanding the dataset with more diverse negative classes, applying adversarial training, and exploring infrared-based imaging to enhance robustness. Overall, this research contributes to improving road safety by providing an accurate, automated detection system that can be integrated into broader traffic safety and driver monitoring technologies.