Embedded Wearable IoT System for Child Safety Based on Hybrid Deep Learning Classification

All over the world, more people are concerned about children because the number of public incidents and child disappearances is increasing. Every year, research shows that 64% of all abduction cases involved children, and one child goes missing every two minutes in the European Union [1]. The Malaysian National Crime Record Center found that child-related crimes went up by 10.5% from 2019 to 2020 [2]. The serious numbers highlight the need for solutions that quickly safeguard children in all environments. Using printed wristbands, surveillance cameras and centrally controlled systems for children has shown it is difficult to offer quick answers to security issues [3]. Such systems often cannot stop major incidents in places where there are many visitors, for example in museums and schools [4, 5]. As a consequence, new research has turned towards using GPS, GSM and wireless communication modules together with the Internet of Things (IoT) to design advanced monitoring systems. Some experts suggest that wearable devices that include GPS and GSM technology could send updates on a child’s location to parents, either through text or mobile application alerts [6-8]. Advanced tracking solutions now use both RFID for indoor positioning and GPS for outdoor tracking [9], along with techniques such as Time of Arrival, Angle of Arrival and Received Signal Strength [10].

A growing number of healthcare experts believe that human action recognition (HAR) technologies are very helpful for continuously monitoring patients with neurological illnesses. Hence, smart monitoring systems and patterns recognition for different applications are vital [11-16]. In this context, artificial intelligence and deep learning have achieved impressive results across various domains of applications, making them promising tools for improving health monitoring [17, 18]. Spotting seizures, for example, is a crucial part of such monitoring systems, as failure to do so can place a person’s life at risk. An important difficulty is that seizure-related events in HAR datasets are much less common than other activities which worsens the performance of traditional classification algorithms. To address this, we proposed an embedded system with hybrid architecture that processes sensors input frames by integrating convolutional neural networks (CNNs) for spatial feature extraction with long short-term memory (LSTM) for temporal sequence analysis. New developments used in wearables help identify and address health problems early which benefits those individuals—primarily children—susceptible to health emergencies. Furthermore, we put into practice biometric sensors and anomaly detection models incorporated into a wearable smart watch on an ESP32 microcontroller according to our system. While previous works only used geofencing [19-21], our system monitors both the child’s and parent’s device location in real time to send an alert when the child walks into an unsafe area. With the simple Flutter-tracker app for parents, they can track the satellite’s condition, set alert distances and receive instant notifications, helping meet the rising need for user-friendly parenting tools.

In addition, the system allows mobile usage and compatibility with various environmental setups since static RFID or Wi-Fi nodes are replaced by a flexible ESP32-based system that uses GPS and GSM for communication. The access to biometric and location details is allowed only to the school’s authorized members. With real-time monitoring, biometric analysis and intelligent detection, our system offers a powerful and flexible method for keeping children safe today. A patron can expand the foundation to elderly care, long-term health care and supportive health care when emergencies arise. The proposed method presents valuable contortions including the following:

1. A comprehensive framework leveraged the tracking capabilities of location and health status by integrating biometric sensors into a smartwatch.

2. The system offers a dual location monitoring mechanism for both child and parent using GPS and developed mobile app with flexible configuration of the safe zones.

3. The proposed model combined CNN and LSTM to integrate the spatial features with the temporal sequence pattern to recognize health anomalies, particularly seizures, in children based on their movements and heart rate.

4. The designed system can be extended easily to scalable use, such as elderly care, long-term patient monitoring, and emergency health interventions.

This section presents the details of the proposed system and all its components: hardware, software, and machine learning algorithms implementation to predicate child’s pattern behavior.

3.1 Hardware implementation

The proposed embedded system is designed as a wearable IoT which include sensing, tracking, and communication. This section presents the main components and focusing on the system functions. The central microcontroller ESP32-C3 SuperMini is used to process data and controls communication with sensors. To track motion and orientation of the child GY-521 (MPU6050) is used. SIM800L Module (SIM800L BO1) is utilized to send and receives SMS or data over mobile networks [30]. ATGM336H GPS Module is responsible for providing real-time geographic coordinates. The Pulse oximeter and heart-rate sensor MAX30102 is designed for monitoring blood oxygen levels and heart rate [31]. The system is powered by Li-Po Battery (3.3V, 1100 mAh). Figure 1 shows the proposed system. Figure 2(a) and (b) show the pub top and bottom layer.

Figure 1. The proposed system

Figure 2. The Pcb top and bottom layers for the proposed system

3.2 Software design

A cross-platform IDE application known as Arduino integrated development environment (IDE), is used to program the proposed micro controller, as exists primarily in Java coding language to operate on Windows and macOS and Linux platforms. Within the IDE code can be edited via a text editor that supports features including text movement capabilities and text search tools and formatting assistance while also providing single-step functions for Arduino boards including our micro controller Esp32-C3 Super Mini program compilation and upload processes.

In addition, Flutter tracking application was used. The mobile application delivered a pair of essential values to the ESP32 which includes the guardian's present GPS position and the security distance selected by the user (10, 15, 20 or 25 meters). The ESP32 obtains GPS coordinates from itself as well as from the received guardian data to perform the distance calculation between both points. The ESP32 sends an instruction to the SIM800L module so it initiates a GSM call to the guardian's phone after calculating distance measurements exceed the defined limit.

If the child moves beyond the predefined threshold, the system marks the child’s location as outside the safe zone. After that, the child’s movement with heart rate are continuously monitored, and these signals are sent to the AI unit to predicate the health status of the child by estimating the pattern behavior using the trained hydride model.

The ESP32 maintains a continuous data transfer to the mobile application that includes the location, quantitative reports of heart rate combined with oxygen level readings, motion information, and the decision of the AI unit to predicate the type behaviors (Normal, Pre-Seizure, and Seizure). The mobile application presents real-time health information and shows the GPS positions of both child and guardian at the same time. User interaction with locations is possible through the map view which utilizes OpenStreetMap integration from the flutter map plugin. The application implements real-time connection monitoring to maintain system reliability. The selected architecture allows microcontrollers such as ESP32 to execute an efficient communication protocol which uses minimal resources as well as the Flutter interface enables user-friendly interaction as shown in Figure 3. Note that the readings of the sensors are not shown as the system was in the disconnection state from the internet. Figure 4 shows a flowchart which demonstrating the overall process and the steps of the proposed system.

Figure 3. Child safety tracker platform

Figure 4. Flowchart of the proposed system

3.3 Machine learning implantation

Artificial intelligence and machine learning techniques have gained a prominent status and are now extensively applied across a wide range of fields. Thus, AI and ML methods can contribute significantly to resolving diverse issues of daily real-life challenges. Activity monitoring and human behavior analysis are considered as essential tasks in numerous types of practical applications in modern life. Human patterns understanding can be used in wide range of applications including healthcare, smart homes, surveillance systems, and security systems. For example, monitoring specific patterns in physical activities and behavioral responses may be very useful to detect anomalies such as medical emergencies, psychological distress, or even unauthorized behavior. The automation of this process with the integration of artificial inelegance techniques is crucial to achieve speed, accuracy, and adaptability.

On the other hand, mentoring children’s activities and their behaviors is very important task for parents, caregiver, and educators because it ensures their safety, offers the required support, and provides greater peace of mind. Therefore, we tended to present an automatic system based on AI to predict kids’ activities and detects any abnormal or risky behavior. For this purpose, we utilized SHAR-100-20 dataset as the primary source of the activity classification to provide the necessary data to train our system. SHAR-100-20 is publicly available dataset collected from 100 participants, designed to simulate 20 classes of physical activities for human actions recognition. We used this particular dataset because it offers a set of very comprehensive variety of activities which are highly relevant to safety mentoring for children’s behaviors. initially, we modify the dataset by restructuring the original output labels of the data to accommodate them for the training part of our system. To achieve that, a certified medical expert was consulted to reclassify the original twenty classes into three types of activities: normal, pre-seizure, and seizure. Then after that we sample 100,000 recordings for each class.

After the modification, the data labels were consolidated into three behavioral categories: normal, pre-seizure, and seizure of 300,000 measurements, few samples from the data are shown in Figure 5. The primary reason of this transformation was to align the dataset with the core objective of our activity recognizer model.

Figure 5. Few samples from the dataset

The learning process of our system can be divided into two steps. Firstly, we used the raw data of the collected sensors of acceleration (accel_x, accel_y, accel_z), gyroscopic movement (gyro_x, gyro_y, gyro_z), and heart rate. For this particular step, we utilized traditional methods of machine learning process where the input of the trained models is a vector of seven parameters and the output is the predication of the behavioral type. To perform the training process, we split the modified data into training part and testing part. We used the training segment of the data to apply the learning process of the predication models.

To evaluate the performance of the learned models, we applied cross validation strategy throughout the training process. A 10-fold cross validation technique was used by randomly dividing the dataset ten times into 70% to 30% segments ratio. In each iteration, we used the training portion to build the predication model, while the remaining portion of the data was utilized to measure the classification accuracy and the other evaluation metrics. Finally, the performance of the trained model was calculated by averaging the results across all ten iterations to provide a comprehensive assessment of its effectiveness.

A broad spectrum of classification methods has been proposed by researchers to improve the recognition performance, each algorithm exhibits unique characteristics, strength and limitations. Consequently, determining the best candidate classification technique for particular dataset and task can be both challenging and demanding task. To overcome this problem, we explored wide range of classification algorithms by training around 18 different models. The implemented classification techniques can be categorized into linear, nonlinear, single classification and ensemble of classification methods. Logistic Regression, Linear Discriminant Analysis, and Support Vector Machine were trained to build predication models as linear classifiers. On the other hand, Decision Tree, Quadratic Discriminant Analysis, and K Nearest Neighbors were trained as non-linear models. Additionally, we extended the scope of the training to include ensemble of classifiers by training Extra Trees Classifier, Random Forest Classifier, and Ada Boost Classifier. The primary objective of deploying various classification techniques is to achieve comprehensive comparative analysis.

The aforementioned classifiers have demonstrated their effectiveness across different types of applications involving static data and tabular input, where each measurement is considered entirely independent. However, these types of classification approaches do not consider the temporal history of the input and rely solely on the current state of entered data sensors. Hence, they are practically unsuitable for problems involving time series inputs, where data pattern and its dependencies are essential factors for accurate classification. Behavior monitoring using physiological signal analysis can be considered as time series problem where the input features are time dependent. In this analysis, the temporal relationship between sensors measurements observations can be very crucial for the classification performance. Therefore, we used in the next phase of our project time series models to implement the predication models.

Figure 6. The proposed model architecture

Table 2. Proposed model configuration

In order to capture the sequential temporal patterns across time domain of the input sensors, we aggregated 20 consecutive samples to create a single window of input segment. This window of input allowed the classification models to learn not only from the data measurements individually but also from the temporal patterns throughout multiple steps over time. The segmented time-windows of the inputs were used to train types of deep learning architectures: a pure CNN to capture the local spatial features across the temporal window, a pure RNN to capture the sequential dependencies relationships of the temporal patterns, and finally a hybrid CNN-RNN architecture network to leverage both local spatial features and temporal sequence learning capabilities. This hybrid architecture model allowed a comprehensive analysis to accommodate spatial and temporal features learning. Figure 6 shows the block diagram of the proposed hybrid architecture network. Table 2 illustrates the details configuration of the proposed network.

3.4 Evaluation process

In order to evaluate the trained models, we used Accuracy, precision, recall, and F1-score (as shown in the following equations) with predication time, size of the model, and number of learnable parameters (for deep learning model). The evaluated metrics can provide a complete analysis and illustrated a trade-off between efficiency and accuracy. The next section presents the results details of the conducted experiments.

Accuracy $\%=\frac{T P+T N}{T P+T N+F P+F N} \times 100$     (1)

Precision $\%=\frac{T P}{T P+F P} \times 100$     (2)

Recall $\%=\frac{T P}{T P+F N} \times 100$     (3)

F $1-$ Score $\%=\frac{2 \times \text { Precision } \text { × } \text { Recall }}{\text { Precision }+ \text { Recall }} \times 100$     (4)