Fatih Özyurt* | Ahmet Mira | Ayşe Çoban
© 2022 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
The COVID-19 pandemic continues to spread around the world at full speed, threatening public health. In response, the World Health Organization recommends various preventive measures to reduce the spread of the COVID-19 virus. Wearing a mask is one of the preventive measures to reduce the contagion of the disease, and many governments around the world advise people to wear masks. One of the prominent symptoms of coronavirus is high fever. A person with a fever above normal is likely to have contracted the corona virus. This requires the identification of people with a high fever in order to prevent the epidemic in the public arena. This situation has caused people who want to enter public places to need masks and officers who control their body temperature. The aim of this study is to detect people who do not wear masks or do not wear them properly, and also to detect people with high fever through a system. The proposed system is designed as a system that can be integrated into automatic door systems. The system was basically implemented by running Mobile Net, one of the deep learning models, on the Raspberry Pi card. In the proposed method, 97.0% accuracy was obtained. Experimental results show that the proposed method can effectively recognize face masks and whether people have a high fever. This work is necessary for many closed areas that will make masks and fever control in public areas.
COVID-19, face-mask detection, raspberry pi, automatic door system
After the slowdown in the spread of the Coronavirus, the world entered the phase of gradually lifting the ban on public facilities and institutions and recycling its economic wheel. However, the removal of this ban does not mean that the virus is over and will not spread again. This virus has affected the world badly in many ways. According to the statistics of the World Health Organization dated December 26, 2021, the Corona virus disease infected more than 276,436,619 people worldwide, causing the death of more than 5,374,744 people worldwide [1]. The global scientific community is making an intense effort to deal with this virus that spreads so quickly and puts society in danger [2]. This respiratory virus spreads mainly through close contact and in crowded and overcrowded areas [3]. In conjunction with lifting the ban, it is necessary to wear masks and maintain a safe distance between people to ensure that this virus does not spread again. In this context, it is a very functional method to check whether people wear masks at the entrance to public areas for precautionary purposes. The existence of a system that will perform this control instead of responsible persons would be a way to reduce costs.
One of the obvious signs that a person has contracted the virus is that they have a high body temperature. In order to prevent this person from entering the public areas, the personnel at the entrances also measure the temperature. Our study aims to develop a more functional method by adding the fever measurement feature to the same system.
The contribution of this study can be summarized as follows;
- Raspberry Pi-based model is proposed for face mask detection using OpenCV Deep Neural Network (DNN), Tensor Flow, Keras, and MobileNetV2 architecture used as an image classifier. The MobileNetV2 used is a lightweight CNN architecture and controls whether people have masks with high-performance success.
- A new scenario was realized by taking into account the temperature measurement of individuals. In addition, the fact that the proposed method is embedded-based shows that it can overcome more complex problems as it will enable many modules to be added in the future.
The following is a breakdown of the article’s structure: Literature review studies are given in section 2. In section 3, Material and Method focuses on methodology. Experimental Results and Discussion provides information on the findings and analysis under section 4. Conclusions and future prospects are discussed in section Conclusion.
The face mask detection problem is a process that can be solved with deep learning algorithms [4]. In this context, when we look at the literature, it is seen that some studies, albeit few, have been done. One of them is Shilpa Sethi et al. This study aims to develop a technology that can accurately detect face masks in public areas (airports, train stations, etc.). However, the available dataset for detecting masks on human faces is relatively small, resulting in difficult training of the model. So, Shilpa Shetty and his co-workers have used the concept of transfer learning to transfer the learned kernels from networks trained for a similar face detection task on an extensive dataset. The dataset covers a variety of face images, including masked faces, unmasked faces, masked and unmasked faces in a single image, and unmasked confusing images. With an extensive dataset containing 45,000 images, Shilpa Sethi and his co-workers technique achieves outstanding accuracy of 98.2% [5].
Another is Wadii Boulila et al. is the work they propose for real-time videos. Wadii Boulila and his co-workers have suggested with regard to exiting works is proposing an efficient and accurate approach for real-time videos. The proposed approach provides accurate detection of face mask wearing and whether it is worn in an appropriate way or not in real-time. To do this, a complete dataset is collected using public datasets and also their own one. In addition, the MobileNetV2 is used as a DL architecture for face mask detection. This approach has the advantage to be fast and suited to edge devices, and it provides excellent results for object detection. The proposed solution can be implemented in real-world surveillance cameras in public areas to check if people are following rules and wearing masks. The classification report of the proposed CNN architecture is depicted that they can conclude that their model achieves excellent values in detecting face mask with 99% accuracy [6].
Teboulbi et al. [7] implemented the model on an embedded vision system consisting of a Raspberry Pi 4 board and a webcam. The experiments of this research are conducted on one original dataset. The dataset consists of a total of 3835 images of faces wearing and not wearing masks. This is a balanced dataset containing two categories, faces with masks (1919 images) and without masks (1916 images) with a mean height of 283.68 and mean width of 278.77. For all simulated deep learning models, as well as DenseNet, InceptionV3, MobileNet, MobileNetV2, ResNet-50, VGG-16, and VGG-19, the TensorFlow-GPU is used as a deep learning framework to train the deep learning models. The hyperparameters used in their experiments are summarized as follows: the batch size is set to 32, the training epochs are from 20 to 40, and the learning rate is set to 0.0001, with the Adam optimizer used to update network weights. The training platform uses Windows 10 OS with Intel® Core TM i7-3770 @3.4 GHz CPU and 16 GB RAM and an NVIDIA GeForce RTX 2070 GPU. MobileNet is one of the deep learning models intended to be utilized in low-hardware cost gadgets. Classification, segmentation, and object identification can be performed by operating the MobileNet model. The overall testing accuracy of this model is 96.32%.
Vasilev et al. [8] introduce a facemask detection model that is based on computer vision and deep learning. The proposed model can be integrated with Surveillance Cameras to impede the COVID-19 transmission by allowing the detection of people who are wearing masks not wearing face masks. The model is integration between deep learning and classical machine learning techniques with Opencv, Tensorflow and Keras.
In this article, we present an artificial intelligence system to monitor the extent of people's obligation to wearing face masks, where this system can be linked with cameras located in a particular institution or university. It will be learned whether a person is wearing a face mask and also information will be obtained by measuring the person's temperature.
Using an artificial intelligence system and by using a computer vision-based camera, and sensors, the system indicates anyone who has a temperature higher than 38 degrees. In the implemented purpose method, the Raspberry Pi was used, which is a small computer that does not exceed the size of credit cards and has a low cost that can be connected to a computer screen or TV screen.
During the model proposal, deep learning was adopted [9], a convolutional neural network (CNN) was used to detect faces, and the architecture used to train the model is (MobileNetV2) [10].
The rest of this paper is organized in sections as follows. Section 2 covers material and method. Findings and discussion in Section 3. Finally, Section 4 concludes the work with conclusion.
In this section, the materials, methods and techniques used in the proposed method are mentioned and their relations with each other, positive and negative features are given.
Our study is divided into two parts, hardware components and software components.
3.1 Hardware components
In this section, information is given about the hardware components used in the purpose method and the electrical circuit obtained.
3.1.1 Raspberry Pi
Raspberry Pi is a single board computer is the name given to all of the small and low power consuming cards that have the necessary hardware for a computer to work on it [11]. These small cards usually have a processor Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and RAM, as well as connections (USB, Ethernet) to exchange information with the outside world. It consumes little electricity and does not contain mechanical/moving parts. Since Raspberry Pi is both a micro-controlled processor and a single-board computer, it is used for various purposes and takes its place in many maker goals. The General-Purpose Input/Output (GPIO) pins on it also allow you to use the Raspberry Pi board computer just like an Arduino, NodeMCU. Here, using the Raspberry Pi has its own advantages. Since we will do a method with a real computer, it provides great convenience in terms of both software and hardware. Raspberry Pi 4 Model B -8GB Ram version was used in this method.
3.1.2 Raspberry Pi camera
Used Raspberry Pi Camera to detect face recognition and Mask and it has these features, there is a fixed focus lens on the 5 MP resolution camera. Offering a static picture resolution of 2592 x 1944 pixels, the camera supports 1080p30, 720p60 and 640x480p60/90 resolutions for video shooting.
3.1.3 Raspberry Pi display
To show the interface and its results, and not need any other hardware like “computer, etc.” Raspberry Pi Touch Screen is used.
3.1.4 MLX90614 non-contact infrared temperature sensor
MLX90614 Temperature Sensor was used to measure body temperature.
3.1.5 SG 90 mini servo motor
Servo motor is used for demonstration purposes instead of automatic door. Tower Pro SG90 is a mini Servo motor ideal for small mechanisms with plastic gears. With the PWM signal that can be received from many microcontrollers, its position can be adjusted easily. Its speed at operating voltage of 4.8 Volts is 0.12 seconds/60 degrees.
3.1.6 RGB LED
RGB LED is used to show the accuracy of the method, RGB – is a term formed by combining the initials of Red (R), Green (G), Blue (B). General working principle; by using these three colors, it can give a lot of color in different combinations. RGB LEDs can give 16 colors with the help of a control circuit. Besides that we used Buzzer, buzzer is used for sound response and alarm in our purpose method. Buzzer is a device designed to transmit sound to large areas.
3.1.7 Electric circuit
As shown in Figure 1, the virtual electric cycle made on the simulation program is shown, In Figure 2, the actual electrical circuit and bindings were created as follows:
•Servo Motor: GPIO-17 is connected with 5V and Ground.
•RGB LED: Connected with GPIO27, GPIO22, 5V and GPIO23.
•BUZZER: Connected with GPIO24 and Ground.
•MLX90614 Temperature Sensor: Connected with SDA, SCL, 5V and Ground.
•Pi 7 touchable Display: Connected via SDA, SCL, 5V, Ground and DSI port.
•Pi Camera: Connected via DSI port.
Figure 1. Virtual hardware representation
Figure 2. Hardware representation
3.2 Software components
In this section, information about the software components, data set, Python libraries and methods used in the purpose method is given.
3.2.1 RGB LED
To train our model, we divided our dataset into two classes. Our first class consists of mask-wearing images and in this case we have 2902 images. Our second class consists of images of people who do not wear masks and in this case we have 2872 images [12]. As shown in Figure 3, the left side is the class of those who do not wear the mask or wear it incorrectly, and the right side is the class of those who wear the mask correctly.
Figure 3. Dataset example
3.2.2 Training mask detection model
After dividing the dataset into two classes, the next step is to train and build the model. To do this, a model was created by training the dataset using Tensorflow and Keras [13] libraries. At the same time, the accuracy of the model was calculated to ensure its efficiency. In this model, CNN was used as the classifier and MobileNetV2 was used as the architecture [14].
MobileNetV2, a family of general purpose computer vision neural networks designed with mobile devices in mind to support classification, detection and more. The ability to run deep networks on personal mobile devices improves user experience, offering anytime, anywhere access, with additional benefits for security, privacy, and energy consumption. As new applications emerge allowing users to interact with the real world in real time, so does the need for ever more efficient neural networks. MobileNetV2 uses depthwise separable convolution as efficient building blocks. And linear bottlenecks between the layers, and shortcut connections between the bottlenecks [10]. The general representation of the MobileNetV2 architecture is given in Figure 4. The Scikit-Learn library was used to print the performance report of the model. Matplotlib library was used to plot the training curve according to the error function.
Figure 4. MobileNetV2 architecture
3.2.3 Face detection using the Haar cascade model
Used opencv to detect real-time face from a live stream via Raspberry Pi Camera. Videos basically consist of frames, which are still images. It performs face recognition [15] for each frame in a video. So when it comes to detecting a face in a still image and a face detecting in a real-time video stream, there is not much difference between them. The Haar Cascade algorithm, also known as the Voila-Jones algorithm, is used to detect faces. It is basically a machine learning object detection algorithm used to identify objects in an image or video. In OpenCV we have several trained Haar Cascade models saved as XML files [16]. This file was used instead of building and training the model from scratch. The file “haarcascade_frontalface_default.xml” was used in this method.
3.2.4 Real-time mask detection and temperature measurement
After real-time face detection from a live stream via the Raspberry Pi Camera, it detects whether a person is wearing a mask based on the mask detection model we trained [17-20]. When the mask is detected, the sensor method is called to get Body Temperature and the body temperature is checked. When the body temperature is more than 38, a signal is sent for the RGB LED to turn Red, the Buzzer to alarm and the Servo motor to turn off. When the mask is attached and the body temperature is less than 38, a signal is sent to turn on the green color and turn on the Servo motor, Figure 5 shows the System structure.
Figure 5. Real-time algorithm structure
Our proposed method was performed step by step as shown above. The results, such as accuracy, sensitivity, and precision, that we obtained in the trained mask detection model are shown in Table 1. The training loss and accuracy of our model are shown graphically in Figure 6.
Three scenarios were created to test the proposed study in real life. The scenarios made are given in Table 2.
In scenario 1, the person put on his mask and had his temperature measured by sensors. His fever is below 38 degrees. As can be seen in Figure 7, the green light is on. (The green light indicates that there is no problem. In this case, the servo motor will work.)
Table 1. Experimental results
|
(%) |
Accuracy |
Precision |
Recall |
F1-Score |
|
Masked |
0.97 |
0.97 |
0.96 |
0.97 |
|
No-Masked |
0.97 |
0.96 |
0.97 |
0.97 |
Figure 6. Train loss, train_acc
In scenario 2, the person did not wear his mask and had his temperature taken by sensors. His fever is below 38 degrees. As can be seen in Figure 8, the red light is on. (The red light indicates a problem. In this case, the servo motor will not work.)
In the 3rd scenario, the person put on his mask and had his temperature measured by sensors. But his fever is above 38 degrees. As can be seen in Figure 9, the red light is on. (The red light indicates a problem. In this case, the servo motor will not work.)
Table 2. States and scenarios
|
States / Scenarios |
1 |
2 |
2 |
|
Mask |
√ |
X |
√ |
|
Body Temperature |
< 38 |
< 38 |
> 38 |
|
Result |
√ |
X |
X |
Its three states are shown in Figure 7, Figure 8 and Figure 9, respectively.
The comparison table of studies similar to our study is given in Table 3. It should be noted that the method proposed in this study also measures fever. As seen in Table 3, the studies are only about whether people wear masks or not.
Figure 7. Case-1 (Mask is attached and Body Temperature is less than 38.)
Figure 8. Case-2(No mask attached and body temperature less than 38.)
Figure 9. Case-.3(Mask is attached and body temperature is greater than 38.)
Table 3. Results of similar studies
|
Study |
Proposed Techniques |
Accuracy (%) |
|
Mata [21] |
CNN model |
60 |
|
Balaji et al. [22] |
VGG-16 CNN |
N/A |
|
Bhuiyan et al. [23] |
YOLO-v3 model |
86 |
|
Kaur et al. [24] |
CNN-based approach |
86 |
|
Fan et al. [25] |
Residual contextual awareness module |
91 |
|
Nagrath et al. [4] |
SSDMNV2 |
92.64 |
|
Proposed method |
MobileNetv2 |
97.0 |
As can be seen in the table above, many applications that control masks have been proposed recently. However, these methods also do not measure fever. When the table is examined, it can be observed that the proposed method has high performance.
Due to the COVID-19 pandemic, the whole world has faced a major health crisis. Scientists and government authorities from all over the world are working hard to control the spread of the coronavirus. It is a known fact that the risk of contracting COVID-19 infection is higher indoors. This makes it mandatory to wear masks, especially indoors. In addition, the most distinctive feature of people infected with COVID-19 is that they have a high fever. In this respect, it is important to control whether a mask is worn at the entrance of closed spaces and to have systems that measure fever. This study is designed as a system that controls both of these conditions. By taking the real-time video frames of the persons through the Raspberry Pi camera, the facial regions of the persons are detected using the OpenCV Library, and the fever measurement is taken by determining the mask status based on the trained model. Then, when any person violates the precautions, the automatic door will not open and a visual and sound alarm will be given, and the automatic door will open when the person takes his own precautions. This method has been developed to make the decision to open automatic doors by controlling the precautions. A success rate of approximately 97% was achieved in this method.
[1] World Health Organization. (2020). Coronavirus disease 2019 (COVID-19): Situation report, 73. https://www.who.int, accessed on 3 April 2022.
[2] Tuncer, T., Dogan, S., Ozyurt, F. (2020). An automated residual exemplar local binary pattern and iterative ReliefF based COVID-19 detection method using chest X-ray image. Chemometrics and Intelligent Laboratory Systems, 203: 104054. https://doi.org/10.1016/j.chemolab.2020.104054
[3] Tuncer, T., Ozyurt, F., Dogan, S., Subasi, A. (2021). A novel COVID-19 and pneumonia classification method based on F-transform. Chemometrics and Intelligent Laboratory Systems, 210: 104256. https://doi.org/10.1016/j.chemolab.2021.104256
[4] Nagrath, P., Jain, R., Madan, A., Arora, R., Kataria, P., Hemanth, J. (2021). SSDMNV2: A real time DNN-based face mask detection system using single shot multibox detector and MobileNetV2. Sustainable Cities and Society, 66: 102692. https://doi.org/10.1016/j.scs.2020.102692
[5] Sethi, S., Kathuria, M., Kaushik, T. (2021). Face mask detection using deep learning: An approach to reduce risk of coronavirus spread. Journal of Biomedical Informatics, 120: 103848. https://doi.org/10.1016/j.jbi.2021.103848.
[6] Boulila, W., Alzahem, A., Almoudi, A., Afifi, M., Alturki, I., Driss, M. (2021). A deep learning-based approach for real-time facemask detection. In 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 1478-1481. https://doi.org/10.1109/ICMLA52953.2021.00238
[7] Teboulbi, S., Messaoud, S., Hajjaji, M.A., Mtibaa, A. (2022). Face mask classification based on deep learning framework. In Advanced Practical Approaches to Web Mining Techniques and Application, pp. 175-188. https://doi.org/10.4018/978-1-7998-9426-1.ch009
[8] Vasilev, I., Slater, D., Spacagna, G., Roelants, P., Zocca, V. (2019). Python deep learning: Exploring deep learning techniques and neural network architectures with Pytorch, Keras, and TensorFlow. Packt Publishing Ltd.
[9] Kiran, K., Kiran, B.V., Sai, D.C., Vamsi, G.V., Salomi, P.R. (2021). Face mask detection using machine learning. SSRN. http://dx.doi.org/10.2139/ssrn.3925736
[10] Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C. (2018). Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510-4520. https://doi.org/10.1109/CVPR.2018.00474
[11] Brock, J.D., Bruce, R.F., Cameron, M.E. (2013). Changing the world with a Raspberry Pi. Journal of Computing Sciences in Colleges, 29(2): 151-153. https://dl.acm.org/doi/abs/10.5555/2535418.2535441
[12] Cabani, A., Hammoudi, K., Benhabiles, H., Melkemi, M. (2021). MaskedFace-Net–A dataset of correctly/incorrectly masked face images in the context of COVID-19. Smart Health, 19: 100144. https://doi.org/10.1016/j.smhl.2020.100144
[13] Gulli, A., Pal, S. (2017). Deep Learning with Keras. Packt Publishing Ltd.
[14] Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C. (2018). Mobilenetv2: The next generation of on-device computer vision networks. In CVPR.
[15] Goyal, K., Agarwal, K., Kumar, R. (2017). Face detection and tracking: Using OpenCV. In 2017 International conference of Electronics, Communication and Aerospace Technology (ICECA), pp. 474-478. https://doi.org 10.1109/ICECA.2017.8203730
[16] Wu, P., Li, H., Zeng, N., Li, F. (2022). FMD-Yolo: An efficient face mask detection method for COVID-19 prevention and control in public. Image and Vision Computing, 117: 104341. https://doi.org/10.1016/j.imavis.2021.104341
[17] Omar, I., Rasha, A. (2021). Automated realtime mask availability detection using neural network. International Journal of Computing and Digital Systems, 10: 1-6.
[18] Rao, M.S., Tejasree, K., Sathwik, P., Kumar, P.S., Sailohith, M. (2021). Real time face mask detection and thermal screening with audio response for COVID-19. Revista Geintec-Gestao Inovacao E Tecnologias, 11(4): 2703-2714.
[19] Hussain, S., Yu, Y., Ayoub, M., Khan, A., Rehman, R., Wahid, J.A., Hou, W. (2021). IoT and deep learning based approach for rapid screening and face mask detection for infection spread control of COVID-19. Applied Sciences, 11(8): 3495. https://doi.org/10.3390/app11083495
[20] Varshini, B., Yogesh, H.R., Pasha, S.D., Suhail, M., Madhumitha, V., Sasi, A. (2021). IoT-enabled smart doors for monitoring body temperature and face mask detection. Global Transitions Proceedings, 2(2): 246-254. https://doi.org/10.1016/j.gltp.2021.08.071
[21] Mata, B.U. (2021). Face mask detection using convolutional neural network. Journal of Natural Remedies, 21(12 (1)): 14-19. http://www.jnronline.com/ojs/index.php/about/article/view/784.
[22] Balaji, S., Balamurugan, B., Kumar, T.A., Rajmohan, R., Kumar, P.P. (2021). A brief survey on AI based face mask detection system for public places. Irish Interdisciplinary Journal of Science & Research (IIJSR). https://ssrn.com/abstract=3814341.
[23] Bhuiyan, M.R., Khushbu, S.A., Islam, M.S. (2020). A deep learning based assistive system to classify COVID-19 face mask for human safety with YOLOv3. In 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT), pp. 1-5. https://doi.org/10.1109/ICCCNT49239.2020.9225384
[24] Kaur, G., Sinha, R., Tiwari, P.K., et al. (2021). Face mask recognition system using CNN model. Neuroscience Informatics, 100035. https://doi.org/10.1016/j.neuri.2021.100035
[25] Fan, X., Jiang, M., Yan, H. (2021). A deep learning based light-weight face mask detector with residual context attention and Gaussian heatmap to fight against COVID-19. IEEE Access, 9: 96964-96974. https://doi.org/10.1109/ACCESS.2021.3 095191
