Hamed A. Mahmood*
| Mohammad Tarik Mohammad | Akram T. Saeed | Mohammad M. Abbas
© 2023 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
The global population of visually impaired individuals continues to rise, as reported by the World Health Organization (WHO). The widespread prevalence of visual impairments necessitates innovative solutions for overcoming challenges related to spatial navigation and obstacle avoidance. This study introduces a multifunction assistive device, designed to augment safety measures and mobility for visually impaired individuals by acting as an artificial vision unit that issues cautionary alerts. The device is comprised of an array of sensors, microcontrollers, and wearable gadgets, providing immediate assistance to its users. Three primary components form the system: a smart glove, a smart shoe, and a smart hat. Insight obstacles are detected by the smart glove, which subsequently alerts the user. Ground obstacles, slopes, and wet surfaces are identified by the smart shoe. The smart hat is equipped with features that facilitate social distancing, maintaining a minimum distance of 2 meters to prevent viral infection, such as that of the Coronavirus, and enhance visibility at night. Users receive system notifications through sound or vibration. System evaluation demonstrated a high detection accuracy ranging from 95% to 99% in ambient obstacle detection, ensuring efficient obstacle avoidance and enhanced personal safety. Furthermore, the equipment effectively alerted users to maintain a safe social distance during the COVID-19 epidemic. A comparative analysis with existing assistive systems revealed that the proposed system offers a broad spectrum of functionalities, including obstacle collision avoidance, flame detection, water collision avoidance, hole detection, day and night visibility, and social distancing alerts. The proposed assistive device represents a practical, affordable, and transportable solution that empowers visually impaired individuals by promoting independence and enhancing overall mobility.
Arduino board, coronavirus, sensors, smart gloves, shoes and a hat, social distancing, visually impaired
Globally, visual impairments afflict approximately 2.2 billion people, encompassing near- or farsightedness. Notably, approximately 1 billion of these cases could have been prevented. This figure includes individuals with moderate to severe distance vision impairment or blindness resulting from conditions such as untreated vision problems (88.4 million), cataracts (94 million), age-related eye problems (8 million), glaucoma (7.7 million), retinopathy (3.9 million), and near vision impairment due to untreated presbyopia (826 million) [1].
To address this global health concern, extensive literature on the subject was reviewed to inform the objective of this research. Maragatharajan et al. proposed a device employing an ultrasonic sensor for object detection within a specific range, alerting the user via a buzzer [2]. Tavde and Khambhati presented a wearable system, namely, a smart shoe for visually impaired individuals, integrating an ultrasonic sensor, buzzer, battery, Arduino Uno, and jumper wires [3]. Sakthi et al. developed an invention allowing visually impaired individuals to navigate based on ultrasonic waves, providing alerts through buzzer sounds or vibrations [4].
A similar project by Batool and Naz utilized the Arduino Pro Mini 328 board and an ultrasonic sensor, worn like a band, which alerts the user by beeping or vibrating upon detecting an object [5]. Argüello Prada et al. aimed to develop a novel assistive system by combining two ultrasonic sensors and one infrared sensor to estimate the proximity and height of an obstacle, delivering this information via audio messages [6]. Yogesh introduced a smart cane with an alarm to assist visually impaired individuals navigating through challenging terrains [7]. Shamila et al. created a visual aid for the visually impaired that included a text-to-speech converter using an ultrasonic sensor, Arduino, and Braille Keyboard [8].
The device proposed by Ibtisam et al. uses an NI myRIO controller, a dual-core ARM (Advanced RISC Machine) Cortex-A9 processor, and an HCSR04 ultrasonic sensor to determine the distance to an object [9]. Singh et al. utilized Raspberry Pi 4 modules with ultrasonic sensors and cameras to help visually impaired individuals recognize objects around them [10]. Meanwhile, Albogamy et al. [11] proposed the use of the Turtlebot 3 robot, which has a microphone for voice recognition, a camera for student ID recognition, and a laser distance sensor. Masud et al. [12] suggested a method using a Raspberry-Pi, Camera, Arduino, and Ultrasonic Sensor attached to a person's stick.
Devi [13] designed a prototype for Raspberry Pi 3 Model B+, utilizing the YOLO method for object detection, and included sensors and a Pi camera for obstacle detection and object recognition, respectively. Kamaruddin et al. [14], used the Internet of Things (IoT) to develop smart assistive shoes, integrating vibration motors and ultrasonic sensors to alert users about obstacles. Hegde et al. [15], created smart glasses based on an IoT and Sonic Wave mechanism, along with a camera module. The user is notified of the distance to an object or person by the Espeak module when they come in front of the glasses. Neugebauer et al. [16], studied the application of a sensory substitution algorithm that translates 3D visual information into audio feedback, integrated with a mobile phone application. Paré et al. [17], tested the Guidance-SSD (GSSD), a novel smartphone application that builds on the "point-to-distance" concept presented by the EyeCane.
Elsonbaty [18] developed a conventional blind stick that serves as a walking aid for the blind by transmitting audio alerts through a headphone attached to a phone when obstacles are detected. Tachiquin et al. [19], created a wearable device for visually impaired pedestrians using a smartphone and a tactile-foot interface, which used augmented GPS (A-GPS) data to resolve orientation issues. See et al. [20], used a depth camera feature on a cellphone for object detection and obstacle avoidance, incorporating a mobile application for navigation that can be used with basic speech and gesture commands. Andrés et al., presented a system comprising a stereo camera as the main sensor, and established two prohibitions: 1. A general-purpose graphics processing unit processes depth and color data (GPGPU). 2. A wearable computer must process images [21].
After reviewing the previous studies and literature that dealt with the safety of the blind from accidents as a result of the obstacles and risks of infection with viral epidemics, and therefore it was found that the objective of this research is to design and build a multipurpose assistive system that overcomes the difficulties that visually impaired people encounter when navigating, seeing obstacles, and interacting with others. The suggested system incorporates numerous sensors, microcontrollers, and wearable gadgets to offer visually impaired people immediate support. The device provides a comprehensive approach to enhancing the safety and freedom of people who are visually impaired by integrating features such as obstacle detection, flame detection, water detection, hole detection, day and night visibility enhancement, and social distancing aid. Investigation for the performance and accuracy of the system will be performed in detecting obstacles, real time alerts, social divergence, and enhancing visibility.
The paper is arranged into, section two as a methodology that describes the system architecture, system design, and software implementation. Section three is concerned with results and performance assessment. introduced, and finally, the conclusion and future work is in section four.
The methodology defines the hardware of the proposed multifunction assistive device. The system comprises three main components: a smart glove, a smart shoe, and a smart hat. The smart glove innovation supported with a flame sensor and ultrasound sensors to detect the fired obstacle or without fire, then notify the user through a sound and vibration to avoid it. The smart shoe has ultrasonic sensors that can spot slopes and obstructions in the ground as well as a water sensor that can detect wet areas. The smart hat design is involved ultrasonic sensor, motion sensor, and light-dependent resistor sensor to facilitate social distancing and improve visibility at night. LED operation is at nightfall to notify car drivers that there is a person in their way and the person is blind. Also, social distancing alarm would be helpful to warn others to stay away from the blind person to maintain a distance of 2 meters from the blind person to ensure safe social distancing during of the Corona pandemic.
2.1 System architecture
With global technological advancement, microcontrollers have become the foundation for the design of a wide range of devices, making their use mandatory not only for sight but also for blind people. The proposed assistive system in Figure 1 includes obstacle sensing and distance sensing through an ultrasonic sensor, HC-SR04 to measure distance, which is processed by Arduino UNO board and Arduino Nano board based on the component of propose system. The active passive buzzer sounds an alarm based on the distance of the obstacle from the device; the shorter the distance, the higher the frequency of the alarm sound produced. the system also checks and detects if there is a flame in front of the person. The alarm sound changes based on that. Furthermore, the system vibrator is activated, as well as to warn of the presence of obstacles or pits, and its speed is also adjusted according to that distance. Light Dependent Resistor sensor (LDR) was added to the proposed device, which controls the operation of the LEDs group at nightfall so that this person can be distinguished as blind by people at night, especially by motorists. The system also contains a water sensor K-0135 to detect the presence of water on the floor and operate the alarm. As mentioned above, we provided the blind person with a social distancing alarm that included an Arduino UNO board and sensors based on an ultrasonic sensor HC-SR04, a motion sensor HC-SR501, LCD with the Hitachi HD44780 driver and a buzzer.
Figure 1. The proposed assistive system
2.2 System design
The proposed system has three separate devices that work together to achieve a desirable requirement, each of them has an Arduino board, sensors, a buzzer, and a vibrator, on which the first part of a smart glove depends. On the Arduino Uno, it is placed on a glove, while the second part, the smart shoe, is based on the Arduino Nano and is placed on a shoe. The third part is a smart hat a social distance alarm based on Arduino Uno, also invented a hat supported by a light-dependent resistor sensor, which operates a group of LEDs at night to tell the car drivers that the person who wears the hat is blind. Where these three devices and the hat together are a complete system.
2.2.1 Smart glove
The first device is in the form of a glove. Figure 2 shows the schematic capture of the glove in which the design is put on the Arduino UNO as the processing unit, and then connects the power from the Arduino to the lines of power on the board. Also connected and placed on the glove are the ultrasound sensor, the flame sensor, and the buzzer.
Figure 2. Schematic capture of the glove
Figure 3. Schematic capture of the shoe
2.2.2 Smart shoe
The second device is in the form of a shoe putting the Arduino Nano on the board and then connecting the power from the Arduino to the line’s power on the board. There are two ultrasonic sensors in this device, one to detect obstacles and the other to detect slopes and pits, first ultrasound sensor and the second ultrasound sensor are placed with different orientations on the shoe model also water sensor is connected to Arduino. The schematic is shown in Figure 3.
2.2.3 Smart hat
The third device is where it is formed as a hat for social distancing purposes. The ultrasonic sensor, motion sensor, buzzer, LEDs, photo resister, and liquid crystal display (LCD) are connected through the Arduino board. The ultrasonic is oriented in the horizontal direction to measure the distance between one person and another. The photoresistor is controlling the operation of the LEDs as a warning sign for the surrounding person at night. Also, the motion sensor is used to detect motion close to the user. The schematic is shown in Figure 4.
Figure 4. Schematic capture of the hat social distance alarm
Figure 5. Smart glove algorithm
2.3 Software implementation
Each of the three parts acts based on a different algorithm to achieve the desired goal.
2.3.1 Smart glove algorithm
The algorithm, as shown in Figure 5, starts by reading the inputs from the two ultrasound sensors and flame sensor and then comparing their values. If the distance between the sensor and the obstacle is within the predetermined ranges and the flame sensor does not detect the flame, the alarm bell will ring at a predetermined frequency. Each range has a different frequency tone, so different sounds are generated, and the blindness distinguishes the different situations, i.e., an increase in the alert frequency to indicate the approaching obstacle. Except for these two conditions, the alarm bell will turn off.
2.3.2 Smart shoe algorithm
The algorithm shown in Figure 6 (a), starts by reading the inputs from the two ultrasound sensors. Each sensor value is compared to a predefined range. first ultrasonic which is responsible for detecting ground obstacles. If the distance between the sensor and the obstacle is within the range, the vibrator vibrates at a certain frequency. The vibration frequency increases as we approach the obstacle in front of it. The second ultrasonic is responsible for detecting slopes. If the distance between the sensor and the floor is within the predefined range, the alarm will trigger to indicate the presence of a pit or a slope. If the distance is less than the threshold, the bell will sound to alert you. Moreover, a water sensor is used to detect water on the floor by playing the alarm bell with a different tone and intermittently according to a certain delay time.
Figure 6. (a) Smart shoe algorithm
2.3.3 Smart hat algorithm
The algorithm shown in Figure 6 (b), is working for the social distance of blindness. At first, ultrasound and motion sensor values are read. If the motion sensor is detecting motion, three messages are shown on the LCD based on the distance measured by ultrasonic. The World Health Organization [22], recommends a safe distance of at least one meter between each other. If the distance is less than two meters, moreover to the message that shows on the LCD, the alarm tune is turned on with different frequencies based on the distance. When a person is detected at a distance of less than 50 cm, the system emits a powerful acoustic on the buzzer and displays the message "stay away." the distance is between 50 cm and 100 cm, a slight acoustic alarm on the buzzer is delivered by the system, and the message says: “Keep away" and when the distance is less than 2 meters, no sound warning on the buzzer is delivered by the system, and the sighted person will read the message saying: “Please keep away”. Optical resistance controls whether the LEDs are on or off.
Figure 6. (b) Smart hat algorithm
To evaluate the effectiveness of the suggested systems, a number of tests and comparisons are made. To emulate a variety of real-world circumstances, three guidance scenarios were chosen. For the shoe test, which was conducted indoors, volunteers moved in a straight line until reaching the obstacles. Nine obstacles are placed at different distances from the starting point (50, 75, 100, 125, 150, 175, 200, 250, and 300 cm) at ground level. Five attempts are made for each obstacle, and the system successfully detected and informed the volunteer on all 45 attempts. All the data are collected and analyzed to determine the system’s ability to detect the real distance from the object, as shown in Figure 7 (a). The system shows high accuracy, close to 99% as shown in Figure 7 (b). high accuracy for the closest obstacle is vital point because it affects the mobility for the user. The accuracy is degradant when the distance exceeded two meters this effect on the mobility of user because it is on safe distance. The two-meter distance is quite enough for the system to avoid the obstacle.
The obstacle of stairs or holes was also tested in the shoe test. Different hole depths were placed on the path of the volunteer (12 different hole depths). The volunteer shows the ability to detect and distinguish between the hole’s depth while using the proposed system, with a different sound frequency corresponding to a specific depth. Five tries are performed for each obstacle, and all 60 tries show the system succeeded. All the data are collected and analyzed to determine the system’s ability to detect the real depth, as shown in Figure 8 (a). The system shows high accuracy, an average close to 95% as shown in Figure 8 (b) high accuracy for the closest obstacle is important because it affects the user ability to avoid a drop-off in the holes or the stairs before reaching the ending point. the accuracy is degradant when the distance increase this is not the effect on the user because the object beyond the foot of the user.
Figure 7. (a) Obstacles distance measurement graph
Figure 7. (b) Obstacles distance measurement accuracy graph
For the glove test, which was conducted indoors, the volunteer walked in a straight line until reaching the ending point. On the volunteer's path, two obstacles, one vertical and one fire-related, were placed. Once the obstacle was detected, the participant had to hear an alarm on the buzzer delivered by the system. The second test was similar to the previous one, and subjects had to avoid the fire obstacle in front of the volunteer. Eight vertical obstacles and six fire obstacles are placed at different distances from the origin point. The test was repeated five times for every obstacle in the all 70 scenarios, and the system detected all the obstacles without any fault. All the data are collected and analyzed to determine the system’s ability to detect the real distance from the obstacles at the vertical level as shown in Figure 9 (a) and the fire obstacle as shown in Figure 10 (a). The system shows high accuracy, with an average close to 97% for vertical obstacles as shown in Figure 9 (b) and 95% accuracy for fire obstacles as shown in Figure 10 (b) which is enough for the volunteer to distinguish obstacles.
Figure 8. (a) Downstair hole distance measurement graph
Figure 8. (b) Downstair and hole distance measurement accuracy graph
Figure 9. (a) Vertical obstacle distance measurement
Figure 9. (b) Vertical obstacle measurement accuracy graph
For the hat test, ask the volunteer to walk into a lighted room then the lights are turned off which makes the LEDs are turned on and vice versa. This light makes the blind person more noticeable to cars, motors, etc. at night in the street and protects him/her from making an accident. for social distance tests. The volunteer was asked to walk in a closed room among other people, Once the person was detected at a distance between the three predetermined ranges, the system showed the message and generated sound based on the range. The system successfully achieves the desired performance. The proposed system prototype is shown in Figure 11. The previous tests validated the ability of the proposed system to achieve the predetermine results.
Figure 10. (a) Fire obstacle distance measurement
Figure 10. (b) Fire obstacle distance measurement accuracy graph
Figure 11. (a) Proposed system prototype: smart glove
Figure 11. (b) Proposed system prototype: smart shoe
Figure 11. (c) Proposed system: smart hat
Also, the proposed system is compared with similar assistive systems. In which the system shows features that are not found in the other systems as shown in the Table 1. Features: A: Obstacle collision avoidance, B: Flame collision avoidance, C: Water collision avoidance, D: Holes collision avoidance, E: Day and night detection F: Social distancing alarm.
Table 1. A comparative analysis of the proposed approach with existing systems for VI
|
Authors |
Year |
Features |
|||||
|
|
|
A |
B |
C |
D |
E |
F |
|
Ibtisam and Annapurna [9] |
2021 |
√ |
x |
x |
x |
x |
x |
|
Elsonbaty [18] |
2021 |
√ |
x |
√ |
√ |
x |
x |
|
Masud et al. [12] |
2022 |
√ |
x |
x |
x |
x |
x |
|
See et al. [20] |
2022 |
√ |
x |
x |
x |
x |
x |
|
Proposed Work |
2022 |
√ |
√ |
√ |
√ |
√ |
√ |
The proposed system developed a multifunctional assistive system to address the challenges faced by visually impaired individuals in navigation, obstacle detection, and social interaction. It provides real-time assistance to visually impaired individuals for barriers that are four meters in front of the user as well as for the detection of stairs, potholes, fire sources, flame, water, day and night visibility enhancement, and social distancing assistance. Extensive testing and evaluation were conducted to assess the system's performance, accuracy, and reliability. The research demonstrated how the suggested method might considerably increase the safety and freedom of those who are blind or visually impaired. The assistive system's overall efficacy and usability were improved by the unification of several features into a single system.
Despite the suggested system's encouraging outcomes, there are still certain limitations and difficulties that must be overcome. One limitation is the requirement for further system optimization and refinement to improve accuracy and dependability in complex and dynamic environment. To maintain robust operation, it could be necessary to make significant enhancements to the system's performance in open spaces and congested regions. Another challenge is the integration of GPS functionality to provide location awareness, it would improve the system's usefulness and independence for those who are blind or visually challenged. The system's capabilities might further be increased by including image processing techniques for word recognition and item identification. In order to make sure that the system satisfies the unique requirements and preferences of people who are visually impaired, user input and involvement are also necessary to create iterative system modifications based on user feedback.
The proposed system has various benefits over existing systems. By including characteristics like obstacle detection, flame detection, water detection, hole detection, and social distancing help, it outperforms traditional assisting devices like guiding dogs and canes. This all-encompassing approach improves the safety, freedom, and mobility of people who are blind. Additionally, the system's affordable and transportable architecture makes it available to a wider user base. The system's affordability and usefulness may be increased by implementing it at a reduced cost by employing widely accessible parts and technology. The suggested system's effect comes from its ability to enhance the quality of life for those who are blind or visually impaired by enabling them to more confidently and independently navigate their environment. By addressing the demands for social distance during the COVID-19 epidemic, it encourages social inclusion.
The suggested multifunctional assistive technology has the potential to considerably improve the quality of life for those who are visually impaired. Although there are still certain restrictions and difficulties to be resolved, the system's contributions, significant findings, and gained experience demonstrate how important it is to enhancing the safety, independence, and general wellbeing of the community of visually impaired people.
[1] World Health Organization site, October 2022. https://www.who.int/news-room/fact-sheets/detail/blindness-and-visual-impairment, accessed on 2 Jan. 2023.
[2] Maragatharajan, M., Jegadeeshwaran, G., Askash, R., Aniruth, K., Sarath, A. (2019). Obstacle detector for blind peoples. International Journal of Engineering and Advanced Technology, 9(1S4): 61-64.
[3] Tavde, K., Khambhati, V. (2021). Smart shoes for blind person. Compliance Engineering Journal, 12(7): 68-71.
[4] Sakthi, S.V., Revanth, G., Sadam, H.M., Manoj, D.R. (2021). Third eye for blind using ultrasonic sensor. International Research Journal of Engineering and Technology, 8(5): 2719-2722.
[5] Batool, A., Naz, S. (2021). Third eye for blind. Asian Journal of Convergence in Technology, 7(2): 1-4. https://doi.org/10.33130/AJCT.2021v07i02.001
[6] Argüello Prada, E.J., Santacruz Forero, L.M. (2022). A belt-like assistive device for visually impaired people: Toward a more collaborative approach. Cogent Engineering, 9(1): 2048440. https://doi.org/10.1080/23311916.2022.2048440
[7] Yogesh, S. (2022). A comprehensive survey on smart stick for visually impaired people. International Research Journal of Modernization in Engineering Technology and Science, 4(2): 39-44.
[8] Shamila, E.S., Varun, S., Soundhur, G., Rupali, S.J., Vishnu, V.U. (2022). A smart graphic eye for visually impaired with text-to-speech converter. Journal of Emerging Technologies and Innovative Research, 9(6): 23-28. https://www.jetir.org/papers/JETIR2206503.pdf.
[9] Ibtisam, Annapurna, V.K. (2021). Third eye for the blind: An innovative wearable technology using NI MYRIO. Journal of Emerging Technologies and Innovative Research, 8(8): 775-780. https://www.jetir.org/papers/JETIR2108466.pdf.
[10] Singh, R., Kamat, R.K., Chisti, F. (2021). Third eye for blind person. International Journal of Innovative Research in Technology, 8(1): 1115-1120.
[11] Albogamy, F., Alotaibi, T., Alhawdan, G. (2021). SRAVIP: Smart robot assistant for visually impaired persons. International Journal of Advanced Computer Science and Applications, 12(7): 345-352. http://dx.doi.org/10.14569/IJACSA.2021.0120739
[12] Masud, U., Saeed, T., Malaikah, H.M., Islam, F.U., Abbas, G. (2022). Smart assistive system for visually impaired people obstruction avoidance through object detection and classification. IEEE Access, 10: 13428-13441. https://doi.org/10.1109/ACCESS.2022.3146320
[13] Devi, P. (2022). Computer vision based assistance system with voice feedback for the visually impaired people. Master thesis.
[14] Kamaruddin, F.S., Mahmood, N.H., Razak, M.A., Zakaria, N.A. (2021). Smart assistive shoes with internet of things implementation for visually impaired people. Journal of Physics: Conference Series, 2107(1): 012030. https://doi.org/10.1088/1742-6596/2107/1/012030
[15] Hegde, P., Devathraj, N., SK, S., Aishwarya, P. (2021). Smart glasses for visually disabled person. International Journal of Research in Engineering and Science, 9(7): 62-68.
[16] Neugebauer, A., Rifai, K., Getzlaff, M., Wahl, S. (2020). Navigation aid for blind persons by visual-to-auditory sensory substitution: A pilot study. PLoS One, 15(8): e0237344. https://doi.org/10.1371/journal.pone.0237344
[17] Paré, S., Bleau, M., Djerourou, I., Malotaux, V., Kupers, R., Ptito, M. (2021). Spatial navigation with horizontally spatialized sounds in early and late blind individuals. PLoS One, 16(2): e0247448. https://doi.org/10.1371/journal.pone.0247448
[18] Elsonbaty, A.A. (2021). Smart blind stick design and implementation. International Journal of Engineering and Advanced Technology (IJEAT), 10(5): 17-20. https://doi.org/10.35940/ijeat.D2535.0610521
[19] Tachiquin, R., Velázquez, R., Del-Valle-Soto, C., Gutiérrez, C.A., Carrasco, M., De Fazio, R., Trujillo-León A., Visconti, P., Vidal-Verdú, F. (2021). Wearable urban mobility assistive device for visually impaired pedestrians using a smartphone and a tactile-foot interface. Sensors, 21(16): 5274. https://doi.org/10.3390/s21165274
[20] See, A.R., Sasing, B.G., Advincula, W.D. (2022). A smartphone-based mobility assistant using depth imaging for visually impaired and blind. Applied Sciences, 12(6): 2802. https://doi.org/10.3390/app12062802
[21] Toro, A.A.D., Bastidas, S.E.C., Bravo, E.F.C. (2020). Methodology to build a wearable system for assisting blind people in purposeful navigation. In 2020 3rd International Conference on Information and Computer Technologies (ICICT), San Jose, CA, USA, pp. 205-212. https://doi.org/10.1109/ICICT50521.2020.00039
[22] World Health Organization. (2022). https://www.who.int/westernpacific/emergencies/covid-19/information /physical-distancing, accessed on 12 Dec. 2022.
