C. Favey | A. Zogaghi | R. Leroux | B. Vivan | R. Farcy
OPEN ACCESS
Presentation and analysis of the first experimentation of autonomous navigation, for a young man suffering from category 4 blindness, associated with disorders of the sensitivity of the limbs, moving thanks to the feelings of the accelerations and decelerations of an anti-collision and anti-fall electric wheelchair.
anti-collision, anti-fall, powered wheelchair, electric wheelchair, navigation, autonomous
For several decades, the number of electric wheelchair users has been steadily increasing [1, 2]. Whether it is end-of-life issues, congenital or acquired pathology, some users of electric wheelchairs are likely to make driving errors due to their disability. Most often it is a lack of control, muscular coordination disorders or attentional disorders, that lead to errors which can result in collisions with their environment (surrounding people, infrastructures or various equipment), falls in height differences (sidewalks, stairs, slopes, etc.), or others. All of which are likely to cause serious human and material damage.
For these reasons, some people with disabilities are excluded from access to an autonomous mobility solution, due to safety purpose. Others see their use limited (speed and acceleration restricted) or others accept the risks and use it with or without supervision.
However, the medical community is unanimous on the need of access to autonomous travel, for the good intellectual, personal, social and cultural development of all people, whether they are disabled or not [3, 4].
This is why we are working since 5 years on the design of an optoelectronic system, which makes it possible to equip any electric wheelchair, in order to secure its movements in all conditions (indoors and outdoors, full sun, night, fog, rain, etc.), to prevent any falls or tipping over in a drop, as well as collisions with nearby obstacles. In order to give access to mobility to those whose disability deprives them, and to allow others to move with more serenity, using less cognitive load.
For the past few weeks, we have been working with a young man in high school integration, presenting almost complete blindness (category IV) coupled with a deterioration of the nervous system, depriving him of sensitivity of the limbs. This latter performs the majority of his trips in a conventional electric wheelchair, piloted by a third person (mainly by one of his school life assistants).
The general goal of this project is to allow him to gain autonomy on a daily basis, by starting with environments or specific routes known and mastered initially, then by gradually expanding, depending on the results and our technological advances.
In this paper it will be presented, the system employed as well as the philosophy governing its development. We will detail the first use of said system by the young student, in an open and large outdoor environment to begin. Then we will analyze his appropriation of the system within his high school, initially empty, then filled with other students. Finally we will discuss the prospects for development in its specific case and the system in general.
2.1 Development’s philosophy
From the inception of the project, the main goal has been to provide maximum autonomy to each user, while keeping them safe. Many past and current projects have pursued and pursue the same goal [5, 6]. However, for now, no system has succeeded yet in providing enough guarantees to convince the health community, whether in France or abroad, of its relevance. As often, this is not explainable by a single reason, but rather, in our opinion, by a set of more or less correlated elements. The targeted population, severely disabled, is partly under guardianship and mostly placed in specialized centers. Their security is currently ensured by limiting their access to the electric wheelchair. So, we must therefore (as well as all projects similar to ours) present enough evidence to demonstrate that the system provides the necessary security for the user and provides him with significant benefits in his daily life. Regarding safety, for all so-called “smart wheelchair” projects, this is ensured by a combination of sensors. But no sensor is perfect. No matter what its price, size, or how much energy it consumes, there is always a situation where it fails. In addition, the generalist compromises governing commercial sensors, in order to make them suitable for the greatest number of uses possible, do not allow good robustness of use. Safety cannot therefore be formally guaranteed. However, there is no such thing as a 100% safe life, including for able-bodied people, if we consider all pedestrian or automobile accidents, during their travels. So what minimum level of security should we provide to the most vulnerable to allow them to regain a certain autonomy? Looking at the current situation, it seems that no one has found the answer to this question yet.
This is the reason why our research team took the initiative to rely on its own sensors, developed in the same philosophy as those for the navigation of people with visual disabilities [7, 8].
Moreover, the entire system has been designed since its beginning with the help of many medical professionals (re-education doctors, occupational therapists, psychomotor therapists, etc.). Directing us towards the choice of a secure movement rather than an autonomous or semi-autonomous system. The main idea is that the system intervenes only in the event of danger and is limited to slowing down or stopping the wheelchair in the direction considered dangerous. No navigation assistance, no automatic chair reorientation or other similar actions. The user remains completely free of his choices and the direction in which he wishes to go. Only the speed is reduced in certain situations. This is a priori the most favorable context for good intellectual, personal, social and cultural development, mentioned above.
2.2 Sensors used and disposition
In the version used for our first meetings with the student, the front part and part of the sides of the wheelchair were protected according to Figure 1. The rest of the contour was simply not yet developed but should not show any major modifications. As the chair's navigation is done primarily from the front, like a car, this did not pose any particular problem.
The following are used on each side of the chair: 2 infrared proximeters, a three-beam laser triangulation range finder and 2 ultrasonic sensors.
Infrared proximeters have the particularity of being immune to the sun, and perform the large majority of obstacle detection. However, they are not very effective with thin and dark obstacles, which is why we have added commercial ultrasonic sensors with a range limited to 1.5 meters, which provide some detections, which were so far a little too late, by the infrared proximeters. It should be noted that considering all the shortcomings of the ultrasonic sensors, they will soon be replaced by another sensor under development. Tri-laser triangulation heads, also immune to sunlight, allow the detection of positive as well as negative height differences.
Their disposition is as follows:
Illustrations:
Figure 1. Anti-collision sensors disposition
Figure 2. Anti-fall sensors disposition
It is important to keep in mind for what follows, that depending on the nature of the detected danger, namely an obstacle or a height drop, the system will not behave the same way.
Indeed, in the case of an obstacle, the latter will slow down proportionally in its direction, depending on the distance separating it, (the closer it gets, the more it slows down), until it reaches a non-zero minimum speed. At this speed, the contacts do not cause damage and the wheelchair no longer has enough power to be able to crush/push the obstacles/surrounding people. It is essential to keep the possibility of movement, even very slow, in the case of a narrow environment, indoors for example, for access to the toilets, to the elevator, to the dining table etc.
Regarding the detection of height differences, the wheelchair is completely stopped in the direction considered dangerous. In fact, there is no question of maintaining a non-zero minimum speed, because whether you go very slowly or very quickly up a staircase, the result differs little...! Directions moving the wheelchair away from the vertical drop will always be authorized, it will be up to the user to find a path accepted by the system.
3.1 Assessment of the user's capacities
A crucial first step was to assess whether the user was able to feel the acceleration and deceleration of the chair, in order to orient himself. In fact, conventionally, the piloting of an electric wheelchair is essentially based on 2 senses: sight and proprioception. The sight to achieve a permanent control of the direction of the chair. Proprioception to precisely orient the wrist, on the joystick, in the desired direction. However, here, our user does not have the first and very partially the second. Thus, his orientation in space and the perception of his movements can only be based on his hearing and his feeling of acceleration/deceleration. The use of hearing for spatialization is relatively common among visually impaired people, however this is usually supplemented by proprioception through the white cane and footsteps on the ground. Which unfortunately cannot be the case here. From the feeling of accelerations/decelerations then emerges a somewhat experimental orientation system which seems promising to us. It is possible to imagine a parallel between the movement of a blind person in a cane and a blind person using our system. The role of the white stick is to detect obstacles and height differences that the user feels through proprioception. In our case, the system detects obstacles and height differences, slows down or stops, transmitting information to the user via deceleration.
This is why it is necessary that the person using our system is well able to experience these positive and negative accelerations.
The first test therefore consisted of placing the user in the wheelchair, which we piloted as a third person, then that the user describes to us, according to him, in which direction the wheelchair was moving, if it moved forward, backward, turned, on which side, if the speed varied, etc. This first test was conclusive, the feelings expressed being satisfactory and in accordance with reality.
So we have moved on to the second phase of testing. This time the user was in control of the chair, the protection system deactivated. This test took place outdoors, in a very open place, without obstacles. The goal was to assess his ability to orientate himself by hearing. The user had to head towards the voice of one of us was emitting. After the easy success of this exercise, we made it more complex, asking him to make a circle around the voice he heard and come back to his point of origin. This also went quite satisfactorily. We therefore decided to continue our progress with the protection system activated, in more complex environments.
So, with the system activated, we repeated this experience of voice guidance, with the instruction to immediately release the control as soon as the onset of deceleration was felt. We placed ourselves as human barriers between the user and the voice he was moving towards. Here again, it was a success, always releasing control well in time. It should be noted that his security as well as ours were ensured by the system, this exercise was therefore without danger, however it was important for us to observe his good appropriation of the decelerations.
We then continued to walk outside, he in a wheelchair, us by his side, guiding him either to the sound of our footsteps, or by using the time system orally.
These first fruitful tests prompted us to go to his high school, which will a priori be the first place in which he will be required to use his wheelchair equipped with our system, autonomously.
3.2 Empty high school
The discovery of the system for the user took place outdoors, in an environment with very few sensory cues, so that he mainly focuses on understanding the system rather than the environment. Now we are in a place known to him, his school, in which he already has several landmarks. These landmarks are mainly based on his hearing and his perception of rotations. Hearing allows many different information to be obtained. First of all, each place has its own sound identity, depending on its echo, its size, its layout, the materials that compose it, etc. So with a little experience, it is possible for the user to identify the building, the room in which he is located, if he has already memorized it before. In addition, what is called echolocation, sometimes developed naturally by congenital visually impaired people, and with work by people who have acquired their blindness, makes it possible to feel the surrounding volumes and objects, with more or less precision, thanks to the rebounds of the air, and therefore of the sound, on any wall. This allows, thanks to the perception of the presence of walls, for example to be in the middle of a passage, to be aware of the orientation of his body within a room, etc. This echolocation complements the human internal gyrometer. This gyrometer is not strictly speaking a sense, but a combination of senses. The composition of which varies according to the capacities and handicaps of each one. For visually impaired people, it is mainly based on muscle proprioception and the inner ear, which allow them to assess how the body has turned in space. Which is essential when walking independently. Since this gyrometer is relative, it is regularly updated thanks to echolocation and the presence of significant volumes which allow the person to perceive whether their body is in the axis assessed by their internal gyrometer.
Here, the young student has very little proprioception capacity due to his handicap. Thus his internal gyrometer, when he is in the chair, is mainly based on his feeling of lateral accelerations and is updated each time a place is recognized with the possibility of knowing the orientation of his body, by ear.
Very quickly, he was able to put in place a valid operating strategy. It is interesting to note that this speed of adaptation can probably be explained by the accumulation of sound cues and recognition of chair rotations, which he has been able to assimilate since the beginning of the year, during his travels in his school with a third person piloting his chair.
This strategy looked like this: he walked through the long and wide corridors of his school using his hearing and internal gyrometer (previously introduced). This method not being perfect, his trajectory could deviate from the followed axis, in a sufficiently subtle way that he did not realize it, and then, the wheelchair found itself at the edge of the corridor, that is to say along storage lockers, either along benches on the sides or just along walls, depending on the location. In fact, the chair would gradually slow down in that direction and the student then had to “scan” with the joystick in other directions than the one in which he was heading, to find one in which the speed of the chair was again unrestricted. The choice of the direction of “scanning” was made in accordance with the sensory cues he had been able to set up previously. Thus, in the vast majority of cases, it would naturally reorient itself towards the center of the corridor on the first attempt, (although this was not the case all the time).
Illustration:
Figure 3. Wall detection and trajectory restoration
Another interesting case was that of stone columns, which dotted the building, arranged right in the middle of the corridors. Of course, some got in the student’s path more than once. The latter having well in mind the existence of these columns, when the system slowed down the wheelchair, it was then a question of knowing whether this was the case because of these columns or because of a deviation towards one of the sides that he wouldn't have felt, having dragged him against a wall or bench. The determination of this was done once again according to his present landmarks and those stored during his course.
Illustration:
Figure 4. Column detection, possible trajectories
3.3 Hallway full of students
After having made several trips back and forth in the empty school, the other students being in class, it came time for noon and therefore for everyone to leave his rooms. We then experience the navigation in real conditions.
We were able to identify three different general types of behavior in other students. Either they became aware of the presence of our wheelchair user and bypassed him and shifted from his path. Either they were too focused on their conversation or their phone to notice anything and then went straight to the chair. Either they were aware of the student in the wheelchair, but having the habit of crossing him daily, usually led by his school assistant, did not pay particular attention to his trajectory, thinking as usual, that the third person would adapt the pace and trajectory of the chair. In addition, it seemed to us that all of these attitudes were accentuated by the fact that eyes are generally turned away from a person in a wheelchair, out of modesty or a desire not to stigmatize the person with a disability. Thus, the test was far from easy for the young user. However, it was a real success for the latter, who from his first attempt was able to fully travel two long corridors filled with other students, slaloming between them, the central columns and the walls. It would seem that the changes in the atmosphere and the surrounding sounds had shifted the student to another set of cues. As the meaning of the masses becomes very difficult to exploit, it was then necessary to rely on more specific elements: students' voices, footsteps, the sound of lockers opening/closing, the sound of doors borrowed by students leaving, felt drafts, etc. What this experience has allowed us to highlight is that there is not an absolute way to find one's way, but rather a variation and permanent adaptation of the senses and meta-senses of any person. This confirmed us as to the feasibility of our objective of equipping this student, in order to give him back his autonomy in several places of his daily life. However this will require several evolutions.
Indeed, although all the obstacles had been correctly detected during our tests, we nevertheless noted some situations which could be problematic afterwards, in the event that the user becomes disoriented and finds himself in a place not longed for. Typically these are mostly overhead obstacles with no detectable ground attachments. The user, in the current version of the system, could head at full speed towards these types of obstacles and cause serious damage to himself. The most illustrious obstacle in this situation is a staircase starting from the ground floor of the school, going up to the first floor, in the middle of a hallway. When moving through this corridor, we have 3 choices: go around the stairs on the left, take the stairs or go around the stairs on the right. However, after having bypassed it, it is possible to go under the slope of the stairs and therefore in some places to reach above at the level of the head or the trunk.
Illustration:
Figure 5. Situation of an obstacle at head height not detected
In order to overcome this type of problem, we are currently developing a new version of our system, part of which will now be oriented upwards and only dedicated to detecting this type of obstacle. It is interesting to note that the thing is not as obvious as it could appear, because in this kind of situations, with the inertia of the wheelchair, the few centimeters covered during the braking phase, could be enough to get out of the obstacle detection zone and thus allow the wheelchair to accelerate again. This effect is accentuated by the angle with which the wheelchair moves towards the obstacle, but also by the natural angle that the obstacle has regarding the ground, making its detection more complex. For information, using ultrasound, passed an angle of 35 degrees on a smooth surface, the sensor no longer receives enough signal to detect anything. For infrared the problem is similar even if it is smaller. So it would seem that we will lead ourselves towards a multibeam laser triangulation solution in order to overcome all these problems.
[1] https://www.alliedmarketresearch.com/electric-wheelchair-market.
[2] https://www.bloomberg.com/press-releases/2019-05-29/global-electric-wheelchair-market-to-surpass-us-2-9-billion-by-2026-coherent-market-insights.
[3] Tefft, D., Guerette, P., Furumasu, J. (1999). Cognitive predictors of young children's readiness for powered mobility. Developmental Medicine & child neurology, 41(10): 665-670. https://doi.org/10.1111/j.1469-8749.1999.tb00520.x
[4] Trefler, E., Fitzgerald, S.G., Hobson, D.A., Bursick, T., Joseph, R. (2004). Outcomes of wheelchair systems intervention with residents of long-term care facilities. Assistive Technology, 16(1): 18-27. https://doi.org/10.1080/10400435.2004.10132071
[5] Babel, M., Pasteau, F., Guégan, S., Gallien, P., Nicolas, B., Fraudet, B. (2015). HandiViz project: clinical validation of a driving assistance for electrical wheelchair. In 2015 IEEE International Workshop on Advanced Robotics and its Social Impacts (ARSO), pp. 1-6. https://doi.org/10.1109/ARSO.2015.7428213
[6] Simpson, R., LoPresti, E., Hayashi, S., Nourbakhsh, I., Miller, D. (2004). The smart wheelchair component system. Journal of Rehabilitation Research & Development, 41. https://doi.org/10.1682/JRRD.2003.03.0032
[7] Farcy, R., Leroux, R., Jucha, A., Damaschini, R., Grégoire, C., Zogaghi, A. (2006). Electronic travel aids and electronic orientation aids for blind people: Technical, rehabilitation and everyday life points of view. In Conference & Workshop on Assistive Technologies for People with Vision & Hearing Impairments Technology for Inclusion.
[8] Villanueva, J. (2011). Contribution a la télémetrie optique active pour l'aide aux déplacements des non-voyants. Doctoral dissertation, Paris 11.
