AR Support System for Therapy in 3 to 8-Year-Old Children with Altered Fine Motor Skills

AR Support System for Therapy in 3 to 8-Year-Old Children with Altered Fine Motor Skills

Andrés Ovidio Restrepo Octavio José Salcedo Parra Norbey Danilo Muñoz Cañón

Facultad de Ingeniería, Universidad Distrital Francisco José de Caldas, Bogotá 110231, Colombia

Facultad de Ingeniería, Universidad Nacional de Colombia, Bogotá 111321, Colombia

Corresponding Author Email:
27 May 2020
5 August 2020
20 September 2020
| Citation



In this project, an augmented reality (AR) system was designed and developed to contribute to therapy in children from ages 3 to 8 who suffer from fine motor skills disorder or seek to improve their skills during their neurodevelopment. The system was designed in the Unity engine along with a Leap Motion sensor so that the kids' hands can interact with scenarios created in the platform. The construction of the system followed the respective phases that describe the cascade methodology for the development of software systems, including the study of requirements and use cases. Additionally, it is intended that the progress of the child is tracked when he/she is performing a set of activities to develop fine motor skills. The Feine Motonik "FeMo" built module, has an 88.9% usability according to the study carried out in this article, which contemplates the test phase with the objective users who describe the situation of disorders in fine motor skills.


augmented reality, fine motor skills, motion controller, neurodevelopment, unity engine

1. Introduction

The neurodevelopment is known as the continuous process of acquiring abilities, knowledge, capabilities, attitudes and functions that present themselves throughout the course of life. Hence, neurodevelopment has a basic role in the life of human beings. The characteristics of normal neurologic development can be reflected on different areas such as: fine and gross motor skills, language, sensorial system and socialization. This is also accompanied by a description of the main changes in development which can be identified in the daily clinical work of pediatricians [1]. In some occasions, an alteration in fine motor skills can be detected meaning that the child has problems to perform fine movements such as cutting a sheet of paper or moving certain small elements. Traditional medicine treats this disorder through therapy by performing exercises to address this shortcoming.

Currently, technology plays an important role within society, so links are established between different areas through technology. In this case, augmented reality (AR) is the focus since it allows the creation of virtual scenarios with 3D graphic design and adding sensors that lead to constant interaction between the user and the scenario. Furthermore, the implementation can be achieved in multiple ways. In certain cases, previous studies have shown the implementation of a keyboard to help locate the fingers of the kid, so he can partake in activities focused on the development of fine motor skills [2].

 It is no secret that technological breakthrough goes hand in hand with medicine and this is no exception since it is intended to create a support system for therapy of children with fine motor skills disorders. On one hand, the baseline lies in the development of haptic systems that have been previously used to seek a constant interaction between the child’s movements and the system [3]. At this point, technology comes into action with tools such as Unity to create scenarios with various 3D elements and their constant interaction with a Leap Motion device that will serve as the sensor for the child’s hands. Then the patient can interact with several elements that can be found in the scenarios shown by the immersive system.

On the other hand, a test is generated to measure fine motor skills. Some metrics are based on scenarios where the child must manipulate objects with different shapes such as feathers or cap containers [4] to assess an estimation of the motor skill. Additionally, some studies related to fine motor skills and virtual reality have shown improvements in the errors during certain activities. This aims at determining the level of accuracy achieved with virtual reality [5]. The immersive system is thus necessary to work on the coordination of muscles, bones and nerves required to per-form fine movements.

2. Related Works

Disorders in neurodevelopment related to fine motor skills is a field of Pediatrics where technological advances can generate a positive impact through support and involvement without ignoring the job of professionals. In this field, the early detection of disorders is important for the neurodevelopment of children through a set of techniques that can determine the course of action that suits the child [6]. Plus, normal characteristics and alarm signs in children must be considered as the maturation of the nervous system’s structure is expected to detect problems in any aspect of neurodevelopment. Early signs of alarm must be identified with parameters tied to relevant alterations in neurological development described as events that the child must be able to perform during his first years of life [1].

Nowadays, therapy for children with fine motor skills disorders consists on the supervised execution of different exercises or activities that stimulate the nervous system. These exercises focus on the development of abilities in fingers, manual tasks and hand-eye coordination [7]. This traditional therapy is not the only strategy since there was a study performed by Chinese researchers to implement acupuncture for therapy in children for fine motor functions. The study revealed that using stimuli-based acupuncture combined with another technique can improve the motor skills of children [8]. These are not the only alternatives available because traditional therapy can be combined into everyday life (school or home environment) showing benefits on an emotional and attitudinal level [9].

To begin treatment or therapy, the kid’s initial condition must be assessed with a test called DCQD’07, a survey for the diagnosis of coordination development disorder. In this test, the child must perform a set of activities and the evaluator will take notes on the level of performance in a pre-established scale [10]. This test is originally in English, but the “Universidad Autónoma de Manizales” translated it into Spanish and was endorsed by the corresponding American institution. Predetermined therapies are included as well as music to help the development of fine motor skills focusing on learning how to play the piano, an activity that involves both the stimulation of the brain and hand-eye coordination [11].

Traditional therapy is mostly used where bi-manual coordination is involved making use of the nerves and pertinent muscles to develop the skills related to the child’s fine motor skills. The activities are ludic and have a group of shapes or figures with which the child will interact [12]. Furthermore, in Colombia therapy is important at an early age since there is a set of studies where the problem was tackled opportunely, and significant advances were made in motor skills [13].

The 3D techniques referring to augmented reality can be thoroughly analyzed to show that they can be applied to an endless number of fields including education, medicine and mobility. For instance, on the education aspect, the use of AR seeks to improve the learning experience by teaching different topics and creating empathy between the teacher and the students. As a result, these techniques are effectively used overseas [14].

The importance of the internet of things cannot be dismissed since it has led to the continuous interaction between devices that enter the IoT and AR techniques. With these two popular technologies as far as Science goes, low-cost smartphones can be used with a combination of IoT and AR to create artificial interactive environments which is an innovative concept [15]. In the AR field, the ceiling in terms of research and innovation has not been grasped and its potential is notorious. There are many benefits of implementing AR techniques in diverse applications and it could mean a giant leap for the civilization. Combining medicine and AR is also one of the research goals looking ahead [16].

AR is being implemented in rehabilitation with the purpose of improving the functional capabilities of a patient such as recovering motor functions for everyday activities. This is an example of the impact that AR and 3D immersion techniques have on the development of motor skills [17]. Videogames are another source of support and have been applied in treatments with physiotherapy resulting in the improvement of the psychomotor development. A Wii console was used to help the patient point at certain spots for the development of motor skills [18].

Fine motor skills are directed to fine movements of the hands in most cases. This is where AR plays a crucial role through certain devices combined with AR techniques there is an effective method to assess and measure the movement of the child’s fingers during therapy [19]. One of the devices is the Leap Motion Controller for the capture of the gestural movements of children in therapy [20]. It can be evidenced that AR can be effectively implemented in therapy for the development of motor skills to obtain notorious results and generate support for traditional therapy.

3. Methodology

3.1 Description of the current situation

3.1.1 Identification of the existing system

In this phase, the current landscape of therapy in children is identified and de-scribed. The children have disorders in fine motor skills. The methodology and techniques used are stated as well as the effect of immersive technologies in the field of medicine, specifically in motor therapy.

3.1.2 Analysis of the existing system

Based on the identification and description of the existing system, current therapies for this disorder must be analyzed so the elements and attributes of the system must be internalized as well as the relation between them. For a systemic approach, the existing system must be broken down and represented through immersive technologies.

3.2 Development process of the proposal

The cascade methodology will be used to develop the Feine Motonik (FeMo) platform after describing the current situation of therapy in children with disorders in motor skills.

3.2.1 Analysis

In this phase, the functionalities of the FeMo platform to build must be defined. Hence, the actions of each module are defined within the platform. The real needs are presented based on the requirements considered in the analysis of the existing system. Additionally, the needs or functions of the platform must be precisely deter-mined.

3.2.2 Design

According to the analysis stage, the characteristics of the Feine Motonik (FeMo) platform are represented to pave way for the implementation. This takes into consideration the architectonic design and the functions to be performed by the FeMo.

3.2.3 Implementation

After the design has been established and the problem is clearly stated, the tools to be used are determined. In this case, the cascade methodology is chosen. For the development engine, Unity is selected. The SDK of the Leap Motion Controller is chosen as the gestural control device and the programming language is Scripting.

3.2.4 Usability tests

With the complete implementation, tests will be carried out to assess the visual impact and manipulation of the FeMo module. The interaction between the user and the system will be seen on usability tests. For the purposes of testing, six children constitute the sample population. They will be exposed to the module and a survey will assess the impact on the public.

3.3 Closing of the project

3.3.1 Analysis and discussion

According to the previous phase, usability tests must be performed on a set of data or aspects supplied by the user to perform the necessary operations for analysis. The work is compared with other authors regarding the idea of immersive systems with AR techniques as a support for conventional motor therapy.

4. Implementation

To develop this platform, the Cascade methodology will be used on every phase: System Analysis, Design, Coding, Testing and Maintenance, as shown in Figure 1. By using the Cascade method as framework, a view of sequential steps is offered. Each stage has well de-fined goals and activities that are the means to accomplish the outlined objectives.

Figure 1. Cascade framework [19]

4.1 System analysis

In this phase, the end user’s needs are analyzed to determine the objectives to be covered, i.e. the definition of what the system must do without going into details. Details will be discussed in the design phase. To develop this stage, a description of the current situation must be made. The state of current therapies for children with disorders in fine motor skills is described to help determine the functionalities of that the system should have.

4.1.1 Description of the current situation

Nowadays, therapy for children with alterations in neurodevelopment related to fine motor skills is focused on the manipulation with the hands of differently-shaped objects. This is meant to stimulate the nerves and muscles of the hands and develop skills in that area. The term manipulate refers to the actions of grabbing, throwing, holding, piling and moving objects with different shapes. This approach stimulates the nerves and muscles related to fine movement. Therapies are carried out by trained staff to offer guidance to the children.

4.1.2 Functionalities of the system

In this section, the functionalities that the platform must be capable of doing are outlined.

  • Interaction with the objects of the environment: The platform must be able to allow the interaction between the child and the objects that are available in the module. For the exercises, a gesture control device (Leap Motion Controller) will enable the interaction. Interacting with objects means grabbing throwing, stacking and holding them.
  • Interaction between objects: Each object must have physical properties since the experience should be realistic. Therefore, physical attributes and laws will be be-stowed upon the objects to create a pragmatic scenario. In this case, the development of the FeMo module will include objects such as squares, spheres and boxes along with the hand module.
  • Friendly and appealing system: The targeted audience is constituted by children, so the platform should be appealing and kid-friendly. There must be some expectations and it must be easy on the eyes.
5. Design

In Figure 2 and Table 1 the use case number one, interacting with the environment displayed by the module, is shown.

Table 1. Requirement: Interacting with the environment




Interacting with the environment from each module


The user will be able to interact with the environment in each module to present an immersive environment. This focuses on interacting with the elements shown.


(1). The user must have installed all the components required by the FeMo.

(2). The user must have executed FeMo.

(3). The interaction through Leap Motion Controller is ready.

(4). A module must be initiated.





(1). The user is inside the module.

(2). The module includes a set of elements destined for therapy.

(3). The user can interact with the elements available in the module.


If the interaction with the elements is not possible because the hands are not visible in the environment, check the connection and the state of the Leap Motion Controller.


The application does not work due to installation issues.


User, Leap Motion Controller

Table 2. Requirement: Manipulate the elements of the module




Manipulating the elements in each module 


The user will be able to manipulate the objects available for the development of therapy. The user can grab, hold, move or release to perform a set of predefined tasks.


(1). The user must have installed all the components required by the FeMo.

(2). The user must have executed FeMo.

(3). The interaction through Leap Motion Controller is ready.

(4). A module must be initiated.





(1). The user is inside the module.

(2). The module includes a set of elements destined for therapy

(3). The user can grab, hold, move and release every object of the module.


If the interaction with the elements is not possible because the hands are not visible in the environment, check the connection and the state of the Leap Motion Controller.


The application does not work due to installation issues.



Table 3. Requirement presenting the control elements




Presenting the control elements of the system


The platform must present the control elements such as the visualization of the instructions for the development of the activities in each module.


(1). The user must have installed all the components required by the FeMo.

(2). The user must have executed FeMo.

(3). The interaction through Leap Motion Controller is ready.

(4). A module must be initiated.





(1). The user is inside the module.

(2). The platform will have spaces to display the control elements.

(3). The platform shows the instruction for the development of the exercise.






User, Leap Motion Controller

Figure 2. Interacting with the environment

In Figure 3 and Table 2 the use case number two, manipulating the elements available in each module, is shown.

In Figure 4 and Table 3 the use case number tree, presenting the control elements of the system, is shown.

5.1 Node diagram

This diagram is used to show the topology used to develop the system taking the hardware as a reference point. This reveals aspects necessary for the understanding of the total system in operation. In the diagram shown on FeMo, the general land-scape can be evidenced where the client uses devices such as PC or laptops. They must have 64-bit Windows 7/8/10 and have installed on them components such as Unity 3D 5.50f3, SDK Leap Motion Orion 3.2.0 and Leap Motion Core Asset Orion 4.1.5.

By having all these components in the device, the interactive (or immersive) system can be used with a Gestural Control Device, Development Language and Environment to perform the activities included in the modules through the synergy achieved in the environment.

In Figure 5 the node diagram is presented in which each core of elements, such as the hardware characteristics and the association to software (interactive system), allow the client to manipulate the module.

Figure 3. Manipulating the elements in the module

Figure 4. Presenting the control elements


Figure 5. FeMo node diagram

6. Construction

In this section, the FeMo module will not described step by step since the objective is not to show how it is developed but the components in the development.

6.1 Development environment

The development environment used to develop FeMo was Unity plus a Gestural Controller which is the Leap Motion Controller. Figure 6 shows the development environment with two different views: the scene which can be modified by the developer and the game view that shows the actual execution of the module. The hand module was imported for user interaction.

6.2 Hand module

Figure 7 shows the hand module in execution and the object will move according to the movements of the user within the capture range of the Leap Motion Controller. The hand module links a set of articulated objects to simulate a human hand so that the experience is realistic during the interaction. This module has green lines that go around the hands which show their collision range.

6.3 Scenario

The established scenario for this FeMo module can be seen in Figure 8. It includes a wooden table for the support of the other objects, spheres and squares with specific colors and two-colored boxes. There is also the hand module, an object where the instruction is indicated and a skybox for the background.

Figure 6. Development environment

Figure 7. Hand module

Figure 8. Scenario

6.4 Starting module

Both Figure 9 and Figure 10 show the simulation of the module. A small checkbox can be seen in the upper right corner where the person that was interacting with FeMo through the Leap Motion Controller. In the execution, the user can be seen grabbing and moving one of the spheres and taking it towards the corresponding box.

Figure 9 shows the interface that indicates to the user the location of the green sphere in the green box. Similarly, in Figure 10, the interface indicates to the user the location of the red sphere in the red box. The two images are in execution time and show two different scenarios for the same action.

Figure 9. Execution for green box

Figure 10. Execution for red box

7. Analysis and Results

7.1 Implementation results

The development of the module was intended as a support mechanism for therapy in children with deficiencies in fine motor skills. The challenge was to involve the gesture control device (Leap Motion Controller) with the scene created in Unity. This leads to interacting with the environment’s objects (cubes and spheres) using the hands. The experience and knowledge acquired are worth the effort during the development of the module.

7.2 Usability tests results

To carry out the usability tests, a group was constituted with six children between the ages of 5 and 8 where the interface of the module was shown to them as well as a simulation of the video of the entire process of interaction with the objects included in the module. After discussing the video, the questions made by the children were answered and then a survey was given to them to assess if the interface is user-friendly, if the communication with the objects was interactive enough and if the tasks were clear. Since our target audience consisted mostly of kids, the questions were modified into a more understandable language and the results were the following.

Question 1 reveals that 100% of the children surveyed answered affirmatively to the first question which sought results related to the magnitude of the interface of the module developed in the FeMo. This had a positive impact on how it was perceived by the public. Hence, it can be stated that the interface is friendly and accomplishes the design functionality that the platform must be user-friendly.

Question 2 shows that 6 out of 6 children answered affirmatively to the second question focused on assessing the impact and level of interaction of the Leap Motion Controller as a gesture control device to interact with the environment. The simulation video allows observing the interaction and was the basis of the children’s arguments when answering this question.

Question 3 is evidenced in Figure 11. This indicates that 66.7% of the children admit that the activity shown in the video was clear, i.e. that they know what the person had to do. For the remaining 33.3% of the children, the task was not clear during the activity. Therefore, this would be a factor that needs improvement in the modules of the FeMo platform. In general, the platform is satisfying 88.9% of the project’s purpose. The results of the survey were positive, and some aspects need to be corrected to achieve 100% functionality.

Figure 11. Clarity of the task

8. Discussion of Results

According to the analysis of results, a platform such as FeMo, that supports conventional therapy for children with altered fine motor skill; there are several factors to consider. One of them is the interface where the user has an immersive experience. The interface was assertive with 88.9% (pondered survey percentages) in the development of the FeMo module.

When the results that focus on the graphic and interactive part of FeMo are clear, the authors agree with the propositions of Viñas-Diz and Sobrido-Prieto [21] where it is stated that the interface to be used in the AR system must be adequate for user experience, having a percentage of more than 80% in terms of usability and friendliness of the platform, to maximize the motor improvements. This strengthens and increases the participation and motivation of the user by having an interface with accomplished motivation levels that is appealing for children and impacts positively on their rehabilitation process. FeMo agrees with this aspect since there is a usability-friendliness of 88.9%.

To conclude, immersive systems that use AR-based techniques can be converted into support systems for conventional motor therapy. It is noteworthy to restate that the research agrees with Viñas-Diz and Sobrido-Prieto [21] who conclude that immersive environments are a complementary therapeutic resource or an alternative to the conventional therapies in the approach of motor function.

9. Conclusions

Immersive systems that use AR techniques offer a set of tools that can be applied for support in therapy. For development of the motor skills of the human body, FeMo has 88.9% usability according to the performed study.

The interface developed in FeMo has a user-friendliness percentage of 100% de-fining that the platform has a positive visual impact on the public to develop the user’s fine motor skills.

In terms of the clarity of the tasks demanded to the user, this aspect needs to be improved with 66.7% seeking a complete environment clear to the user.

Currently, AR systems all over the world are used with therapeutic purposes giving significant advances in the development of the patient’s motor skills.

When immersive systems are created using AR techniques, the target population must be considered to determine how to distribute and create the interface since there is a motivation purpose that seeks to maximize motor skills.


[1] Medina-Alva, M., Kahn, I.C., Huerta, P.M., Sánchez, J.L., Calixto, J.M., Sánchez, S.M.V. (2015). Neurodesarrollo infantil: características normales y signos de alarma en el niño menor de cinco años. Revista Peruana de Medicina Experimental y Salud Publica, 32(3): 565-573.

[2] Chang, M.L., Shih, C.H. (2014). Improving fine motor activities of people with disabilities by using the response-stimulation strategy with a standard keyboard. Research in Developmental Disabilities, 35(8): 1863-1867.

[3] Endo, T., Kawasaki, H. (2015). A fine motor skill training system using multi-fingered haptic interface robot. International Journal of Human-Computer Studies, 84: 41-50.

[4] Lin, C.K., Meng, L.F., Yu, Y.W., Chen, C.K., Li, K.H. (2014). Factor analysis of the contextual fine motor questionnaire in children. Research in Developmental Disabilities, 35(2): 512-519.

[5] Mohamadipanah, H., Parthiban, C., Nathwani, J., Rutherford, D., DiMarco, S., Pugh, C. (2016). Can a virtual reality assessment of fine motor skill predict successful central line insertion? The American Journal of Surgery, 212(4): 573-578.

[6] Ponce-Meza, J. (2017). Early care in children with neurodevelopmental disorders. Propósitos y Representaciones, 5(1): 403-422.

[7] Zapata, M. (2015). Fisioterapia infantil para desarrollo de la motricidad fina.

[8] Tan, X.R., Liu, Z.H., Li, R.H., Xie, J.S., Zhao, Y.L., Luo, G.J., Zhao, W.J., Jin, B.X. (2016). Clinical observation on acupuncture plus occupation therapy for fine motor functions in children with spastic cerebral palsy. Journal of Acupuncture and Tuina Science, 14(15): 328-332.

[9] Juste-Cuesta, L., Rubio-Belmonte, C., Franco-Castellano, I., Gómez-Martínez, M. (2016). Estudio de caso. Terapia del movimiento inducido por restricción en parálisis cerebral infantil. Rehabilitación, 50(2): 125-128.

[10] Salamanca, L., Naranjo, M., González, A. (2012). Traducción al español del cuestionario para diagnóstico de trastorno del desarrollo de la coordinación. Revista Ciencias de la Salud, 10(2): 31-42.

[11] Lampe, R., Thiennel, A., Mitternacht, J., Blumenstein, T., Turova, V., Alves-Pinto, A. (2015). Piano training in youths with hand motor impairments after damage to the developing brain. Neuropsychiatr Disease and Treatment, 11: 1929-1938.

[12] El Wahab, M.A., Hamed, N.E.S. (2015). Effect of hand-arm bimanual intensive therapy on fine-motor performance in children with hemiplegic cerebral palsy. Egyptian Journal of Medical Human Genetics, 16(1): 55-59.

[13] Fernández, F.J., Gomez-Conesa, A., Pérez, J. (2016). Early physiotherapy effects in preterm infant gross and fine motor development during the first 18 months of life. Physiotherapy, 102(1): 198-199.

[14] Yilmaz, R.M. (2016). Educational magic toys developed with augmented reality technology for early childhood education. Computers in Human Behavior, 54: 240-248.

[15] Camacho, J., Oropeza, E., lozoya, O. (2017). Internet de las cosas y Realidad Aumentada: Una fusión del mundo con la tecnología. ReCIBE, 6(1).

[16] Diez-Alegre, M.I., Muñoz-Hellín, E. (2013). Empleo de sistemas de realidad virtual sobre la extremidad superior en niños con parálisis cerebral. Revisión de la literatura. Fisioterapia, 35(3): 119-125.

[17] Sebastián, M.P.Y., Yagüe Sebastián, M.M., Lekuona Amiano, A., Sanz Rubio, M.C. (2016). Los videojuegos en el tratamiento fisioterápico de la parálisis cerebral. Fisioterapia, 38(6): 295-302.

[18] V. Robles-García, "Virtual Reality : a validated and effective tool to evaluate and treat finger movement impairments in Parkinson's disease", Ph.D. dissertation, Dept. Bioloxía Celular e Molecular, Universidade da Coruña, 2014

[19] Soloriio, M. (2013). Metodología en Cascada.

[20] Iosa. M., Morone.,G., Fusco, A., Castagnoli, M., Fusco, F.R., Pratesi, L., Paolucci, S. (2015). Leap motion controlled videogame-based therapy for rehabilitation of elderly patients with subacute stroke: A feasibility pilot study. Topics in Stroke Rehabilitation, 22(4): 306-316.

[21] Viñas-Diz, S., Sobrido-Prieto, M. (2016). Virtual reality for therapeutic purposes in stroke: A systematic review. Neurología, 31(4): 255-277.