AI-Driven Eye-Controlled Virtual Keyboard Systems for Motor-Impaired Users: A Systematic Literature Review of Biosignal and Vision-Based Interaction Modalities

Technological advances have improved the quality of life, but not all individuals can benefit equally. Disability, the prevalence of which is increasing due to health, environmental, and personal factors, affects approximately 1.3 billion people-or 16% of the world's population, according to the WHO (2023) [1]. People with disabilities often experience poorer health, a higher risk of mortality, and limitations in daily activities [1-5]. Motor disabilities, particularly those that limit hand and finger movements resulting from accidents, neurodegenerative disorders (e.g., ALS), cerebral palsy, stroke, or spinal cord injury [6-9], pose significant barriers to accessing digital devices that require precision input [10-13]. Assistive technology (AT) plays a crucial role in supporting the independence of people with disabilities by providing devices that improve quality of life [14-16]. The WHO in 2024 reported that more than 2.5 billion people currently need at least one assistive device, with this figure projected to increase to 3.5 billion by 2050; however, this need remains largely unmet [17]. This gap has driven human-computer interaction (HCI) research toward touchless interaction, in which physical keyboards prove less effective for individuals with motor disabilities, thereby positioning eye-controlled virtual keyboards as a critical alternative [18-23]. These systems enable hands-free communication, information access, and device control through eye movement and biological signal-based interfaces. Specifically, EOG and EEG-based approaches facilitate device control via eye movements and brain activity, demonstrating effectiveness in applications such as virtual typing, wheelchair navigation, smart home control, robotics [2, 5, 7, 10, 20, 23-36].

In the development of eye-controlled virtual keyboards, artificial intelligence (AI) plays a key role in enabling fast, accurate, and completely touchless interactions [22-24, 36, 37]. Typically, such a system involves several main stages: acquisition of eye movement data or natural electrical signals, noise removal, pattern detection, feature extraction, and classification. Within an AI-based virtual keyboard system, eye movement information can be obtained through various modalities, including vision-based input, such as high-resolution cameras and eye trackers [38-45], as well as biosignal-based inputs, such as electrooculography (EOG) [46-56] and electroencephalography (EEG) [57-63].

Vision-based approaches remain widely adopted due to their technological maturity and relative ease of implementation. However, biosignal-based methods offer unique advantages, particularly for users with severe motor impairments. These advantages include robustness to varying lighting conditions, reduced reliance on posture and head stability, and direct physiological representation of eye movements. In contrast, conventional non-AI approaches, such as thresholding, rule-based decision-making, or template matching, have limitations in recognizing complex eye movement patterns. process [38, 60, 62]. Therefore, many studies have shifted to AI-based approaches that utilize machine learning (ML) and deep learning (DL) algorithms, including SVM, KNN, CNN, and LSTM, to enhance the accuracy, speed, and robustness of classification in eye-controlled virtual keyboard systems [4-10, 28-32, 45-51].

Although several previous literature reviews have discussed AI-based HCI, including emotion and facial expression recognition using various sensors [64] and eye gaze-based applications [65], these studies have generally focused on a single input modality in isolation. To date, a systematic review that specifically and comparatively synthesizes the implementation of artificial intelligence models across the two major technological pathways, biosignal-based approaches (EOG or EEG) and computer vision-based approaches (camera or eye tracker), particularly for virtual keyboard systems, remains absent. A thorough understanding of the differences in characteristics between electrical signals and visual data is crucial for determining the most effective artificial intelligence architectures to address technical constraints and user physiological variations.

To address this gap, the present study provides a systematic literature review (SLR) that comprehensively maps and compares the integration of artificial intelligence across diverse eye movement input modalities, both biosignal-based and vision-based. By incorporating these two approaches, this article offers an in-depth analysis of the trade-offs between technical accuracy, signal stability, and user comfort across different input modalities. Finally, the review evaluates dataset characteristics, participant profiles, and performance metrics to formulate a future research agenda that is more inclusive of people with motor disabilities.

3.1 AI approaches and technical mapping for eye-controlled virtual keyboard

Based on a review of 59 studies, 61% of eye-controlled virtual keyboard systems have now transitioned to using Artificial Intelligence (AI), employing either Machine Learning or Deep Learning methods. As illustrated in Figure 4, this key trend indicates a marked shift from conventional rule-based methods (17%) and non-AI vision approaches (22%) toward systems capable of autonomous pattern recognition. The underlying reason for this shift is AI's superiority in processing eye signals, which are often nonlinear and noisy, a technical challenge that has been difficult to overcome using traditional thresholding methods alone.

Figure 4. Distribution of approaches used in the eye-controlled virtual keyboard systems

3.1.1 Model evolution: rationalizing architecture selection

The technical mapping in Table 2 shows the evolutionary trajectory across three main phases:

1. Classical ML for efficiency (2019–2020): In the early stages, research relied heavily on architectures such as Time Delay Neural Networks (TDNN) and Artificial Neural Networks (ANN). These models were chosen for their stability and low computational resource requirements. For example, Tang et al. [9] used a TDNN to classify nine gaze directions with 91–92% accuracy, while Martinez-Ledezma et al. [34] applied an MLP-based ANN to recognize eye-blink patterns.
2. Deep Learning for adaptive representation (2021–2023): Subsequently, the field shifted strategically toward the use of Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks. CNNs are adopted to automate spatial feature extraction in vision-based systems, while LSTMs, as demonstrated by Reyes et al. [25], are used to capture temporal dependencies in EEG/EOG signals. This transition aims to minimize the need for complex manual feature engineering. The technical rationale for the dominance of CNNs over RNNs and traditional ML models lies in their structural efficiency for spatial hierarchical mapping. CNNs are uniquely capable of identifying eye-movement signatures, such as blink spikes or saccade directions, within noisy signal streams through weight-sharing, which significantly reduces the computational footprint during inference compared to LSTMs. This provides a crucial performance trade-off: while LSTMs are better at capturing long-term temporal dependencies, CNNs offer the millisecond-level responsiveness essential for a fluid, real-time typing experience.
3. Hybridization and metaheuristic optimization (2023–2025): Recent studies have begun to focus on integrating Deep Learning with metaheuristic optimizers to balance precision and real-time execution speed. Examples include Zeng et al.'s MIDF-NET [50] and Son et al.'s hierarchical CNN [8], which achieved high accuracy, up to 92.16%, through multi-scale signal derivative processing. Furthermore, hybrid systems, such as the Tobii DCNN [28] and YOLOv3 [51], combine real-time processing capabilities with optimized model depth.

3.1.2 Performance trade-offs and modality integration

The data in Table 2 highlight a crucial technical trade-off: while deeper models like CNNs can offer superior accuracy, they pose significant latency challenges on embedded hardware. Conversely, simpler ML models remain a relevant option for portable systems due to their lower power consumption. Ultimately, the convergence of these various AI approaches facilitates the integration of multiple modalities, such as EOG, EEG, and vision, to provide a more responsive and layered input system for users with motor disabilities.

3.2 Classification and distribution of input modalities

Based on a review of 59 studies summarized in Table 2, the choice of input modality is determined by the balance between signal precision and user comfort. The data distribution in Figure 5 confirms the dominance of biological signals over vision-based techniques in virtual keyboard development.

Figure 5. Input modalities and acquisition techniques

3.2.1 Biological signals: EOG Stability and EEG complexity

Biosignal-based systems are the most widely explored category in the literature, encompassing 36 articles, divided into two main modalities: EOG and EEG.

1. EOG dominance (22 pure articles and 2 hybrid)

EOG is the most dominant modality due to its ability to directly record the eye's electrical activity with a high Signal-to-Noise Ratio (SNR). The technical rationale for using EOG lies in its reliability in detecting eye movement directions in up to nine directions through the placement of three to six electrodes. Critical analysis suggests a shift in algorithms from static thresholding to CNN [8, 30] and RNN [36] models to address inter-individual signal variability. The main drawback of EOG is its invasiveness; however, recent trends are beginning to address this through wearable devices such as the JINS MEME smartglasses [53] or microcontroller-based wireless systems [18, 52]. Additionally, two studies adopted a hybrid EOG-based approach, combining it with sEMG for enhanced muscle control [19] and physiological ECG/PPG signals [4]. This approach aims to improve system reliability against inter-user variability and environmental disturbances that often occur in biosignal-based assistive applications.

2. EEG potential (10 pure and 2 hybrid)

EEG was used in 10 studies as a single modality to capture users' cognitive intentions and attention levels, typically through wireless devices such as the NeuroSky MindWave [29, 35, 58, 63]. While offering the potential for deeper cognitive engagement, EEG faces significant challenges due to its high sensitivity to facial muscle artifacts and non-neural noise, necessitating adaptive models such as ANNs and LSTM networks to maintain stability and classification accuracy [4, 25, 34]. Two hybrid studies with EEG were found, namely the integration of EEG and EOG in one hybrid framework [31], the integration of EEG with motion sensors and facial expression detection [61], which confirms that this biosignal fusion is still relatively unexplored despite having high complementary potential.

3.2.2 Eye tracking and computer vision technologies: Accessibility vs. robustness

1. Hardware-based gaze tracking (14 articles).

This group relies on active infrared sensors and the Pupil Center Corneal Reflection (PCCR) principle, as implemented in Tobii, Gazepoint, and Arrington Research devices [2, 23, 28, 38, 40, 41]. The primary focus of this approach is the precise mapping of gaze coordinates (x, y) for direct cursor navigation. However, this approach is susceptible to the Midas Touch phenomenon, requiring additional validation mechanisms such as dwell time, smooth pursuit, or time-series-based prediction [40-42]. This category also includes Augmented Reality (AR) integration via HoloLens 2 [37], fixation monitoring, and gaze-based supervised typing systems.

2. Standard camera-based computer vision (9 articles).

This approach is non-invasive and leverages standard RGB cameras on webcams or smartphones to increase accessibility and reduce hardware costs [11, 13, 45]. The technical focus is on detecting facial landmarks using libraries such as MediaPipe, Dlib, or OpenCV, with algorithms such as the Eye Aspect Ratio (EAR) used to extract eye actions, particularly blinks, as input triggers [3]. While easy to implement, standard camera-based systems exhibit lower robustness to variations in lighting and head position than infrared sensor-based systems, often requiring additional preprocessing to maintain consistent performance.

This critical synthesis reveals a tension between the robustness and accessibility of existing modalities. EOG-based systems excel in temporal resolution and reliability in dark environments but are constrained by the physical invasiveness of electrodes. Conversely, vision-based systems offer high spatial precision for cursor navigation, but are susceptible to interference from lighting and head position. The lack of a “middle ground” solution that balances the stability of EOG with the practicality of RGB cameras represent a major gap in the development of truly adaptive assistive technology.

3.3 Dataset and participant

The characteristics of the datasets and test participants from 59 studies reveal a consistent pattern: 56 studies used private datasets from direct experiments, only one study used a public dataset [50], and 2 studies combined both [28, 45], utilizing datasets such as SBVPI and HideMyGaze. In terms of participants, healthy individuals dominate (49%), followed by studies that do not specify the participants' condition (41%), those with a mix of healthy participants and individuals with disabilities (8%), and finally, studies focusing exclusively on individuals with disabilities (2%). The number of participants ranges from 3 to 60, typically falling between 10 and 30, depending on the experimental design. The age range spans from 17 to 75 years, with a predominance of young adults (17–25 years), mostly students with normal vision, followed by adults (26–45 years), who are often involved in studies using EOG or eye-tracking systems. Older adults (46–75 years) appear only in studies involving individuals with motor disabilities such as ALS, stroke, or cerebral palsy [3, 5, 22, 44]. The distribution of test participants' conditions in eye-controlled virtual keyboard systems is illustrated in Figure 6.

Figure 6. Percentage of participants' conditions involved in testing the eye-controlled virtual keyboard systems

In general, studies using private datasets and healthy participants focus on the development and validation of eye movement classification models, while studies involving participants with disabilities emphasize the implementation of virtual control systems and assistive interfaces. This pattern indicates that most studies are still at the proof-of-concept stage, with testing conducted under stable physiological conditions to ensure signal reliability and algorithm accuracy. Participants without disabilities produce low-noise signals and consistent movement patterns, which facilitate the calibration and training of models such as CNNs and SVMs. However, they introduce data bias due to differences in signal patterns, muscle fatigue, and cognitive responses in users with disabilities [73-75], which degrade model performance and limit the external validity of the system [75-77]. This observation aligns with Bissoli et al. [2], Lin et al. [18], and Qin et al. [59], who noted that most biological signal-based systems are still tested on small samples of healthy participants under controlled conditions. All three studies emphasize the need for further validation with users with disabilities and expansion to more heterogeneous test participant samples to address physiological biases, ensure safety, and enhance the clinical and social relevance of the developed systems.

From an ethical perspective, all research involving humans requires ethical approval to ensure participant safety and privacy [34]. However, limited access to disability communities and rehabilitation facilities constrains participant recruitment in this research area. As highlighted by recent research ethics reviews [78, 79], research involving individuals with disabilities must adhere to the principles of responsible inclusion. These principles include ensuring the accessibility of informed consent materials, adapting safe and equitable test procedures, and respecting participant autonomy. Moving forward, research should prioritize transitioning to the clinical stage through a co-design approach that directly involves users with disabilities. This requires strengthening cross-institutional collaboration among laboratories, hospitals, and disability communities, as well as establishing comprehensive ethics protocols and generating representative public datasets. This step will help ensure that the assistive technology developed is not only technically valid but also relevant and socially beneficial, in accordance with the recommendations of the Global Report on Assistive Technology [80].

3.4 The metrics evaluation

The eye-controlled virtual keyboard systems demonstrated high levels of accuracy and usability. Most studies reported accuracy values above 90%, indicating that methods utilizing EOG, EEG, and camera-based gaze tracking signals are capable of reliably recognizing gaze direction and eye blinks for use as control commands. In terms of usability, testing using the System Usability Scale (SUS) achieved a score of 89.9 for healthy users and 92.5 for users with disabilities [2]. This suggests that the system was perceived as easy to understand and comfortable to use by both groups. This finding is supported by another study, which reported a usability score of 90.6 for users with disabilities, with an average typing speed of 15.8 characters per minute (CPM) compared to 21.7 CPM for healthy users [11]. Notably, despite their slightly slower typing speed, users with disabilities reported higher comfort and satisfaction, underscoring the effectiveness of assistive technology in enhancing digital accessibility [61].

In terms of system performance, most models are able to recognize eye movements with an accuracy of 97–99% [5], achieving 100% precision for right and right-left movements and a real-time wheelchair control success rate of 92.5%. Other studies report accuracies of 94–96% [46] , with some reaching 99.7% for both vertical and horizontal channels [36]. A combined EOG and EEG system also demonstrates high performance, with an accuracy exceeding 92%, further confirming the effectiveness of a multimodal approach [51]. Interestingly, system performance remains relatively stable under varying usage conditions. For example, the accuracy rate remains high (91–94%) whether the smartphone is held in the hand or placed on a stand [13]. A significant decrease in accuracy (up to 50%) occurs only in dim lighting conditions and when the face is turned sideways [31]. In addition, the stability of the EOG signal was successfully improved through the use of headband electrodes, which reduced interference amplitude by up to 3.6 times on the vertical axis and 5.8 times on the horizontal axis, while increasing signal consistency by up to 12.1% [4].

Several studies have also evaluated system performance across various age groups and medical conditions. Average accuracy reached 94% in young adults, 93.27% in older adults, and 90.37% in patients with ALS [6], demonstrating the system's adaptability to diverse user characteristics. The integration of sEMG and EOG signals further improved command recognition accuracy to over 92% [19], while a hybrid BCI system achieved an accuracy of 96.65% ± 1.44% with an average response time of 2.65 seconds [31]. Other research has focused on functional efficiency and usability. Several studies report a 100% success rate for gaze-based activities [41], a 43.05% reduction in input errors compared to gaze-only methods [37], and a 25% increase in word selection efficiency on a priority-based keyboard layout compared to a traditional QWERTY keyboard [62]. Furthermore, single blinks have been shown to be the most stable form of input, with accuracy reaching 98% [63].

Overall, the results across various studies indicate that eye-controlled virtual keyboard systems have achieved an average accuracy of 90–99%, a response time of under three seconds, a very low error rate (ERR ≈ 0), and a usability score above 85. These findings reinforce that eye movement-based technology, whether utilizing EOG, EEG, or camera-based signals has reached a high level of technological maturity and is ready for broader testing in clinical contexts and assistive technology applications for individuals with motor disabilities.

3.5 Research gap

Based on the comparative synthesis presented in Tables 2-4, the current body of research on eye-controlled virtual keyboard systems exhibits clear empirical tendencies rather than a unified methodological consensus. One notable pattern is the frequent adoption of EOG as the primary input modality, often paired with machine learning or deep learning models, most commonly CNN-based architectures, due to their favourable balance between signal responsiveness and computational efficiency. However, this review indicates that this dominant configuration has not yet resolved several persistent and interrelated limitations.

A first research gap concerns limited generalizability arising from population bias and dataset constraints. Across modalities and algorithms, the majority of studies rely on small-scale, laboratory-based datasets involving predominantly young and healthy participants, with minimal inclusion of users with motor disabilities [2, 10, 11, 22, 44]. As reflected in Table 3, this limitation is not modality-specific but is shared by both biosignal-based and vision-based approaches. Consequently, performance metrics, particularly accuracy, should be interpreted cautiously, as they may not generalize to real-world assistive environments characterized by fatigue and user variability.

Second, the review reveals a misalignment between signal characteristics and model assumptions. Biosignal-based modalities such as EOG and EEG are inherently non-stationary and sensitive to artifacts, baseline drift, and electrode-related variability [4, 5, 18, 34, 51, 54, 55]. While CNNs and other deep learning models are increasingly used to improve feature extraction, most implementations implicitly assume short-term signal stability and are evaluated under controlled conditions. This creates a gap between algorithmic performance and sustained system reliability during prolonged interaction.

Third, as summarized in Table 4, robustness–usability trade-offs remain insufficiently addressed across modalities. Vision-based systems offer non-contact interaction but are highly vulnerable to environmental variability, head movement, and optical interference [2, 3, 11, 22, 45]. Conversely, biosignal-based systems provide stable temporal responsiveness but often compromise user comfort due to electrode placement and maintenance requirements. Despite these complementary strengths and weaknesses, most systems remain single-modality, limiting their ability to adapt to dynamic usage conditions.

A further gap emerges at the system level. High-performance models, whether based on CNNs, LSTMs, or hybrid architectures, are predominantly evaluated offline, with limited attention to real-time latency, embedded deployment, or long-term operational stability [4, 9, 23, 30, 37, 40, 46, 55, 56]. This disconnect constrains the practical translation of research prototypes into deployable assistive technologies. Finally, from a human–computer interaction perspective, issues such as visual fatigue, cognitive load, involuntary activation, and user discomfort are widely acknowledged but rarely integrated into model design or evaluation criteria [19, 23, 30, 39, 41, 44, 58] As reflected in Table 4, these factors play a critical role in real-world adoption yet remain secondary to accuracy-driven optimization. Taken together, these gaps indicate that the field's current limitations do not stem from the absence of effective sensing modalities or learning techniques. Rather, they arise from insufficient integration between empirical modality trends, adaptive modeling strategies, system-level constraints, and user-centered evaluation.

3.6 Discuss and recommendations

Based on the identified gaps, future research should focus on strengthening the integration and adaptability of existing approaches rather than prioritizing a single modality or algorithm family. A key agenda is the development of more representative and longitudinal datasets that encompass users with motor disabilities, diverse age groups, and long-term use sessions. This approach is crucial for enabling cross-subject and cross-condition validation as a more standardized and clinically meaningful evaluation practice [59]. To address the non-stationary nature of biosignals and the high inter-user variability observed across modalities, future systems must adopt adaptive learning mechanisms capable of handling temporal shifts and individual physiological differences. Techniques such as transfer learning, incremental adaptation, and lightweight reinforcement learning hold significant potential for supporting this goal without incurring excessive computational burden [33, 47, 48, 51, 52] Crucially, these approaches must be evaluated in continuous real-time operation, not merely in short-term experimental scenarios. Based on the synthesis of modality characteristics presented in Table 4, the development of a multimodal interaction framework represents a crucial research direction. Rather than replacing currently dominant approaches—such as EOG-based control or CNN-based classification—multimodal systems can leverage the complementary strengths of different modalities by dynamically adjusting the weights of sensor contributions in response to environmental conditions, signal degradation, or user fatigue [24, 29, 31, 42, 48, 53]. This strategy directly addresses the trade-off between system robustness and user comfort identified in this review.

At the system level, future research should explicitly incorporate implementation-oriented evaluation metrics, such as latency, energy consumption, calibration time, and hardware compatibility, particularly for embedded and wearable assistive platforms [4, 37, 40, 46, 55]. These metrics are essential for bridging the gap between experimental performance and the feasibility of real-world implementations. Finally, future research should elevate human-centered design to a level of importance equal to algorithmic accuracy. Adaptive interfaces that consider visual fatigue, cognitive load, and user comfort, combined with the development of dry electrodes and non-invasive sensors, represent key factors for sustainable use in everyday activities [19, 26, 39, 51, 52]. The integration of these systems into real-world assistive ecosystems, such as virtual keyboards, smart wheelchairs, and smart home environments, must be pursued through close collaboration with end users. Such collaboration is essential to ensure a tangible functional impact on improving independence and quality of life [2, 31, 42, 52, 58].