Hybrid Spatial–Temporal Stabilization for Real-Time Static Arabic Hijaiyah Letter Recognition Using Polynomial Regression

Sign language serves as the primary communication medium for the Deaf and Hard of Hearing (HoH) community, enabling information exchange through hand configurations, finger articulation, and spatial movements [1, 2]. Recent advances in computer vision and machine learning have accelerated the development of Sign Language Recognition (SLR) systems capable of automatically identifying hand gestures from RGB video streams [3-5]. Despite this progress, achieving stable real-time recognition remains challenging. This limitation arises from geometric inconsistencies in detected hand landmarks caused by variations in camera distance, hand orientation, illumination, and motion dynamics [6, 7]. Such instability produces temporal jitter and spatial distortion, which degrade feature consistency and classification performance.

Several techniques have been explored to mitigate landmark fluctuations, including Kalman filtering, Gaussian Mixture Models (GMM), and particle filtering [8-12]. Although effective in specific scenarios, these approaches often involve substantial computational overhead or exhibit limited robustness against distance-induced scaling effects. Meanwhile, landmark-based detectors such as MediaPipe Hands provide efficient and reliable real-time tracking [13, 14], but they lack intrinsic stabilization mechanisms. As a result, these methods remain susceptible to inter-frame noise, particularly during subtle or rapid hand movements.

Advanced SLR models based on CNNs, RNNs, and Vision Transformers have demonstrated strong classification performance [15], yet most focus on feature representation rather than on stabilizing geometric structures of the hand. Consequently, recognition accuracy often deteriorates under dynamic environmental conditions or modest deviations in camera placement [15]. Prior studies have attempted noise reduction—such as Kalman-based temporal filtering or fixed-distance normalization [16-18]—but these approaches struggle with nonlinear landmark deformation or lack adaptability to hand-motion dynamics.

This gap highlights the need for a lightweight, geometry-aware stabilization technique capable of reducing landmark jitter while preserving the topological relationships among fingers and the palm. Polynomial regression smoothing, including Savitzky–Golay filtering, has been widely applied to temporal signals for noise reduction while maintaining shape fidelity [16, 17], yet its application to multi-point 2D/3D landmark trajectories remains underexplored. Likewise, inter-finger distance averaging may reduce hand-size variation [18], but static approaches fail to accommodate real-time motion.

To address these limitations, this study proposes a hybrid spatial–temporal stabilization framework that integrates polynomial regression smoothing and dynamic distance-based normalization to enhance the geometric consistency of MediaPipe landmark sequences. The method aims to mitigate temporal jitter, maintain inter-limb proportionality, and improve class separability without increasing computational complexity. The contributions of this work are threefold:

(1) introducing a hybrid polynomial–distance smoothing mechanism tailored for real-time landmark stability;

(2) developing an enhanced feature representation based on stabilized landmarks, relative coordinates, and palm-orientation cues; and

(3) providing empirical validation through extensive testing on static Arabic–Hijaiyah gestures under varying camera angles and distances.

In this study, the scope is explicitly limited to static Hijaiyah letter gestures, enabling a focused evaluation of geometric stabilization without involving temporal sequence modeling.

This study proposes a hybrid geometric stabilization framework designed to improve the accuracy and temporal consistency of a vision-based SLR system. The stabilization layer is positioned between the landmark detection stage and the gesture classification module, and operates directly on the 21 landmarks provided by the MediaPipe Hands skeleton model. The objective is to suppress temporal jitter and scale variation while preserving the anatomical structure of the hand, thereby producing more discriminative features for Hijaiyah gesture recognition, and Figure 1 illustrates the canonical landmark configuration adopted in this work.

The MediaPipe Hands framework defines 21 hand landmarks distributed across the wrist, finger joints, and fingertips. Landmark 0 corresponds to the wrist, while the remaining landmarks cover the MCP, PIP, DIP, and TIP joints of each finger. This configuration enables detailed modeling of finger articulation and palm geometry. In this study, the raw (x, y, z) coordinates of these 21 landmarks form the baseline feature set and serve as the input to the proposed hybrid stabilization pipeline.

Figure 1. Configuration of 21 hand landmarks based on the MediaPipe hand tracking model

3.1 Polynomial regression layer

The Polynomial Regression Layer is responsible for temporally smoothing the trajectory of each landmark. Let $D_t$ denote a generic distance or coordinate value at frame $t$. A polynomial regression model of order $n$ is defined as:

$\hat{D}_t=a_0+a_1 t+a_2 t^2+\cdots+a_n t^n$   (1)

An exploratory study was conducted on more than 1,000 randomly sampled sequences (60–90 frames each) to select the appropriate polynomial order. The results are summarized in Table 1.

Table 1. Polynomial order selection

Based on these observations, a hybrid polynomial scheme is adopted. Palm-region landmarks—0, 1, 5, 9, 13, and 17—are filtered using a second-order polynomial ($n=2$) due to their relatively smooth and low-curvature motion. In contrast, finger-joint and fingertip landmarks—2-4, 6-8, 10-12, 14-16, and 18-20—are smoothed with a third-order polynomial ($n=3$), allowing the model to capture natural finger curvature while suppressing high-frequency noise. This selective configuration improves temporal stability without oversmoothing fine geometric patterns that are essential for discriminating between similar Hijaiyah letters.

A minimum of 10 frames per trajectory is required for reliable polynomial fitting. The temporal filtering is applied independently to each landmark coordinate (x, y, z) prior to further processing.

3.2 Distance smoothing and geometric normalization

To address spatial variations caused by changes in hand-to-camera distance, the second layer of the pipeline operates on inter-landmark distances. For a pair of landmarks $(i, j)$, the Euclidean distance in the normalized MediaPipe space is defined as:

$D_{i j}=\sqrt{\left(x_i-x_j\right)^2+\left(y_i-y_j\right)^2+\left(z_i-z_j\right)^2}$   (2)

Let:

• $D_{i j}(t)$: raw distance between landmarks $i$ and $j$ at time $t$,
• $D_{\text {ref }}(t)=D_{0,9}(t)$: reference distance between the wrist (0) and the middle-finger base (9),
• $D_{\text {ref }}^{\mathrm{cm}}$: fixed real-world reference distance in centimeters.

An adaptive normalization is then defined as:

$D_{i j}^{\mathrm{norm}}(t)=\frac{D_{i j}(t)}{D_{\mathrm{ref}}(t)} \times D_{\mathrm{ref}}^{\mathrm{cm}}$    (3)

This formulation converts all distances into a camera-distance-invariant representation, where inter-finger distances are expressed relative to a stable anatomical reference. The choice of the (0, 9) pair as reference is motivated by its comparatively low sensitivity to local finger movement and its robustness across different orientations, making it suitable as a scale anchor.

In implementation, distances are initially measured in pixel units (derived from image coordinates) and converted into centimeters using the general form.

Distance$_{\mathrm{cm}}=\frac{\text {Distance}_{\mathrm{px}}}{\text {RefDist}_{\mathrm{px}}} \times$ RefDist$_{\mathrm{cm}}$   (4)

where, RefDist$_{\mathrm{px}}$ is the pixel distance between the chosen reference landmarks and RefDist$_{\mathrm{cm}}$ is its real-world length. This conversion ensures geometric consistency when the user moves closer to or farther from the camera.

3.3 Exponential smoothing layer

To further suppress high-frequency noise in distance-based features, an exponential smoothing layer is applied on top of the normalized distances. For a given normalized distance sequence $D_t^{\text{norm }}$, exponential smoothing is defined as:

$S_t=\alpha D_t^{\text {norm }}+(1-\alpha) S_{t-1}$   (5)

where, $S_t$ denotes the smoothed value at time $t$ and $\alpha \in(0,1]$ is the smoothing factor. A parameter sweep was performed for $\alpha \in\{0.1,0.2,0.3,0.5\}$, yielding the qualitative behavior summarized in Table 2.

In this work, α = 0.2-0.3 is used depending on gesture type and session conditions, providing a practical trade-off between responsiveness and temporal stability.

Table 2. Smoothing parameter behavior

3.4 Hybrid stabilization workflow

The proposed hybrid model integrates the three layers described above into a unified stabilization pipeline:

• Temporal filtering (Polynomial Regression): Each landmark coordinate is smoothed using a second- or third-order polynomial depending on its anatomical role, reducing jitter while preserving natural motion.

• Spatial normalization (Distance Smoothing): Inter-finger distances and reference distances are normalized using Eq. (3) and converted into a camera-distance-invariant representation via Eq. (4).

• Output refinement (Exponential Smoothing): Normalized distances are further filtered using exponential smoothing (Eq. (5)) to attenuate residual noise and micro-oscillations.

The result is a temporally coherent and scale-invariant description of hand geometry that can be directly used as input to lightweight classifiers such as k-Nearest Neighbors (k-NN).

3.5 Reproducibility settings

To ensure reproducibility of the proposed framework, all experiments were conducted under the following settings:

• Frame rate: 30 FPS; resolution: 720 × 480
• Maximum polynomial order: 3 (with hybrid order-2 and order-3 assignment as described in Section 3.1)
• Exponential smoothing factor: 0.2-0.3
• Reference landmark pair for scale normalization: (0, 9)
• Minimum frames for polynomial fitting: 10
• Each gesture recorded in at least 5 cycles per session
• Implementation: Python with MediaPipe Hands, NumPy, and scikit-learn.

These settings are kept constant across baseline and proposed experiments to enable fair comparison.

3.6 Experimental setup

The baseline dataset was constructed using a purely vision-based tracking approach. The MediaPipe Hands model was employed to extract 21 real-time landmarks from the right hand, yielding 63 raw features $\left(x_0, y_0, z_0\right), \ldots,\left(x_{20}, y_{20}, z_{20}\right)$ per frame. MediaPipe was selected due to its robustness, high frame rate, and suitability for commodity RGB cameras. A total of 28 static Arabic/Hijaiyah gesture classes were recorded, from Alif to Ya, across multiple recording sessions with controlled variations in camera distance, hand rotation, and illumination. All samples were stored in CSV format, preserving the same column structure—timestamp, frame index, camera distance estimate, gesture label, and landmark coordinates. Figure 2 provides an overview of the 28 Hijaiyah hand gestures included in the dataset.

Figure 2. The 28-class Arabic/Hijaiyah hand gesture set

This baseline dataset represents the unprocessed landmark distribution used both as a reference for comparison and as input to the subsequent stabilization pipeline.

3.6.1 Baseline definition

The baseline model in this study represents a conventional landmark-based recognition pipeline without any explicit geometric stabilization mechanism. Specifically, the baseline utilizes raw 21-point hand landmarks extracted directly from the MediaPipe Hands framework, preserving the original (x, y, z) coordinates across frames.

Unlike the proposed method, the baseline does not incorporate:

(i) polynomial regression smoothing,

(ii) distance-based normalization,

(iii) exponential smoothing, or

(iv) orientation encoding.

This design ensures that the baseline reflects a commonly adopted lightweight landmark-based recognition approach in the literature.

For fair comparison, both baseline and proposed methods share identical dataset, classifier, and evaluation protocol.

3.6.2 Proposed method definition

The proposed method extends the baseline landmark-based representation by introducing a hybrid spatial–temporal stabilization pipeline. This framework integrates multiple complementary components designed to enhance geometric consistency and temporal coherence of hand landmark features.

Specifically, the proposed method incorporates:

(i) polynomial regression smoothing to reduce temporal jitter in landmark trajectories,

(ii) distance-based normalization to mitigate scale variation caused by hand-to-camera distance changes,

(iii) exponential smoothing to suppress residual high-frequency noise, and

(iv) palm-orientation encoding to enrich discriminative spatial features.

All transformations are applied sequentially to the raw landmark coordinates, producing a stabilized and scale-invariant feature representation.

Importantly, the proposed method operates under the same experimental conditions as the baseline, including identical dataset, classifier configuration, and evaluation protocols. This ensures that performance improvements are solely attributed to the effectiveness of the hybrid stabilization framework.

3.7 Proposed dataset collection (hybrid-stabilized landmark acquisition)

A second dataset, referred to as the proposed hybrid-stabilized dataset, was collected using the same recording setup but with the hybrid stabilization pipeline applied online during acquisition. For each frame, the 21 landmarks from MediaPipe were processed through four sequential modules:

3.7.1 Polynomial regression smoothing

Temporal jitter and micro-oscillations in each landmark trajectory were attenuated using the hybrid polynomial scheme. Palm-related landmarks (0, 1, 5, 9, 13, 17) were smoothed with second-order polynomials, while fingertip and joint landmarks (2-4, 6-8, 10-12, 14-16, 18-20) used third-order polynomials. This configuration preserves subtle curvature in finger movements that is crucial for discriminating visual similarities among Hijaiyah letters.

3.7.2 Distance smoothing using exponential filtering

Camera distance and all inter-finger distances were further stabilized with exponential smoothing using a small smoothing factor (α = 0.2-0.3). This step reduces high-frequency depth noise and fluctuations caused by hand translation toward or away from the camera, resulting in more coherent modeling of hand scale and finger spread.

3.7.3 Relative landmark feature transformation

To reduce sensitivity to global hand displacement within the image plane, all landmarks were transformed to a wrist-centered coordinate system:

$\left(x_i^{\prime}, y_i^{\prime}, z_i^{\prime}\right)=\left(x_i-x_0, y_i-y_0, z_i-z_0\right)$

where, $\left(x_0, y_0, z_0\right)$ corresponds to the wrist landmark. This Relative Landmark Feature (RLF) representation retains the intrinsic geometry of the hand while discarding absolute positional offsets, improving robustness to user movement and framing differences.

3.7.4 Palm normal and orientation encoding

To incorporate 3D hand-orientation information, a palm normal vector $\mathbf{n}=\left(n_x, n_y, n_z\right)$ was computed using the cross product of vectors $\overrightarrow{P_0 P_5}$ and $\overrightarrow{P_0 P_{17}}$.

The resulting normal was normalized and converted into yaw and pitch angles:

yaw $=\arctan 2\left(n_y, n_x\right)$, pitch $=\arctan 2\left(n_z, \sqrt{n_x^2+n_y^2}\right)$

These orientation descriptors capture pronation, supination, and palm tilt, which are particularly important for discriminating visually similar Hijaiyah gestures.

The final proposed dataset thus combines:

• hybrid polynomial filtering,
• distance and camera-depth smoothing,
• RLF-based geometric normalization, and
• palm normal orientation features.

All data were stored in a unified CSV structure, enabling direct comparison with the baseline dataset.

3.8 Evaluation protocols

To rigorously assess the impact of the proposed stabilization framework, two complementary evaluation protocols were employed: (i) offline temporal separability analysis and (ii) session-aware robustness testing.

3.8.1 Offline evaluation (80/20 temporal split)

For the offline evaluation, a maximum of 400 frames per gesture class was retained to construct a balanced dataset of 11,200 samples. For each gesture, the first 80% of frames (in temporal order) were used for training, and the remaining 20% were reserved for testing. All features were standardized using z-score normalization. Gesture classification was performed using a k-Nearest Neighbor classifier with $k=3$. This protocol quantifies the degree to which the proposed stabilized features improve class separability under controlled temporal conditions.

3.8.2 Session-aware robustness evaluation

To approximate real-world deployment conditions, the full dataset was partitioned into five temporal sessions based on timestamp quantiles, reflecting different recording sessions and environmental configurations (e.g., distance, orientation, and illumination). A 5-fold session-aware cross-validation scheme was adopted, where in each fold one entire session was held out exclusively for testing, while the remaining sessions were used for training. Gaussian noise was added to the test features to emulate sensor-level perturbations and minor calibration mismatches. This evaluation scenario deliberately increases difficulty and is designed to measure the robustness of the proposed representation against distribution shifts.

The proposed methodology thus combines temporal, spatial, and orientation-aware stabilization mechanisms tailored for landmark-based Arabic SLR. The next section presents experimental results demonstrating the effectiveness of the hybrid-stabilized dataset in improving stability and recognition performance relative to the baseline.

To support the improvement of the accuracy of hand movement detection in the SLR system, in Figure 3, the author designs a proposed hybrid approach based on Polynomial Regression and Distance Smoothing Modeling which functions as a geometric stabilization layer, a comparative analysis of the system architecture between the conventional model and the proposed Polynomial Regression model, namely.

Figure 3. Comparison diagram of proposed model