Ju Wang
| Lianpeng Yun* | Jiyuan Zhang
© 2023 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
With advancements in deep learning and image processing technology, evaluations of lower limb rehabilitation training have witnessed improved accuracy and efficiency. However, traditional image processing techniques frequently neglect the inherent sequence-level characteristics during action recognition and fail to exploit the comprehensive full-field data when discerning rehabilitation training action standards. Addressing these limitations, a novel method was proposed. Initially, the GaitSet algorithm was employed to recognize the lower limb rehabilitation training actions, ensuring complete consideration of sequence-level features. Subsequently, leveraging the full-field optical flow tracking approach, challenges associated with discerning the standards of lower limb rehabilitation training actions were examined. It is anticipated that this novel methodology can offer an enhanced tool for evaluating the effectiveness of lower limb rehabilitation training in the realm of rehabilitation medicine. Such advancements could potentially contribute to optimizing rehabilitation outcomes and augmenting patients' quality of life.
lower limb rehabilitation training, action recognition, original sequence-level features, GaitSet algorithm, full-field optical flow tracking method, rehabilitation effect evaluation
With advancements in contemporary medical technology, the evaluation of lower limb rehabilitation training effects has emerged as a pivotal component in clinical medicine and rehabilitation science research [1-5]. The criticality of lower limb movement in facilitating daily human activities cannot be understated. Activities ranging from walking and running to climbing stairs and jumping are contingent upon the coordinated actions of the lower limbs [6-11]. Consequently, for patients navigating the recovery journey post-surgery or injury, accurate and timely assessment of rehabilitation outcomes is crucial as it profoundly influences adjustments to the rehabilitation process and, ultimately, the realization of rehabilitation goals [12-16]. Such evaluations are posited to equip medical professionals and rehabilitation therapists with a deeper understanding of the patient's progress, enabling them to tailor training regimens accordingly. Moreover, a transparent and prompt feedback system offers patients a lucid view of their rehabilitation trajectory, potentially bolstering their motivation and assurance, factors integral to fostering the rehabilitation process.
In this landscape, the integration of deep learning and image processing technology in assessing rehabilitation outcomes has been noted [17, 18]. These technologies adeptly register the nuances of patient movements and the evolution of these actions throughout the rehabilitation tenure, laying the foundation for a more methodical evaluation [17, 18]. However, despite their transformative potential in rehabilitation medicine, certain impediments and complexities are observed in their practical deployment [19-21].
A notable limitation lies in the propensity of several image processing techniques to disregard intrinsic sequence-level characteristics during the action recognition phase, potentially undermining the precision and efficiency of the process [22, 23]. These attributes encapsulate pivotal temporal and spatial dimensions of action sequences, elements paramount for meticulous action recognition. Furthermore, in discerning the standards of rehabilitation training actions, a comprehensive utilization of full-field information remains elusive in many techniques [24, 25]. This full-field perspective is cardinal, offering an all-encompassing view that likely mirrors the true essence of the rehabilitation effect more faithfully.
Addressing these challenges, research was undertaken from a bifocal perspective. In Section 2, an innovative approach rooted in the GaitSet algorithm was delineated for recognizing lower limb rehabilitation training actions, placing a premium on intrinsic sequence-level features. The anticipated outcomes from this approach hint at enhanced precision and efficiency in recognizing rehabilitation actions, fortifying the foundational elements of outcome evaluation. Section 3 pivots to the full-field optical flow tracking methodology, focusing on the intricacies of recognizing the benchmarks set for lower limb rehabilitation training actions. The overarching aspiration is to harness this methodology to cultivate a more nuanced and precise recognition and evaluation paradigm.
In conclusion, this study aims to enhance evaluation methods in lower limb rehabilitation. The findings may provide medical professionals with a deeper understanding of the rehabilitation process, potentially improving patient outcomes and quality of life—a valuable contribution to the field of rehabilitation medicine.
In a departure from traditional feature extraction methods, the convolutional network is capable of obtaining higher-level abstract features, all while preserving spatial structural information. As a result, actions are described with heightened precision. It is on this premise that recognition of lower limb rehabilitation training actions was executed in this study, leveraging the GaitSet algorithm. This algorithm is notable for its dual extraction of both frame-level and sequence-level features, a capacity that facilitates comprehensive utilization of temporal and spatial information in action sequences.
The frame-level features expose the specifics of a single action frame, while the sequence-level features denote the evolution of actions over time. It is this combined approach that equips the algorithm to thoroughly comprehend complex actions.
For the identification of lower limb rehabilitation training actions, the architecture of the GaitSet network incorporates four primary components:
(1) The frame-level feature extraction module, a component designed to extract features of a single action frame via a convolutional neural network (CNN). This module bears the main responsibility for capturing the spatial features present in each frame. Through the use of a filter, the convolutional layer scans the input image and extracts pivotal features within the image, such as edges and corner points. This process equips the frame-level feature extraction module with a rich array of single-frame information, laying the groundwork for the subsequent recognition of actions.
(2) The sequence-level feature extraction module, a component that extracts features from an array of action frames. This is achieved via the long short-term memory (LSTM) network, or another neural network structure, apt for processing time series data. This module is chiefly tasked with capturing the time series features of action sequences, enabling the network to grasp the dynamic temporal changes of actions, and thereby accurately identifying various action sequences.
Let Z represent the sequence-level feature extraction module (i.e., the Set Pooling module), x the frame-level features, and π a permutation of the silhouettes of lower limb rehabilitation training action for any given sequence. Then there were:
$G=Z\left(\left\{x^q \mid q=1,2, \ldots, h\right\}\right)=Z\left(\left\{a^{\pi(q)} \mid q=1,2, \ldots, h\right\}\right)$ (1)
Figure 1. Network structure of the constructed model
(3) A multi-layer global pipeline module, designed to function as an integrative tool. This module combines frame-level and sequence-level features, thereby crafting a more comprehensive feature representation. Through this integration, higher-level abstract features are captured, enhancing the accuracy of action recognition.
(4) A horizontal pyramid mapping module, which partitions the feature map into diverse bar-shaped regions, subsequently extracting features within each region. This process allows for the simultaneous capture of both local and global action features, furnishing detailed action descriptions. This module proves instrumental in recognizing complex actions, particularly in the context of lower limb rehabilitation training actions.
The constructed model's network structure is depicted in Figure 1. The process of extracting features, which underpinned the recognition of lower limb rehabilitation training actions via the GaitSet algorithm, can be subdivided into the following steps:
· Data preparation: Videos of lower limb rehabilitation training actions were converted into consecutive frames, and the human silhouettes in each frame were isolated employing background subtraction.
· Frame-level feature extraction: The extracted silhouettes from each frame were fed into the CNN for the frame-level feature extraction. During this process, the CNN extracted the spatial features of silhouettes through multiple convolution, activation, and pooling operations.
· Sequence-level feature extraction: The frame-level features were subjected to a time series model to extract the time series features of action sequences.
· Feature fusion: The multi-layer global pipeline module fused frame-level and sequence-level features to construct a more comprehensive feature representation. As the GaitSet core, this step allowed the network to simultaneously capture spatial and temporal information of actions, thus improving the accuracy of action recognition.
· Feature mapping: The horizontal pyramid mapping module divided the fused feature map into different bar-shaped regions, subsequently extracting features in each region. This process enabled the network to simultaneously capture both local and global features of actions, thereby providing more detailed action descriptions.
· Action recognition: All extracted features were fed into a classifier for action recognition. During this process, the classifier differentiated between various action classes by calculating the distance between feature vectors.
In addition to the construction of the recognition model, this research also endeavored to generate an energy map of lower limb rehabilitation training actions. This energy map was aimed at not only capturing sequence features of the actions in a more comprehensive manner but also balancing computational costs and recognition performance effectively.
From a technical standpoint, the creation of the energy map primarily relied on the compression of silhouette pairs and the averaging of actions over an extended time range. In particular, the energy map compressed a series of consecutive action frames into a single image, a strategy designed to curtail computational complexity and conserve computational resources. Furthermore, the process of averaging action silhouettes over a long duration served to better eliminate any noise that might have remained from the preprocessing stage, thereby improving the accuracy of action recognition.
From a practical application perspective, it is noteworthy that the features of lower limb rehabilitation training actions often manifest across the entire sequence of actions, rather than within a single action frame. Let H denote the length of the lower limb rehabilitation training action sequence, j the time, and (c,i) the plane coordinates of the two-dimensional image. The energy map expression was thus given as:
$Z(c, i)=\frac{1}{H} \sum_{j=1}^H L_j(c, i)$ (2)
The updating of the triplet loss function plays a crucial role, as it maximizes the inter-class spacing while minimizing the intra-class spacing, thereby learning the optimal distance metric in the mapping space. This mechanism encourages samples from the same class to cluster in the feature space, while ensuring samples from different classes are clearly distinguished, thereby enhancing the recognition between sample classes.
However, the direct application of the triplet loss function when recognizing lower limb rehabilitation training actions may lead to uneven data distribution, which could result in some data being overly emphasized in training, while other data are overlooked. This imbalance may hinder network convergence, making the network prone to overfitting, and subsequently leading to poor performance on unseen data. Consequently, in this study, the loss function was constructed with a focus on the joint supervision training of both triplet and Softmax loss functions.
Let ABj be the loss, n be the anchor example, q be the positive example, h be the negative example, and H be the number of all examples in the Batch. The expression of the triplet loss function was given as follows:
$A B_j=\sum_j^H\left[y(n, q)^2-y(n, h)^2+b a r\right]_{+}$ (3)
Let ABm be the loss, Cj be the feature vector of the j-th lower limb rehabilitation training action image, ij be the true label of Cj, l be the bias, SJij and SJq be the weight vectors distinguishing Cj into ij class and q class, O be the total number of classes, SJijCj+lij be the scores of the lower limb rehabilitation training action images in class ij, and H be the total number of training samples. The expression of the Softmax loss function was given as follows:
$A B_m=-\frac{1}{H} \sum_j^H \log \frac{\exp \left(S_{i_j}^J C_j+l_{i_j}\right)}{\sum_q^O\left(S_{q}^J C_j+l_{q}\right)}$ (4)
Let β and α be the weights of the two loss functions Triplet and Softmax, then the final joint loss function was given as follows:
$L O S S=\beta A B_{\mathrm{j}}+\alpha A B_j$ (5)
When β was equal to 0 and α was not equal to 0, it was the Softmax loss function. When α was equal to 0 and β was not equal to 0, it was the triplet loss function. When neither β nor α was equal to 0, it was the joint loss function.
In utilizing image processing technology to recognize and measure the effects of lower limb rehabilitation training, certain specific indices are given particular attention. These include the range of motion for patients, gait analysis (such as step size, speed, and gait symmetry), and functional testing to evaluate a patient's performance in specific activities. It is vital to note that these indices must be used in conjunction with specific rehabilitation goals and the individual differences among patients. Factors such as the image processing technology and the models employed can influence the accuracy and reliability of these indices. Figure 2 exhibits a two-dimensional motion schematic diagram formed by the training actions.
Figure 2. Corresponding two-dimensional motion formed by training actions
This study recognizes the differential changes in the above indices using the full-field optical flow tracking method. This method provides a comprehensive acquisition of dynamic training information. By tracking the position changes of pixel grayscale in consecutive frames, the method captures the continuity and immediacy of lower limb rehabilitation training actions, thereby offering a wealth of motion data. Additionally, the method is highly sensitive to minor action changes. In the context of lower limb rehabilitation training, even small changes in movement can signify progress. Hence, the method's accurate recognition and tracking of these subtle movement changes can aid in evaluating the training's efficacy more accurately.
In essence, the full-field optical flow tracking method is predicated on two assumptions: the brightness of an object remains constant over time, and the object only undergoes minor motions. Based on these assumptions, the method can track the changes of pixel grayscale in continuous frame images, obtaining pixel displacement information. This process enables the recognition of the types of lower limb rehabilitation training actions and an assessment of those actions' quality and effectiveness. Moreover, the tracking method can be utilized for a detailed analysis of actions, such as identifying key points in actions and the motion paths of these key points.
If the lower limb rehabilitation training action image changed from frame a to e, the corresponding pixel b(c,i) in the image plane at time f changed to the pixel h(c+∆j, i+∆i) at time (j+∆j) after time period ∆j. Let K(c,i,j) and K(c+∆c,i+∆i, j+∆j) be the pixel grayscale at these two positions, respectively, and ∆M be the pixel displacement generated during this motion process. After decomposing ∆M into ∆c and ∆i, which obtained the optical flow components in the c and i directions, respectively, then there were:
$v_x=\frac{\Delta x}{\Delta t}, v_y=\frac{\Delta y}{\Delta t}$ (6)
$a_{\mathrm{c}}=\frac{\Delta c}{\Delta j}, a_y=\frac{\Delta i}{\Delta j}$ (7)
Based on the two assumptions, it was assumed that when pixel b moved to pixel h, its grayscale K was fixed. Given the grayscale of a pixel at any time, the classic equation of calculating the full-field optical flow tracking was as follows:
$K(c, i, j)=K(c+\Delta c, i+\Delta i, j+\Delta j)$ (8)
Let K/c, K/i, S/r, S/t, S/u and K/j be the partial derivatives of pixel grayscale K in c and i directions and time t, the above equation was expanded by Taylor series as follows:
$K(c, i, j)=K(c+\Delta c, i+\Delta i, j+\Delta j)=K(c, i, j)+\frac{\partial K}{\partial c} a_x+\frac{\partial K}{\partial i} a_y+\frac{\partial K}{\partial j} \Delta j+\mathrm{D}\left(\mathrm{y}j^2\right)$ (9)
With K/c, K/i, S/r, S/t, S/u and K/j being all known and D(yj2) being negligible, after eliminating S(o,p,q) at both ends of the equation, then there were:
$\frac{\partial K}{\partial c} \mathrm{a}_c+\frac{\partial K}{\partial i} a_i+\frac{\partial K}{\partial j} \Delta j=0$ (10)
For the discrete problem, let ∂K/∂j ∆j and ∂K/∂j ∆j be the grayscale variation at the same pixel position between any two lower limb rehabilitation training action images, then there were:
$\frac{\partial K}{\partial j} \Delta j=K(c, i, \Delta j)-K(c, i, j)$ (11)
The above two equations were combined, which obtained:
$\frac{\partial K}{\partial c} a_c+\frac{\partial K}{\partial i} a_i=K(c, i, j)-K(c+\Delta c, i+\Delta i, \Delta j)$ (12)
Compared with the pixels in the initial lower limb rehabilitation training action frame image, let |K0| be the image grayscale gradient, then the pixel displacement w(c,i,j) of corresponding pixels in any frame image was calculated by the following equation:
$W(c, i, j)=\frac{\left|K_0(c, i)-K(c, i, j)\right|}{\left|\nabla K_0\right|}$ (13)
$W(c, i, j)=\frac{\left|K_0(c, i)-K(c+\Delta c, i+\Delta i, j+\Delta j)\right|}{\left|\nabla K_0\right|} +\sqrt{\left(o_0\right)^2+\left(x_0\right)^2}$ (14)
|K0| was obtained by calculating the following equation:
$\left|\nabla K_0\right|=\sqrt{\left(\frac{\partial K}{\partial c}\right)^2+\left(\frac{\partial K}{\partial i}\right)^2}$ (15)
The above three equations were combined, which obtained:
$\frac{\left|K_0(c, i)-K(c, i, t)\right|}{\sqrt{\left(\frac{\partial K}{\partial c}\right)^2+\left(\frac{\partial K}{\partial i}\right)^2}} =\frac{\left|K_0(c, i)-K(c+\Delta c, i+\Delta i, j+\Delta t)\right|}{\sqrt{\left(\frac{\partial K}{\partial c}\right)^2+\left(\frac{\partial K}{\partial i}\right)^2}}+\sqrt{\left(o_0\right)^2+\left(x_0\right)^2}$
Let round(*) be the floor and ceiling functions, then the optical flow component of the integer value was further solved based on the following equation:
$\Delta w=\sqrt{\left[ { round }\left(\frac{\frac{\partial K}{\partial c}}{\left(\frac{\partial K}{\partial c}\right)^2+\left(\frac{\partial K}{\partial i}\right)^2}\right)\right]^2+\left[ { round }\left(\frac{\frac{\partial K}{\partial c}}{\left(\frac{\partial K}{\partial c}\right)^2+\left(\frac{\partial K}{\partial i}\right)^2}\right)\right]^2} \cdot \Delta j$
Then the obtained pixel displacement of lower limb rehabilitation training actions was converted into the spatial real displacement, which recognized the differential changes of indexes. Figure 3 shows a schematic diagram of the spatial real displacement conversion.
Figure 3. Schematic diagram of spatial real displacement conversion
The data presented in Table 1 facilitated a comparative analysis of the performance of different methodologies for the recognition of various lower limb rehabilitation training actions. It was observed that all methodologies demonstrated notable accuracy in recognizing the action of traversing stairs, attributed to its relatively fixed action pattern and distinct features. Within this context, the method proposed in this study along with the pose estimation method stood out, particularly the former, which achieved an impressive accuracy of 97.42%.
Actions demanding a high level of balance, such as walking with crutches, propelling a wheelchair, and maintaining single-leg balance, were also recognized with remarkable accuracy by the proposed method. This accuracy was notably higher than those achieved by the other four methods. For the actions of walking independently, transitioning from sitting to standing, and squatting to standing, both the skeleton model recognition method and the proposed method delivered commendable results. In contrast, the recurrent neural network method based on time-series data and the pose estimation method displayed lower accuracy.
Overall, the proposed method exhibited superior performance in recognizing all actions, accurately identifying the action modes in both simple (e.g., independent walking) and complex actions requiring high coordination and balance (e.g., walking with crutches and single-leg balance). This demonstrates its robust capability in recognizing lower limb rehabilitation training actions.
An inspection of the data in Table 2 revealed a comparative analysis of the parameters' count, computational load, and accuracy of various methods. The method proposed in this study required the fewest parameters, only 0.9586M, indicating a simpler model complexity and an easier training and optimization process. Contrarily, the pose estimation method required the most parameters, reaching 162.5174M, which led to a more complex training process and a higher demand for computational resources.
The computational load of the proposed method was 169.5284M, significantly advantageous over the pose estimation method and the recurrent neural network method based on time-series data, whose computational loads were 13062.4172M and 751.6285M, respectively. This suggests that the proposed method excels in operational efficiency and response speed. Furthermore, the proposed method showcased high accuracy levels in both overall and average class accuracy, reaching 88.24% and 79.42%, respectively.
In conclusion, the proposed method demonstrated superior performance in terms of parameters' count, computational load, and accuracy. Especially in computational efficiency, the method showed promise for real-time performance and faster response speed in practical applications. The high accuracy also ensured its recognition reliability. Therefore, it is inferred that the proposed method has substantial application value and research potential in recognizing lower limb rehabilitation training actions.
The image processing technology proposed in this study was employed to quantify the muscle strength output value, thereby assessing the effectiveness of lower limb rehabilitation training. The full-field optical flow tracking method was applied initially to track pixel grayscale position changes of moving objects across different frame images, resulting in pixel displacement data. The analysis of this displacement data yielded information on relative joint motion and velocity and acceleration of various body parts.
These data points were then converted into real-world motion through suitable transformations. A biomechanical model was subsequently applied to estimate muscle strength. This model, grounded in human skeletal and muscular structures, typically employs joint angles and velocities, along with other physiological parameters such as muscle length and velocity, to estimate muscle strength.
Inspection of Figure 4 reveals that the muscle strength output values of patients of varying heights increase in tandem with the duration of rehabilitation training. This trend suggests a gradual recovery of muscle strength as rehabilitation training progresses. Furthermore, at any given time point, the muscle strength output value of patients with a height of 180cm was marginally superior to that of patients with a height of 170cm. This difference is attributed to factors such as body size and muscle mass. It is understood that taller patients, due to their larger muscle mass, demonstrate higher muscle strength output values during rehabilitation training. However, this does not necessarily imply superior recovery in taller patients, as their muscle strength output standards are inherently higher.
In light of this understanding, it is crucial to consider the relative improvement in muscle strength output value, rather than the absolute values, when assessing recovery progress. A more comprehensive study could also explore the influence of other factors, such as age, gender, and pre-existing health conditions, on muscle strength output during lower limb rehabilitation training. This could contribute to a more nuanced understanding of the effectiveness of rehabilitation training and might guide personalized training programs for optimal results.
Examination of the data in Figure 5 reveals an increasing trend in muscle strength output values for patients with body weights of both 70kg and 90kg, concomitant with the progression of rehabilitation training time. This trend substantiates the positive impact of rehabilitation training on muscle strength enhancement.
For any given time point, it is evident that the muscle output value for patients weighing 90kg marginally surpasses that of patients weighing 70kg. This observation is attributed to the greater muscle mass associated with heavier patients, which results in higher muscle strength output values during the course of rehabilitation training.
Nevertheless, this should not be misconstrued to imply superior recovery in heavier patients, as their need for higher muscle strength output standards to support their weight must be factored in.
Table 1. Comparison of action recognition accuracy using different methods
|
Methods |
Walking Independently |
Walking with Crutches |
Movement Using a Wheelchair |
Walking up and down Stairs |
Sitting-Standing Conversion |
Squatting-Standing Action |
Single-Leg Balance |
|
Recurrent neural network method based on time series data |
74.15 |
51.62 |
54.28 |
84.51 |
71.35 |
75.18 |
71.95 |
|
Pose estimation method |
70.35 |
53.98 |
61.47 |
96.25 |
84.15 |
83.51 |
82.19 |
|
Skeleton model recognition method |
86.59 |
57.42 |
63.58 |
81.74 |
70.69 |
81.69 |
83.61 |
|
Multi-modal fusion recognition method |
71.24 |
61.35 |
60.47 |
92.57 |
81.42 |
80.27 |
70.39 |
|
Method proposed in this study |
87.62 |
78.47 |
75.14 |
97.42 |
85.65 |
86.51 |
88.24 |
Table 2. Computational efficiency comparison using different methods
|
Methods |
Number of Parameters |
Calculation Amount |
Overall Accuracy (%) |
Average Class Accuracy (%) |
|
Recurrent neural network method based on time series data |
62.5841M |
751.6285M |
62.38 |
52.19 |
|
Pose estimation method |
162.5174M |
13062.4172M |
71.24 |
50.84 |
|
Skeleton model recognition method |
13.2659M |
1612.3528M |
83.69 |
70.61 |
|
Multi-modal fusion recognition method |
25.4182M |
362.5174M |
81.27 |
74.15 |
|
Method proposed in this study |
0.9586M |
169.5284M |
88.24 |
79.42 |
Figure 4. Muscle strength output during lower limb rehabilitation training under different heights
Figure 5. Muscle strength output during lower limb rehabilitation training under different body weights
Figure 6. Impedance variation curves of different patients along with knee joint angles under different impedance training schemes
Insight from Figure 6's rehabilitation training data, derived from three distinct patients, reveals that as the knee joint angle transitions from 60° to 180°, the impedance values for all patients incrementally rise. This trend suggests that with an increase in the knee joint angle, the impedance force that needs to be overcome during rehabilitation training intensifies. This is because a larger angle necessitates greater muscle strength for maintenance, thereby augmenting the impedance.
Upon individual patient analysis, it is observed that Patient 3 experiences the most rapid increase in impedance, while Patient 1 reflects the slowest rise. These differences can be attributed to their respective physical conditions and recovery progress. Patient 3, possessing stronger muscle strength, exhibits a higher impedance as the angle increases. In contrast, Patient 1, with weaker muscle strength, shows a slower impedance increase. This underscores the need for personalization in rehabilitation training, with appropriate training intensity and impedance being selected based on individual physical conditions and rehabilitation needs. Such data provide valuable insights for physicians or rehabilitation therapists in developing tailored rehabilitation training plans.
Analysis of the data in Table 3 demonstrates that the proposed method outperforms those based on both optical flow features and time series analysis across most angles in recognizing key indices, such as range of motion, gait, and functional testing. In recognizing the range of motion index, the proposed method achieves a recognition rate between 86.9% and 96.5% at various angles, superior to the other two methods. This suggests that the proposed method yields more accurate results in recognizing range of motion.
In the recognition of the gait index, the proposed method exhibits superior performance at most angles compared to the other two methods, particularly with recognition rates of 94.6%, 96.8%, and 93.6% at the angles of 108˚, 126˚, and 144˚, respectively. These rates significantly exceed those achieved by the other two methods.
The proposed method also excels in recognizing the functional testing index, specifically achieving recognition rates of 92.6% and 90.4% at angles of 18˚ and 36˚, respectively. These rates notably surpass those of the other two methods.
In summary, the proposed method demonstrates superior performance compared to the other two methods in recognizing range of motion, gait, and functional testing. This suggests that the proposed method is highly accurate and stable in recognizing lower limb rehabilitation training action standards at multiple angles, thereby supporting physicians and patients in evaluating the rehabilitation training effect with greater precision in practical clinical applications. This could facilitate the development of more suitable rehabilitation plans.
Table 3. Experimental result comparison of lower limb rehabilitation training action standard recognition at multiple angles
|
Actions |
Methods |
0° |
18° |
36° |
54° |
72° |
90° |
108° |
126° |
144° |
162° |
180° |
|
Range of motion |
Method proposed in this study |
91.5 |
92.5 |
91.6 |
93.5 |
95.8 |
92.6 |
94.1 |
94.7 |
96.5 |
91.5 |
86.9 |
|
Method based on optical flow features |
85.7 |
90.3 |
87.7 |
90.6 |
91.7 |
90.4 |
83.2 |
89.6 |
91.8 |
87.4 |
83.1 |
|
|
Method based on time series analysis |
81.4 |
90.6 |
87.8 |
90.5 |
93.8 |
85.8 |
87.5 |
92.5 |
84.7 |
81.8 |
84.7 |
|
|
Gait |
Method proposed in this study |
86.2 |
90.4 |
93.1 |
91.8 |
85.1 |
83.6 |
81.3 |
94.6 |
96.8 |
93.6 |
92.9 |
|
Method based on optical flow features |
81.7 |
87.6 |
84.8 |
83.4 |
80.6 |
81.4 |
76.9 |
91.8 |
92.5 |
91.5 |
85.4 |
|
|
Method based on time series analysis |
80.6 |
83.4 |
88.6 |
81.8 |
80.5 |
80.5 |
74.7 |
83.5 |
91.1 |
84.8 |
81.6 |
|
|
Functional testing |
Method proposed in this study |
73.5 |
92.6 |
90.4 |
85.4 |
83.1 |
81.4 |
84.2 |
85.4 |
82.4 |
81.5 |
72.6 |
|
Method based on optical flow features |
68.2 |
85.3 |
81.6 |
71.6 |
74.9 |
73.6 |
76.9 |
71.6 |
70.3 |
74.5 |
69.3 |
|
|
Method based on time series analysis |
61.4 |
87.1 |
87.9 |
73.8 |
71.6 |
79.5 |
73.1 |
79.7 |
76.4 |
78.3 |
64.8 |
The present research focused on the identification and analysis of lower limb rehabilitation training actions, spanning the entire process from pixel displacement calculations of lower limb movements to the determination of muscle strength output values based on these calculations. The research also delved into a profound exploration and analysis of these aspects. The experimental results led to the following comprehensive conclusions:
(1) In the context of pixel displacement calculations of lower limb movements, the algorithm introduced in this study accurately determined the displacement of the actions, providing a precise foundation for the computation of muscle strength output.
(2) The method introduced in this study not only precisely calculated muscle strength output values, but also accounted for factors such as height and weight, leading to results that were more accurate and personalized.
(3) When considering action recognition, specifically the recognition of range of motion, gait, and functional testing, the introduced method demonstrated a significantly superior performance compared to other methods. This highlights the high accuracy and stability of the proposed method.
(4) The calculation load and the number of parameters required by the proposed method were substantially lower than those required by other methods, making the proposed method highly efficient and practical for use.
In summary, the method studied in this research for the recognition and analysis of lower limb rehabilitation training actions is highly accurate, stable, and efficient, with a low calculation load and fewer parameters. This method holds noteworthy clinical application value. Future work could focus on further optimization of the proposed method to enhance its utility in lower limb rehabilitation training. For instance, incorporating more personalized factors could make the recognition and analysis of lower limb rehabilitation training actions more accurate and personalized.
This paper was supported by Science and Technology Department project of Jilin Province (Grant No: 20230101233JC) and Education Department project of Jilin Province (Grant No.: JJKH20200572KJ).
[1] Tsai, T.C., Chiang, M.H. (2023). A lower limb rehabilitation assistance training robot system driven by an innovative pneumatic artificial muscle system. Soft Robotics, 10(1): 1-16. https://doi.org/10.1089/soro.2020.0216
[2] Fang, Q., Xu, T., Zheng, T., Cai, H., Zhao, J., Zhu, Y. (2022). A rehabilitation training interactive method for lower limb exoskeleton robot. Mathematical Problems in Engineering, 2022: 2429832. https://doi.org/10.1155/2022/2429832
[3] Shang, Z., Yan, H., Wang, T. (2022). The development of assistive device for active lower limb rehabilitation training. In 2nd International Conference on Mechanical, Electronics, and Electrical and Automation Control (METMS 2022), Guilin, China, pp. 284-290. https://doi.org/10.1117/12.2635134
[4] Yuan, J., Zheng, Z., Bao, S., Du, L., Zhou, T. (2022). Upper and lower limb linkage design and training trajectory planning of rehabilitation robot. In 2022 IEEE International Conference on Real-time Computing and Robotics (RCAR), Guiyang, China, pp. 310-315. https://doi.org/10.1109/RCAR54675.2022.9872195
[5] Yin, Q., Hu, A., Li, Q., Wei, X., Yang, H., Wang, B., Zhang, G. (2021). Compound lower limb vibration training rehabilitation robot. Concurrency and Computation: Practice and Experience, 33(6): e6059. https://doi.org/10.1002/cpe.6059
[6] Nan, Q. (2021). Design of sitting lower limb rehabilitation training robot. Journal of Physics: Conference Series, 1939(1): 012006. https://doi.org/10.1088/1742-6596/1939/1/012006
[7] Luo, C., Wei, J., Liu, Q., et al. (2021). Development of a new lower limb rehabilitation robot for bedside training. Journal of Physics: Conference Series, 2026(1): 012060. https://doi.org/10.1088/1742-6596/2026/1/012060
[8] Guo, J., Zhao, K., Guo, S. (2021). Design of the lower limb rehabilitation training system based on virtual reality. In 2021 IEEE International Conference on Mechatronics and Automation (ICMA), Takamatsu, Japan, pp. 1304-1309. https://doi.org/10.1109/ICMA52036.2021.9512760
[9] Cha, B., Lee, K.H., Ryu, J. (2021). Deep-learning-based emergency stop prediction for robotic lower-limb rehabilitation training systems. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 29: 1120-1128. https://doi.org/10.1109/TNSRE.2021.3087725
[10] Zhou, T., Yuan, J., Ma, S., et al. (2021). Integration mode of standing balance and trajectory training based on lower limb rehabilitation exoskeleton. In 2021 IEEE International Conference on Robotics and Biomimetics (ROBIO), Sanya, China, pp. 1252-1257. https://doi.org/10.1109/ROBIO54168.2021.9739337
[11] Ma, H. (2021). Research on promotion of lower limb movement function recovery after stroke by using lower limb rehabilitation robot in combination with constant velocity muscle strength training. In 2021 7th International Symposium on Mechatronics and Industrial Informatics (ISMII), Zhuhai, China, pp. 70-73. https://doi.org/10.1109/ISMII52409.2021.00022
[12] Ren, L.Y., Wang, C., Feng, P. (2021). Research on key techniques of lower limb rehabilitation training based on human surface EMG signal. In Proceedings of the 2021 10th International Conference on Bioinformatics and Biomedical Science, Xiamen, China, pp. 95-99. https://doi.org/10.1145/3498731.3498745
[13] Wang, Y.L., Wang, K.Y., Wang, W.L., Han, Z., Zhang, Z.X. (2021). Appraisement and analysis of dynamical stability of under-constrained cable-driven lower-limb rehabilitation training robot. Robotica, 39(6): 1023-1036. https://doi.org/10.1017/S0263574720000879
[14] Zhao, C., Zhang, Q., Li, Y., Yang, J., Sun, B., Wang, Y. (2020). Rehabilitation training analysis based on human lower limb muscle model. In Robotics and Rehabilitation Intelligence: First International Conference, ICRRI 2020, Fushun, China, pp. 389-400. https://doi.org/10.1007/978-981-33-4932-2_28
[15] Masengo, G., Zhang, X., Yin, G., Alhassan, A.B., Dong, R., Orban, M., Mudaheranwa, E. (2020). A design of lower limb rehabilitation robot and its control for passive training. In 2020 10th Institute of Electrical and Electronics Engineers International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), Xi'an, China, pp. 152-157. https://doi.org/10.1109/CYBER50695.2020.9278952
[16] Wang, Y.L., Wang, K.Y., Wang, W.L., Yin, P.C., Han, Z. (2019). Appraise and analysis of dynamical stability of cable-driven lower limb rehabilitation training robot. Journal of Mechanical Science and Technology, 33: 5461-5472. https://doi.org/10.1007/s12206-019-1040-4
[17] Wang, C., Xia, J., Wei, J., et al. (2019). Multi-Arm lower-limb rehabilitation robot for motor coordination training after stroke. The Journal of Engineering, 2019(14): 478-484. https://doi.org/10.1049/joe.2018.9403
[18] Chen, L., Lu, C. (2022). Image enhancement algorithm-based ultrasound on pelvic floor rehabilitation training in preventing postpartum female pelvic floor dysfunction. Computational and Mathematical Methods in Medicine, 2022: 8002055. https://doi.org/10.1155/2022/8002055
[19] Wang, M., Li, C., Zhang, W., Chen, R., Li, X. (2023). Study on brain mechanism of rehabilitation training of articulation disorder in cleft lip and palate patients based on functional magnetic resonance imaging. Journal of Biomedical Engineering, 40(1): 125-132. https://doi.org/10.7507/1001-5515.202006068
[20] Yin, P., Wang, H. (2022). Evaluation of nursing effect of pelvic floor rehabilitation training on pelvic organ prolapse in postpartum pregnant women under ultrasound imaging with artificial intelligence algorithm. Computational and Mathematical Methods in Medicine, 2022: 1786994. https://doi.org/10.1155/2022/1786994
[21] Zhang, X., Wang, Z., Liu, J., Bi, L., Yan, W., Yan, Y. (2021). Value of rehabilitation training for children with cerebral palsy diagnosed and analyzed by computed tomography imaging information features under deep learning. Journal of Healthcare Engineering, 2021: 6472440. https://doi.org/10.1155/2021/6472440
[22] Hong, S., Hong, S., Jang, J., Kim, K., Hyung, W.J., Choi, M.K. (2023). Amplifying action-context greater: Image segmentation-guided intraoperative active bleeding detection. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 11(4): 1261-1270. https://doi.org/10.1080/21681163.2022.2159533
[23] Shi, Y. (2023). Image recognition of skeletal action for online physical education class based on convolutional neural network. IEIE Transactions on Smart Processing & Computing, 12(1): 55-63. https://doi.org/10.5573/IEIESPC.2023.12.1.55
[24] Zhang, C., Wang, M., Zhou, L. (2023). Recognition method of basketball players' throwing action based on image segmentation. International Journal of Biometrics, 15(2): 121-133. https://doi.org/10.1504/IJBM.2023.129216
[25] Ashrafi, S.S., Shokouhi, S.B., Ayatollahi, A. (2023). Still image action recognition based on interactions between joints and objects. Multimedia Tools and Applications, 82(17): 25945-25971. https://doi.org/10.1007/s11042-023-14350-z
