Upper Limb Action Identification Based on Physiological Signals and Its Application in Limb Rehabilitation Training

Upper Limb Action Identification Based on Physiological Signals and Its Application in Limb Rehabilitation Training

Chao ZhangJi Zou Zhongjing MaQian Wu Zhaogang Sheng Zhen Yan 

School of Automation, Beijing Institute of Technology, Beijing 100081, China

School of Electronic Information Engineering, Changchun University, Changchun 130022, China

Corresponding Author Email: 
mazhongjing@bit.edu.cn
Page: 
1887-1894
|
DOI: 
https://doi.org/10.18280/ts.380633
Received: 
19 September 2021
|
Accepted: 
2 December 2021
|
Published: 
31 December 2021
| Citation

OPEN ACCESS

Abstract: 

Upper limb motor dysfunction brings huge pain and burden to patients with brain trauma, stroke, and cerebral palsy, as well as their relatives. Physiological signals are closely related to the recovery of patients with limb dysfunction. The joint analysis of two key physiological signals, namely, surface electromyographic (sEMG) signal and acceleration signal, enables the scientific and effective evaluation of upper limb rehabilitation. However, the existing indices of upper limb rehabilitation are incomplete, and the current evaluation approaches are not sufficiently objective or quantifiable. To solve the problems, this paper explores upper limb action identification based on physiological signals, and tries to apply the approach to limb rehabilitation training. Specifically, the upper limb action features during limb rehabilitation training were extracted and identified by time-domain feature method, frequency-domain feature method, time-frequency domain feature method, and entropy feature method. Then, the evaluation flow of upper limb rehabilitation, plus the relevant evaluation indices, were given. Experimental results demonstrate the effectiveness of the proposed composite feature identification of upper limb actions, and the proposed evaluation method for limb rehabilitation.

Keywords: 

physiological signals, upper limb motor function, upper limb action identification, limb rehabilitation

1. Introduction

Brain trauma, stroke, and cerebral palsy are the main causes of upper limb dysfunction, which brings huge pain and burden to patients and their relatives [1-6]. Upper limb movement is closely related to daily life. One of the important research directions of upper limb rehabilitation is whether patients with upper limb dysfunction can gradually regain normal upper limb motor function through rehabilitation training [7-12]. To a certain extent, the internal activity and state of muscles can be reflected by the surface electromyographic (sEMG) signal, a bioelectrical signal transmitted between surface muscles and the surrounding neurons [13-17]. Meanwhile, the inertial information of muscle movement is contained in the acceleration signal. These two types of physiological signals are closely associated with the rehabilitation of patients with limb dysfunction. The joint analysis of these two key physiological signals enables the scientific and effective evaluation of upper limb rehabilitation [18-21].

The motion capture system is considered an increasingly interesting tool for rehabilitation evaluation. Focusing on the evaluation of injured shoulder rehabilitation, Vitali et al. [22] combined the low-cost Mocap system with video processing technology to quantify the outreach. Their approach overcomes the difficulty in capturing the extremely complex shoulder actions. Guo et al. [23] proposes a collection method for sEMG signal and interactive force signal in the rehabilitation training process. The signals were collected by driving specific actions with external forces. In addition, Guo fitted a fuzzy neural network, and verified its prediction ability during rehabilitation training. Pundik et al. [24] analyzed and studied the response of traumatic brain injury (TBI) patients to a new therapy with an electromyogram orthosis. Their study includes baseline data collection / equipment manufacturing (5 weeks), clinical training (9 weeks), and family utilization (9 weeks). During the experiments, the orthosis was introduced to the treatment based on exercise learning. The experimental results demonstrate that the patients saw improvements in action range, tension, muscle strength, Fugl-Meyer, square test result, and the Chedoke evaluation score during clinical training. Rehabilitation training with upper limb exoskeleton robots has been proven to be a scientific and effective strategy. Based on surface electromyography, heart rate variability, and the integrated instantaneous heart rate, Wang et al. [25] put forward a fatigue judgement method fusing multiple information, developed a bioelectrical signal collection system for upper limb exoskeleton robots, and completed the acquisition and processing of both EMG signal and electrocardiosignal. Guo et al. [26] adopted a depth model to process the original surface electromyographic signal, examined the feasibility of the recognition of upper limb motion intentions and the real-time control of the auxiliary equipment for upper limb rehabilitation training. On this basis, they designed a lightweight convolutional neural network (CNN) and support vector machine (SVM) for EMG signal identification, and compared the offline and online performance of the two models.

The existing indices of upper limb rehabilitation are incomplete, and the current evaluation approaches are not sufficiently objective or quantifiable. Failing to effectively identify upper limb actions, the current approaches cannot fully reflect the execution of actions in upper limb rehabilitation training, or the rehabilitation state of upper limb muscles, not to mention the tracking of long-term training effect. To solve the problems, this paper explores upper limb action identification based on physiological signals, and applies the approach to limb rehabilitation training. The main contents are as follows: (1) The upper limb action features during limb rehabilitation training were extracted and identified by time-domain feature method, frequency-domain feature method, time-frequency domain feature method, and entropy feature method; the specific steps of the feature extraction algorithm were explained. (2) The evaluation flow of upper limb rehabilitation was clarified, the limb rehabilitation training effect was summarized as three aspects: motor function rehabilitation state, balance function rehabilitation state, and coordination function rehabilitation state, and the specific evaluation indices were enumerated. (3) The relative activation degree and activation time of the muscles were analyzed through rehabilitation training; the proposed composite feature identification of upper limb actions, and limb rehabilitation evaluation approach were validated through experiments.

2. Upper Limb Feature Extraction Based on Physiological Signals

Traditionally, the sEMG signal and acceleration signal from the upper limbs, including shoulder muscles, arm muscles, forward arm muscles and hand muscles, are acquired through segmentation. Despite reflecting the difference between dynamic upper limb actions, the traditional method cannot be directly applied to upper limb action recognizers to classify upper limb actions. Therefore, it is necessary to extract effective upper limb action features from physiological signals of different segments. In this paper, the upper limb actions in limb rehabilitation training are identified by time-domain feature method, frequency-domain feature method, time-frequency domain feature method, and entropy feature method.

2.1 Time-frequency domain feature extraction

The mathematical statistical method was adopted to process the time variation of the waveforms of the collected sEMG signal and acceleration signal. The acquired parameter features of the signals are the time-domain features of upper limb actions. Let am={a1,a2,...,aM} be the time series of physiological signal samples collected from a segment, with M being the signal length. The degree of oscillation of the collected physiological signals was characterized by the number NZERO of zero points being passed, i.e., the number of times that the acquired signal curve passes through the time axis:

$\begin{aligned}

&N_{\text {ZERO }}=\sum_{m=2}^{M} \operatorname{sgn}\left(-a_{m} a_{m-1}\right) \\

&\operatorname{sgn}(a)=\left\{\begin{array}{l}

1, \text { if } a \geq 0 \\

0, \text { otherwise }

\end{array}\right.

\end{aligned}$        (1)

Then, the mean absolute value of the physiological signal samples from a segment:

${{V}_{ABS}}=\frac{1}{M}\sum\limits_{m=1}^{M}{\left| {{a}_{m}} \right|}$       (2)

To reflect the energy variation of the collected physiological signals between segments, the mean absolute value was added by a weight θ to obtain the weighted mean absolute value:

$\begin{aligned}

&V_{W A B S}=\frac{1}{M} \sum_{m=2}^{M} \theta\left|a_{m}\right| \\

&\theta= \begin{cases}1, \text { if } & 0.25 M \leq m \leq 0.75 M \\

0.5, \text { otherwise } & \end{cases}

\end{aligned}$       (3)

The root mean square (RMS) can be calculated by:

${{V}_{ROOT}}=\sqrt{\frac{1}{M}\sum\limits_{m=1}^{M}{a_{m}^{2}}}$       (4)

In order to characterize the waveform complexity of the collected physiological signals, the absolute values of the difference between the adjacent sample points were accumulated, and the cumulative sum was defined as the waveform length of the collected physiological signals:

${{V}_{WAVE}}=\sum\limits_{m=1}^{M-1}{\left| {{a}_{m+1}}-{{a}_{m}} \right|}$        (5)

To prevent the physiological signal length from affecting the identification result, this paper introduces the mean waveform length feature to reflect the change law of the collected physiological signals:

${{V}_{AW}}=\frac{1}{M}\sum\limits_{m=1}^{M-1}{\left| {{a}_{m=\text{1}}}-{{a}_{m}} \right|}$       (6)

The EMG signal is the superposition between the action potentials of the motor units in various muscle fibers. The level of the action potentials can be characterized by Wilson amplitude, i.e., the number of amplitude differences between two adjacent sample points being greater than the preset threshold FZ. This parameter was introduced to characterize the contraction degree of upper limb muscles:

$\begin{aligned}

&V_{W I L}=\sum_{m=1}^{M-1} g\left(\left|a_{m+1}-a_{m}\right|\right) \\

&g(a)= \begin{cases}1, \text { if } & a \geq F Z \\

0, \text { otherwise } & \end{cases}

\end{aligned}$       (7)

The collected physiological signals were subjected to Fourier analysis, and the power spectrum analysis was conducted to extract frequency domain features. The extracted features can reveal the distribution and variation of the frequency bands of the collected physiological signals. Let γj and ηj be the frequency and spectrum intensity of the collected physiological signals, respectively; N be the length of the power spectrum. Then, the median frequency, mean frequency, and peak frequency of the collected physiological signals can be respectively calculated by:

${{F}_{M}}=\sum\limits_{j={{F}_{M}}}^{N}{{{\eta }_{j}}=\frac{1}{2}}\sum\limits_{j=1}^{N}{{{\eta }_{j}}}$       (8)

${{F}_{AV}}=\sum\limits_{j=1}^{N}{{{\gamma }_{j}}{{\eta }_{j}}/}\sum\limits_{j=1}^{N}{{{\eta }_{j}}}$       (9)

${{F}_{PE}}=max\left( {{\eta }_{j}} \right),j=1,....,N$       (10)

The wavelet coefficient set, which contains rich information, can be obtained through wavelet decomposition of different mother wavelets. Let ξ be the decomposition coefficient for the j-th layer component obtained through wavelet decomposition; Qj,l be the l-th coefficient on the corresponding layer. Then, the wavelet coefficient energy can be taken as the time-frequency domain feature of the collected physiological signals:

$TF{{C}_{j}}=\sqrt{\frac{1}{\xi }\sum\limits_{\xi =1}^{\xi }{Q_{j,\xi }^{2}}}$       (11)

This paper selects db6 wavelet to perform three-layer wavelet decomposition of the collected physiological signals. The TFCj of each layer was solved, and taken as the time-frequency feature of that layer for analysis.

2.2 Multiscale entropy feature extraction

Approximate entropy, permutation entropy, fuzzy entropy, and sample entropy, as measures of the orderliness of the target system, are widely adopted to analyze physiological signals like EMG signal and electroencephalogram (EEG) signal. The fuzzy entropy FE can measure the fuzziness of the fuzzy domain. Let n be the dimensionality of the collected physiological signals; jSEMij be the similarity between two samples; s be the mean similarity of SEMij. Then, FE can be calculated by:

$\begin{aligned}

&\operatorname{FE}(M, n, s)=\lim _{M \rightarrow \infty}\left(\ln \Phi_{n}-\Phi_{n+1}\right) \\

&\Phi_{n}=\frac{1}{(M-n)} \sum_{i=1}^{M-n}\left[\frac{\sum_{j=1, i \neq i}^{M-n} S E M_{i j}^{n}}{(M-n-1)}\right]

\end{aligned}$       (12)

Approximate entropy is often used to quantify the regularity and unpredictability of the fluctuations of the time series of signals. For a signal time series, the approximate entropy is positively correlated with its complexity. Similar to approximate entropy, sample entropy can measure the complexity of signal time series. But the latter boasts two advantages: independence of data length, and high consistency. The sample entropy is under the balanced impacts from the variation in the dimensionality n of the collected physiological signals, and the variation in the mean similarity s of SEMij. The sample entropy can be calculated by the following steps:

Step 1. Construct a n-dimensional vector of the collected physiological signals in turn:

${{A}_{n}}\left( i \right)=\left[ a\left( i \right),a\left( i+1 \right),...,a\left( i+n-1 \right) \right]$       (13)

where, i=1,2,...,M-n.

Step 2. Define the distance between vectors An(i) and An(j) as the largest difference between the corresponding elements:

$u\left[ {{A}_{n}}\left( i \right),{{A}_{n}}\left( j \right) \right]=\underset{l=0,1,...,n-1}{\mathop{max}}\,\left| a\left( i+l \right)-a\left( j+l \right) \right|$       (14)

where, i, j=1,2,...,M-n, and ij.

Step 3. Count the number Xni(s) of u[An(i),An(j)] smaller than the mean similarity s, and compute the ratio Yni(s) of Xni(s) to the total number of distances M-n+1:

$Y_{i}^{n}\left( s \right)=\frac{X_{i}^{n}\left( s \right)}{N-n+1}$        (15)

Step 4. Compute the mean Yn(s) of Yin(s):

${{Y}^{n}}\left( s \right)=\frac{1}{M-n}\sum\nolimits_{i=1}^{M-n}{Y_{i}^{n}\left( s \right)}$       (16)

Step 5. Construct a n+1-dimensional vector of the collected physiological signals in turn, and repeat Steps 1-4 to obtain Yn+1(s).

Step 6. Compute the sample entropy of the physiological signal sample series:

$SE\left( n,s \right)=-\underset{M\to \infty }{\mathop{lim}}\,\frac{{{Y}^{n+1}}\left( s \right)}{{{Y}^{n}}\left( s \right)}$       (17)

Step 7. Due to the limited length of the collected physiological signals, denote the natural logarithm as ln, and estimate the sample entropy of the series of M samples, before applying the method to upper limb action identification:

$SE\left( n,s,M \right)=-ln\frac{{{Y}^{n+1}}\left( s \right)}{{{Y}^{n}}\left( s \right)}$       (18)

Any of the above entropy metrics, whether approximate entropy, permutation entropy, fuzzy entropy, or sample entropy, can quantify the regularity and complexity of signal time series. In real-world limb rehabilitation training, however, the physiological signal time series generated by the complex physiological system are extremely complex in time and space dimensions. The upper limb actions may be recognized insufficiently with only one metric. Hence, this paper adopts multiscale entropy to identify the upper limb actions in limb rehabilitation training.

Multiscale entropy is essentially derived from the sample entropies of signal time series on different scales. The derivation process is relatively simple. But multiscale entropy does not apply to the analysis of non-stationary nonlinear signals, and could not reflect the low-frequency components in the collected physiological signals. Therefore, empirical modal decomposition (EMD) was carried out to extract the sample entropy features of the collected physiological signals. Specifically, EMD was implemented to extract the intrinsic mode functions (IMFs) from the signal time series. Then, the sample entropy was calculated from the cumulative sum of the IMFs, and taken as the feature of the signal time series. This strategy fully retains the rich information in the various frequency components of the signal time series, and remains robust to the variation in low-frequency components, providing an efficient feature for identifying upper limb actions. The EMD process is as follows:

Step 1. Extract the maximum and minimum points of the original signal time series a(p).

Step 2. Fit the upper envelope BHmax(p) between maximum points and the lower envelope BHmin(p) between minimum points of the original signal time series, using the third-order spline difference function.

Step 3. Compute the mean n(p)=BHmin(p)+ BHmax(p))/2 of BHmax(p) and BHmin(p).

Step 4. Extract the details of z(p)=a(p)-n(p) the signal time series.

Step 5. Judge whether z(p) satisfies the two conditions of IMFs. If yes, treat z(p) as a component of IMF; otherwise, repeat Steps 1-4 on z(p) until it satisfies the two conditions.

Step 6. Subtract the extracted IMF component from a(p). Repeat Steps 1-5 to obtain the next IMF, until all IMFs are obtained. Define the remaining monotonic function with no extreme as the residual CA. Let C be the number of IMFs Ω. Then, the final decomposition of the EMD can be expressed as:

$a\left( p \right)=\sum\limits_{i=1}^{C}{{{\Omega }_{i}}\left( p \right)}+CA$      (19)

The EMD-based sample entropy feature can be derived from the cumulative sum of the IMFs obtained through the EMD. Starting from the first IMF, the smallest scale, to the end of the complete physiological signals, the total cumulative number QSk(t) of IMFs can be expressed as:

$QS_{{}}^{c}=\sum\nolimits_{i=1}^{c}{{{\Omega }_{i}}\left( p \right)}$        (20)

Taking residual CA as the C+1-th IMF, QSC+1 corresponds to the original signal time series a(p). The sample entropy feature based on the EMD can be calculated by:

$SEF\left( c,n,s \right)=SE\left( QS_{IMF}^{c}\left( p \right),n,s \right)$        (21)

3. Evaluation of Upper Limb Function Rehabilitation

As shown in Figure 1, the upper limb rehabilitation evaluation system consists of two modules: rehabilitation expert service platform, and upper limb rehabilitation training data acquisition platform. During upper limb rehabilitation training, the electrodes on the body of the patient collect EMG signal from different muscles, the internal sensors capture the angle and angular speed of elbow joints. The collected data are uploaded to the platform. Based on the action identification results, the doctor evaluates the rehabilitation effect, designs a personalized rehabilitation training program for the patient, or adjusts the current training scheme. Figure 2 shows the complete process of upper limb rehabilitation evaluation.

Figure 1. Architecture of upper limb rehabilitation evaluation system

Figure 2. Flow of upper limb rehabilitation evaluation

Drawing on the medical knowledge of upper limb rehabilitation and the clinical experience of authoritative doctors, the physiological signals were collected through experiments, and the number of distributions for the different levels of physiological signals meeting the rehabilitation evaluation by authoritative doctors. On this basis, a rehabilitation evaluation matrix was established for the collected physiological signals. The rehabilitation evaluation vector of the single-path physiological signal can be defined as:

${{\delta }_{i}}=\left\{ {{G}_{i1}}/G\text{  }{{G}_{i2}}/G\text{  }...\text{  }{{G}_{i5}}/G \right\}$       (22)

The upper limb rehabilitation effects were divided into three aspects: motor function rehabilitation state, balance function rehabilitation state, and coordination function rehabilitation state. On motor function rehabilitation state, the muscle strength of the patient was measured by the ratio of the measured integrated EMG IEMGMEA to the standard maximum integrated EMG IEMGmax:

$MS=\frac{IEM{{G}_{MEA}}}{IEM{{G}_{\max }}}\times 100{\scriptstyle{}^{0}/{}_{0}}$       (23)

The muscle tension of the patient was measured by the ratio of the RMS of measured EMG VROOT-MEA to the maximum RMS VROOT-max:

$MT=\frac{{{V}_{ROOT-MEA}}}{{{V}_{ROOT-\max }}}\times 100{\scriptstyle{}^{0}/{}_{0}}$       (24)

The greater the VROOT-MEA, the poorer the muscle function, the higher the degree of spasm, and the less optimistic the rehabilitation. The range of motion for upper limb joints was measured by the ratio of the maximum angle of upper limb movement ANGMEA to the reference maximum angle ANGmax:

$JR=\frac{AN{{G}_{MEA}}}{AN{{G}_{\max }}}\times 100{\scriptstyle{}^{0}/{}_{0}}$       (25)

The evaluation vectors for muscle strength, muscle tension, and the range of motion of joints are denoted as h1, h2, and h3, respectively. Based on the above metrics, the evaluation matrix of the upper limb motor function can be established as:

$H_{1}=\left[\begin{array}{l}

h_{1} \\

h_{2} \\

h_{3}

\end{array}\right]$       (26)

The evaluation result on the motor function can be vectorized as:

${{U}_{1}}={{\omega }_{1}}\circ {{H}_{1}}$       (27)

The balance function rehabilitation state was evaluated by only one factor: the balance ability of upper limb actions. The balance ability was measured by the ratio of the sample entropy SEMEA of the measured angular speed to the maximum sample entropy SEmax:

$BA=\frac{S{{E}_{MEA}}}{S{{E}_{\max }}}\times 100{\scriptstyle{}^{0}/{}_{0}}$       (28)

The greater the SEMEA, the more balanced the upper limb movement, and the more optimistic the rehabilitation. Let h4 be the evaluation vector for the angular speed of upper limb movement. Then, the evaluation matrix of the upper limb balance function can be established as:

${{H}_{\text{2}}}=\left[ {{h}_{\text{4}}} \right]$       (29)

The evaluation result on the balance function can be vectorized as:

${{U}_{\text{2}}}={{H}_{\text{2}}}$       (30)

Similar to the balance function rehabilitation state, the coordination function rehabilitation state was also evaluated by one factor: the co-contraction ratio of integrated EMG. The balance ability of upper limb movement was measured by the ratio of the co-contraction ratio of measured integrated EMG IEMG-SSRmax to the co-contraction ratio of the standard maximum integrated EMG IEMG-SSRmax:

$CA=\frac{IEMG-SS{{R}_{MEA}}}{IEMG-SS{{R}_{\max }}}\times 100{\scriptstyle{}^{0}/{}_{0}}$       (31)

Let h5 be the evaluation vector for the coordination of upper limb movement. Then, the evaluation matrix of the upper limb coordination function can be established as:

${{H}_{3}}=\left[ {{h}_{5}} \right]$       (32)

The evaluation result on the coordination function can be vectorized as:

${{S}_{3}}={{R}_{3}}$       (33)

4. Experimental Results and Analysis

The information fusion between sEMG signal and acceleration signal is crucial to the identification of upper limb actions. This paper extracts multiple features from the main feature segments of the upper limb actions of patients. Figure 3 is the scatterplot of the features of the two types of physiological signals. It can be observed that sEMG signal and acceleration signal are not greatly disturbed by external factors, and are highly discriminable. This is because the patient’s action trajectories vary significantly between stages of rehabilitation. Nevertheless, the action features of different patients in different stages may overlap, for different patients have different habits in executing the same actions. It is necessary to assist the action identification with sEMG signal information that accurately mirrors the fine actions of the upper limbs.

The proposed upper limb action identification method, which fuses sEMG signal with acceleration signal, is dominated by sEMG signal. The reason is very simple: the details on hand muscles, wrist muscles, and elbow muscles conveyed by sEMG signal are the key to the successful recognition of upper limb movement.

This paper carries out experiments separately on time-domain features, time-frequency domain features, entropy-domain features, and fused features (which merge the features from different domains). On this basis, the authors analyzed how a single information, i.e., the source of sEMG signal influences the identification of dynamic upper limb actions. Table 1 compares the feature identification accuracy on different patients. The fused features led to the highest accuracy and best effect of upper limb action identification.

Feature extraction was performed on the physiological signals from six patients. The extracted features were divided into a training set and a test set. During the experiments, 140 feature samples were extracted randomly each time. 80 samples were allocated for training, and the remaining 60 for testing. Three different models, namely, KNN, ANN, and SVM, were applied separately to identify upper limb actions. The results in Table 2 suggest that the identification accuracy was significantly improved using the fused signal, resulting in an ideal identification effect. For all three classification models, more than 90% of upper limb actions were identified accurately, using the fused features of sEMG signal and acceleration signal. Therefore, the proposed physiological signal feature extraction approach adapts well to different upper limb action identification models.

Following the proposed flow for upper limb rehabilitation evaluation, the final evaluation results were obtained for the upper limb habilitation of the six patients (Table 3). The proposed evaluation model achieved largely the same results with clinical evaluation, reflecting the feasibility and effectiveness of our model.

Figure 3. sEMG signal features of different patients

Table 1. Feature identification accuracy on different patients

Patient number

Time-domain features

Time-frequency domain features

Entropy-domain features

Fused features

1

72.5

82.6

76.4

87.1

2

68.4

86.5

74.1

85.7

3

78.2

82.4

86.8

90.7

4

61.8

72.4

72.5

88.9

5

63.2

81.2

74.4

81.4

6

64.1

74.9

81.5

83.3

Mean accuracy

69.2

75.4

76.8

84.2

Table 2. Mean accuracy of upper limb action identification

Model

KNN

ANN

SVM

Signaled source

Acceleration signal

sEMG signal

Fused signal

Acceleration signal

sEMG signal

Fused signal

Acceleration signal

sEMG signal

Fused signal

Patient 1

68.2

81.4

92.8

73.5

82.8

92.4

72.4

85.2

95.6

Patient 2

65.4

85.2

92.6

68.7

83.1

93.8

74.2

85.4

97.2

Patient 3

72.5

85.4

96.2

76.4

85.4

92.1

84.5

86.2

96.8

Patient 4

75.1

87.4

95.2

75.4

88.4

92.5

74.1

86.3

95.8

Patient 5

82.5

85.7

96.8

82.4

83.7

95.8

82.7

86.7

95.7

Patient 6

73.8

86.9

97.2

76.4

86.5

96.7

74.2

88.5

96.2

mean

74.8

86.1

94.8

76.2

86.8

96.2

74.8

86.4

96.8

Note: KNN and ANN are short for k-nearest neighbors and artificial neural network, respectively.

Table 3. Evaluation results on upper limb motor function

Patient number

1

2

3

Memberships corresponding to the five levels

Cured

0.23

0.45

0.35

Basically cured

0.42

0.65

0.48

Slightly dysfunctional

0.26

0.55

0.68

Moderately dysfunctional

0.28

0.22

0.46

Strongly dysfunctional

0.27

0.35

0.28

Evaluation results

Moderately dysfunctional

Basically cured

Moderately dysfunctional

Patient number

4

5

6

Memberships corresponding to the five levels

Cured

0.41

0.37

0.29

Basically cured

0.39

0.42

0.47

Slightly dysfunctional

0.53

0.58

0.36

Moderately dysfunctional

0.53

0.55

0.38

Strongly dysfunctional

0.46

0.33

0.28

Evaluation results

Basically cured

Moderately dysfunctional

Slightly dysfunctional

Finally, 13 muscles were selected from the shoulder muscles, arm muscles, fore arm muscles, and hand muscles of a patient subjecting to different rehabilitation trainings. The relative activation degree and activation duration of these muscles were tested in each training. The test results are presented in Figures 4 and 5. The relative activation degree of the muscles was characterized by motor function, balance function, and coordination function, while the activation duration of these muscles was characterized by training duration and training intensity. As shown in Figures 4 and 5, after the patient completed rehabilitation trainings (i.e., active training, anti-resistance training, and passive training), both the relative activation degree and activation duration of the muscles were improved. The results are consistent with the evaluation results on the patient’s upper limb motor function. This further confirms the effectiveness of our evaluation model.

Figure 4. Relative activation degree of upper limb muscles of a patient in different rehabilitation trainings

Figure 5. Activation duration of upper limb muscles of a patient in different rehabilitation trainings

5. Conclusions

This paper probes into upper limb action identification based on physiological signals, and applies the method to limb rehabilitation training. Firstly, multiple methods, namely, time-domain feature method, frequency-domain feature method, time-frequency domain feature method, and entropy feature method, were adopted to extract and identify the upper limb action features during limb rehabilitation training. Next, the authors explained the flow of the evaluation for upper limb rehabilitation, and selected the specific evaluation indices. Through experiments, the feature identification accuracy on different patients, as well as the mean accuracy of upper limb action identification were tested, revealing that the proposed physiological signal feature extraction approach adapts well to different upper limb action identification models. In addition, the effectiveness of our evaluation model was further verified by comparing the evaluation results on upper limb motor function of a patient with the relative activation degree and activation duration of upper limb muscles of the patient in different rehabilitation trainings.

Acknowledgements

This work was supported by the Project of Changchun Bureau of Science and Technology, Project Name: Rehabilitation massage system based on the output of the standard mechanical evaluation technique (Grant No.: 21ZGM31), and by the project of Jilin Provincial Bureau of Science and Technology (Grant No.: 20200301054RQ). This work was supported by the "Climbing Plan" of Changchun University, relevant technical achievements have been applied for the high-level talent-entrepreneurship team project in Ninghai County, Zhejiang Province.

  References

[1] Oubre, B., Daneault, J.F., Jung, H.T., Whritenour, K., Miranda, J.G.V., Park, J., Ryu, T., Kim, Y., Lee, S.I. (2020). Estimating upper-limb impairment level in stroke survivors using wearable inertial sensors and a minimally-burdensome motor task. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 28(3): 601-611. https://doi.org/10.1109/TNSRE.2020.2966950

[2] Antelis, J.M., Montesano, L., Ramos-Murguialday, A., Birbaumer, N., Minguez, J. (2016). Decoding upper limb movement attempt from EEG measurements of the contralesional motor cortex in chronic stroke patients. IEEE Transactions on Biomedical Engineering, 64(1): 99-111. https://doi.org/10.1109/TBME.2016.2541084

[3] Li, C., Rusák, Z., Hou, Y., Young, C., Ji, L. (2014). Upper limb motor rehabilitation integrated with video games focusing on training fingers' fine movements. International Journal of Robotics and Automation, 29(4): 359-368. https://doi.org/10.2316/Journal.206.2014.4.206-3984)

[4] Cruz, V.T., Bento, V.F., Ribeiro, D.D., Araújo, I., Branco, C.A., Coutinho, P. (2014). A novel system for automatic classification of upper limb motor function after stroke: An exploratory study. Medical Engineering & Physics, 36(12): 1704-1710. https://doi.org/10.1016/j.medengphy.2014.09.009

[5] Sucar, L.E., Orihuela-Espina, F., Velazquez, R.L., Reinkensmeyer, D.J., Leder, R., Hernández-Franco, J. (2013). Gesture therapy: An upper limb virtual reality-based motor rehabilitation platform. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 22(3): 634-643. https://doi.org/10.1109/TNSRE.2013.2293673

[6] Papatheodorou, N., Pino, A., Kouroupetroglou, G.T., Constantinides, V., Andreadou, E., Papageorgiou, C.C. (2019). Upper limb motor skills performance evaluation based on point-and-click cursor trajectory analysis: application in early multiple sclerosis detection. IEEE Access, 7: 28999-29013. https://doi.org/10.1109/ACCESS.2019.2901926

[7] Sun, T., Hu, Q., Gulati, P., Atashzar, S.F. (2021). Temporal Dilation of Deep LSTM for Agile Decoding of sEMG: Application in Prediction of Upper-limb Motor Intention in NeuroRobotics. IEEE Robotics and Automation Letters, 6(4): 6212-6219. https://doi.org/10.1109/LRA.2021.3091698

[8] Chen, X., Shi, Y., Wang, Y., Cheng, Y. (2021). Deep learning-based image automatic assessment and nursing of upper limb motor function in stroke patients. Journal of Healthcare Engineering, 2021: 90559411. https://doi.org/10.1155/2021/9059411

[9] Yang, Z., Guo, S., Liu, Y., Hirata, H., Tamiya, T. (2021). An intention-based online bilateral training system for upper limb motor rehabilitation. Microsystem Technologies, 27(1): 211-222. https://doi.org/10.1007/s00542-020-04939-x

[10] Maistrello, L., Rimini, D., Cheung, V.C., Pregnolato, G., Turolla, A. (2021). Muscle synergies and clinical outcome measures describe different factors of upper limb motor function in stroke survivors undergoing rehabilitation in a virtual reality environment. Sensors, 21(23): 8002. https://doi.org/10.3390/s21238002

[11] Kaur, A., Kumar, A. (2017). SEMG based classification using wavelet function for around shoulder muscles. Journal of Engineering Science & Technology Review, 10(4): 109-114. https://doi.org/10.25103/jestr.104.15

[12] Sbargoud, F., Djeha, M., Guiatni, M., Ababou, N. (2019). WPT-ANN and belief theory based EEG/EMG data fusion for movement identification. Traitement du Signal, 36(5): 383-391. https://doi.org/10.18280/ts.360502

[13] Vijayvargiya, A., Prakash, C., Kumar, R., Bansal, S., Tavares, J.M.R. (2021). Human knee abnormality detection from imbalanced sEMG data. Biomedical Signal Processing and Control, 66: 102406. https://doi.org/10.1016/j.bspc.2021.102406

[14] Kalani, H., Moghimi, S., Akbarzadeh, A. (2016). Towards an SEMG-based tele-operated robot for masticatory rehabilitation. Computers in Biology and Medicine, 75: 243-256. https://doi.org/10.1016/j.compbiomed.2016.05.014

[15] Zhou, Y., Zeng, J., Li, K., Hargrove, L.J., Liu, H. (2020). sEMG-driven functional electrical stimulation tuning via muscle force. IEEE Transactions on Industrial Electronics, 68(10): 10068-10077. https://doi.org/10.1109/TIE.2020.3026280

[16] Yao, T., Gao, F., Zhang, Q., Ma, Y. (2021). Multi-feature gait recognition with DNN based on sEMG signals. Mathematical Biosciences and Engineering, 18(4): 3521-3542. https://doi.org/10.3934/mbe.2021177

[17] Veer, K., Sharma, T., Kumar, A. (2016). Development of prosthetic arm using body actioned SEMG signals. Journal of Innovative Optical Health Sciences, 9(6): 1650023. https://doi.org/10.1142/S1793545816500231

[18] Zhang, L., Shao, Q., Yang, J., Yang, M., Tao, N., Song, Z. (2017). Muscle Degree of Comfort Assessment Rule Based on sEMG. China Mechanical Engineering, 28(6): 708. 

[19] Nurhanim, K., Elamvazuthi, I., Izhar, L.I., Ganesan, T., Su, S.W. (2016). Development of a model for sEMG based joint-torque estimation using Swarm techniques. 2016 2nd IEEE International Symposium on Robotics and Manufacturing Automation (ROMA), Ipoh, Malaysia, pp. 1-6. https://doi.org/10.1109/ROMA.2016.7847833

[20] Wang, X., Dong, D., Chi, X., Wang, S., Miao, Y., Gavrilov, A.I. (2021). sEMG-based consecutive estimation of human lower limb movement by using multi-branch neural network. Biomedical Signal Processing and Control, 68: 102781. https://doi.org/10.1016/j.bspc.2021.102781

[21] Göğüş, F.Z., Tezel, G., Özşen, S., Küççüktürk, S., Vatansev, H., Koca, Y. (2020). Identification of apnea-hypopnea index subgroups based on multifractal detrended fluctuation analysis and nasal cannula airflow signals. Traitement du Signal, 37(2): 145-156. https://doi.org/10.18280/ts.370201

[22] Vitali, A., Regazzoni, D., Rizzi, C., Maffioletti, F., Falconi, L. (2020). Quantitative assessment of upper limb rehabilitation through digital motion acquisition. Computer-Aided Design & Applications, 17(6): 1266-1277. https://doi.org/10.14733/cadaps.2020.1266-1277

[23] Guo, S., Cai, H., Guo, J. (2020). A novel fuzzy neural network-based rehabilitation stage classifying method for the upper limb rehabilitation robotic system. 2020 IEEE International Conference on Mechatronics and Automation (ICMA), Beijing, China, pp. 261-266. https://doi.org/10.1109/ICMA49215.2020.9233519

[24] Pundik, S., McCabe, J., Kesner, S., Skelly, M., Fatone, S. (2020). Use of a myoelectric upper limb orthosis for rehabilitation of the upper limb in traumatic brain injury: A case report. Journal of Rehabilitation and Assistive Technologies Engineering, 7: 2055668320921067. https://doi.org/10.1177%2F2055668320921067

[25] Wang, W., Li, H., Kong, D., Xiao, M., Zhang, P. (2020). A novel fatigue detection method for rehabilitation training of upper limb exoskeleton robot using multi-information fusion. International Journal of Advanced Robotic Systems, 17(6): 1729881420974295. https://doi.org/10.1177%2F1729881420974295

[26] Guo, B., Ma, Y., Yang, J., Wang, Z., Zhang, X. (2020). Lw-CNN-based myoelectric signal recognition and real-time control of robotic arm for upper-limb rehabilitation. Computational Intelligence and Neuroscience, 2020: 8846021. https://doi.org/10.1155/2020/8846021