Emimal Murugiah*
| Jino Hans William | Inbamalar Tharcis Mariapushpam | Mahiban Lindsay Nesaian
© 2024 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
This study underscores the imperative role of feature extraction and classification in the domain of electromyography (EMG) signals. The efficacy of myoelectric pattern recognition hinges upon the judicious selection of pertinent features. In this research endeavor, we introduce a novel approach that leverages two distinct feature sets: conditional spectral moments and refined time-domain descriptors, tailored to augment the precision of myoelectric signal classification. The conditional spectral moments, derived from the time-frequency distribution of EMG signals, encapsulate nuanced variations in muscle activity and movement dynamics. This augmentation facilitates seamless differentiation of hand gestures, a pivotal advancement in the context of prosthetic applications. Concurrently, we enhance the time-domain descriptors by convolving them with wavelet filter coefficients, thereby extracting both spatial and temporal characteristics of muscle activity. The extracted features are subjected to classification using Support Vector Machines (SVM), K-Nearest Neighbor (KNN), Decision Tree (DT), and Ensemble Bagging classifiers. To elucidate the efficacy of our proposed features, comprehensive experiments are conducted employing the benchmark databases Ninapro DB1 (comprising 52 classes) and DB2 (comprising 49 classes). Our findings underscore the superiority of the proposed features, particularly when applied in conjunction with ensemble classifiers. Specifically, the classification accuracies of the modified time-domain descriptor feature exhibit substantial enhancements, achieving 87.1% and 85.3% accuracy rates for Ninapro DB1 and DB2, respectively. Notably, the conditional spectral moments outperform with remarkable classification accuracies of 92.9% and 90.8% for Ninapro DB1 and DB2, respectively. This marked improvement in EMG signal classification corroborates the enhanced precision in decoding hand movements, thus imparting significant advancements in the realm of hand prosthesis control.
electromyography (EMG), myoelectric, prosthesis, pattern recognition, feature extraction, spectral moments, classification
Myoelectric is a term that refers to the electric properties of muscles, and myoelectric-controlled prostheses utilize the electrical signals emanating from muscles [1]. The EMG acquired from the muscles in the residual limb is used by the myoelectric prostheses to control its functionality. Electromyography is a diagnostic procedure used to assess the health of muscles and the nerve cells that control them. EMG non-invasively measures muscle activity and extracts reliable information from the muscles, making it commonly used by researchers to determine discriminative information. Pattern recognition (PR) techniques extract insightful information from the EMG signals and interpret the neural control signals into movement commands for movement recognition. Prosthesis controllers receive commands to implement movements using EMG patterns based on pattern classification. This allows users to manage their myoelectric prosthesis more easily and with more control using the EMG-PR technique [2]. It is challenging to operate artificial hands with the same degree of dexterity and complexity as biological hands. Nevertheless, PR has long been used for controlling myoelectric prosthetic devices. PR enables more intuitive control that is easier for both humans and computers to learn. It also allows the independent control of numerous degrees of freedom (DOF), viz., simultaneous, sequential, or semi-sequential control, bringing the prosthesis closer to natural arm functions. The limb movement can be accurately decoded and used to control a prosthetic hand using a suitable PR-based and signal-processing technique combined with machine learning algorithms. There have been many studies that show that the pre-processing of EMG signals, the choice of good features, and the right classification methods are very important to the overall performance of myoelectric systems [3-5].
Several feature extraction schemes have been proposed for EMG feature extraction and classification. Kuzborskij et al. [6] utilized time domain features like mean absolute value (MAV), wavelength, and histogram for classifying large-scale data sets. Waris et al. [7] investigated time-domain features like MAV, Zero Crossing (ZC), Willison Amplitude (WAMP), Slope Sign Changes (SSC), Waveform Length (WL), Cardinality (CARD), and Myopulse Rate (MYOP) for intuitive control for upper-limb prosthesis. A combination of time-domain features by Hudgins et al. [8], namely the Hudgins Time Domain feature set, paved the way for multifunction myoelectric control. Khushaba et al. [4] derived the Time-Dependent-Power Spectral Descriptor (TD-PSD) feature from the power spectral descriptors, which was proven to find the muscular activities. Phinyomark and Scheme [9] have proposed the mean prominence of local peaks and valleys, which excel in the HTD feature set. Frequency domain features like autoregressive model coefficients proposed by Al-Timemy et al. [10] and the logarithm of Fourier transform (FT) moments feature proposed by Al-Timemy et al. [5] for multifunction prosthesis are available for EMG classification. Wavelet transform-based feature by Chu et al. [11], wavelet-based iterative feature, namely, ternary pattern by Turker et al. [12], energy features by Karnam et al. [13], and variational mode decomposition (VMD)-based feature extraction methods [14] were developed for efficient EMG classification. A spatiotemporal feature developed by Jabbari et al. [15], namely spatiotemporal warping (STW), outperformed the traditional features.
A classifier is required to determine the gesture label of the signal after feature extraction. Several classifiers have been suggested and tested, including linear discriminant analysis (LDA), SVM [16], MLP [17], and KNN [18]. Techniques known as ensemble machine learning combine the findings of numerous learners to produce better results [19].
EMG pattern recognition can be widely categorized into traditional and deep learning (DL) methods, while the former depends on feature engineering and the latter on feature learning [20]. Wei et al. [21] proposed a Multi-View Convolutional Neural Network (CNN) framework with DL techniques for hand gesture classification. An improved DL-based framework was framed by Pancholi et al. [22] for myoelectric prosthesis control. An EMG signal classification model using CNN was presented by Atzori et al. [23], with results comparable to state-of-the-art machine learning techniques. Tsinganos et al. [24] used temporal CNN to classify EMG signals into 52 classes. Côté-Allard et al. [25] utilized the transfer learning technique with CNN to classify EMG signals.
Despite the fact that quite a few state-of-the-art approaches are available for feature extraction and classification of EMG signals, the robustness of the classification technique poses a challenge to the researchers. There are a lot of challenges in classifying hand gestures with multiple degrees of freedom [26, 27], and getting the model to work with a lot of different motions, like moving the upper and lower limbs, is important for making prosthetic control better [28]. Although notable progress has been made in DL methods for EMG-based hand gesture classification, it is essential to recognize that these achievements often entail considerable computational expenses. These expenses stem from the extensive data needed for training deep learning models such as CNN and the relatively large number of parameters that have to be learned. The number of model parameters varies widely, ranging from 34k to 95k [29], 104k to 30k [30], and 30k to 549k [31], extending to several million parameters. Hence, a straightforward and resource-efficient model suitable for operation on constrained platforms is a crucial element in the development of machine-learning-based prosthetic devices. Moreover, robust feature extraction and classification have to be developed to classify EMG signals.
The contributions of this research are:
(1) To extract conditional spectral moment features from the EMG signal for capturing the variations of different hand gestures.
(2) To obtain a modified TD-PSD by convolving the low-pass decomposition filter coefficients of the wavelet transform with the TD-PSD [5].
(3) To classify the features extracted from the EMG signals with SVM, KNN, DT, and Bagged Ensemble DT classifiers.
A feature extraction and classification framework has been proposed to improve the classification accuracy of EMG signals for hand prostheses to address the above-mentioned challenges. The conditional spectral moment and TD-PSD were extracted from the EMG signal. Spectral moments seem to be a potential strategy for EMG classification and have been employed in several applications [24, 25]. The conditional spectral moment is a morphological feature and is more noise-resistant, which has been extracted from the spectrogram of the EMG signal [32]. Dynamic hand movements that involve muscle activations in different sequences with varying intensities result in asymmetric EMG signal distribution. The conditional spectral moments identify and differentiate patterns of muscle activity that exhibit asymmetric distributions.
In the context of EMG-based hand gesture classification, it is important to recognize that individual hand gestures give rise to unique patterns of muscle activity. TD-PSD is a useful method for measuring how sensitive a signal is to changes in time. This lets us tell the difference between and describe the different patterns of muscle activity that are linked to different hand gestures. In the current work, the TD-PSD proposed by Al-Timemy et al. [5] is utilized as the base feature for EMG classification. These features are derived directly from the time domain through the use of FT and Parseval's theorem. The derivatives utilized in the power spectral moments are susceptible to noise interference; hence, it becomes crucial to apply feature value normalization to mitigate noise effects. To achieve this, a normalization step is incorporated by exponentiating the log-scaled amplitudes to an appropriate power [5]. The TD-PSD feature was mixed with wavelet transform filter coefficients to include both the spatial and temporal aspects of hand gesture EMG signals. This made the modified TD-PSD feature representation. When you combine the base feature set with wavelet transform filter coefficients, you get feature values from the combination of channels that were thought about, which helps with spatial focus. The rationale for integrating the wavelet filter coefficients with the time domain descriptors is to combine the information from both the time and wavelet domains. Also, this approach captures temporal and frequency-related characteristics. This is due to the fact that the EMG signals encompass frequency components associated with muscle activity and motion. The fusion of these frequency components with the statistical characteristics of time-domain descriptors enhances the discriminative capability of the extracted features. During muscle contractions or movements, the primary power of the EMG signal is concentrated within the 20 Hz to 500 Hz frequency range. The low-pass decomposition filter is adept at noise removal while retaining the essential signal components. This study introduces a changed TD-PSD feature that takes advantage of the benefits of wavelet transform filter coefficients and power spectral descriptors.
This work is illustrated as follows: The EMG dataset used for this research is detailed in Section 2.1. In Section 2.2, the proposed conditional spectral moments and the modified TD-PSD features are elaborated. The feature sets utilized from the literature for comparing the accuracy of the proposed features are detailed in Section 2.3. The details of the classifiers employed for classifying the extracted features are given in Section 2.4. The classification outcomes achieved using the proposed features with the Ninapro databases, as well as the corresponding discussions, are presented in Section 3. Section 4 describes the conclusion and outlines future work.
2.1 EMG dataset
The proposed feature is evaluated with two databases, Ninapro DB1 [33] and DB2 [34], a widely used benchmark dataset for myoelectric prostheses. A collection of 52 movements were selected from the DB1, which were split up into three exercises, namely A, B, and C. Exercise A is comprised of 12 basic finger movements; Exercise B incorporates 17 hand postures and basic wrist movements; and Exercise C contains 23 functional and grasping movements of the hands correlated with daily life actions. DB2 comprises 49 hand gestures from 40 subjects and includes finger gestures and functional and grasping movements. The details of the datasets utilized are summarized in Table 1. OttoBock MyoBock 13E200 sEMG electrodes were used to record the surface EMG (SEMG) data. These electrodes amplified, filtered, and rectified the signals. We used MyoBock 13E200-50 electrodes to measure the muscle activity of DB2 in 40 healthy individuals. sEMG signals were gathered from 12 electrodes placed on the forearm, flexor Digitorum Superficialis, muscle extensor Digitorum Superficialis, triceps brachii, and biceps brachii.
Table 1. Details of the dataset
|
Parameter |
Ninapro DB1 |
Ninapro DB2 |
|
No. of subjects, classes |
27, 52 |
40, 49 |
|
No. of repetitions |
10 |
6 |
|
No. of channels |
10 |
12 |
|
Sampling frequency (Hz) |
100 |
2000 |
|
Duration (sec) |
5 |
5 |
|
Total patterns |
14040 |
11760 |
|
Training patterns |
8424 |
7840 |
|
Test patterns |
5616 |
3920 |
2.2 Proposed feature extraction
2.2.1 Conditional spectral moment
Figure 1 depicts the work carried out in this paper. The raw, multichannel EMG signal undergoes a pre-processing stage, a feature extraction stage, and finally the classification stage. The conditional spectral moment of a non-stationary EMG signal is obtained from the EMG signal spectrum. This is taken from the time-frequency distribution, $P(t, \omega)$ of the signal and is given by:
$\left\langle\omega^i\right\rangle_t=\frac{1}{P(t)} \int \omega^i P(t, \omega) d \omega$ (1)
P(t) defines the marginal distribution and describes the order.
Eq. (1) is termed the ‘instantaneous frequency’ or mean. The variance, or ‘instantaneous bandwidth’ is given by:
$\sigma_{\omega / t}^2=\int\left(\omega-\langle\omega\rangle_t\right)^2 P(\omega \mid t) d \omega$ (2)
Skewness and kurtosis are the third and fourth-order moments, respectively. This is described as:
$C_{\omega / t}=1 \sigma_{\omega / t}^3 \int\left(\omega-\langle\omega\rangle_t\right)^3 P(\omega \mid t) d \omega$ (3)
$K_{\omega / t}=1 \sigma_{\omega / t}^4 \int\left(\omega-\langle\omega\rangle_t\right)^4 P(\omega \mid t) d \omega$ (4)
The first four moments mentioned above, namely mean, variance, skewness, and kurtosis, are extracted from the EMG signal for the Ninapro DB1 and DB2 for each class of movements.
2.2.2 Modified TD-PSD
In the work TD-PSD [5], an improvement in classification accuracy of $6 \%$ to $8 \%$ was reported. In the current work, a modified TD-PSD feature is proposed and obtained from the TD-PSD by convolving each descriptor $\left(f_1, f_2, f_3, f_4, f_5, f_6\right)$ mentioned in Figure 1 with the low-pass wavelet filter coefficients of the input EMG signal.
The time domain descriptors $\left(f_1, f_2, f_3, f_4, f_5, f_6\right)$ are extracted from the pre-processed EMG signal, and each feature takes the shape of $(1 \times M)$, wherein $M$ is the length of the EMG channel. These features are reshaped into an array of sizes $(a \times b)$, such that the product of $a$ and $b$ gives $M$. The wavelet filter coefficient is obtained by wavelet decomposing the input EMG with a decomposition level of 2. The reshaped feature array $\left(R_1, R_2, \ldots, R_6\right)$ is convolved with the filter coefficient $L$, which is further flattened and concatenated, forming a new feature $F 4$.
The EMG signal is decomposed with a wavelet transform, which uses the Daubechies wavelet family, ‘db44’. The reason behind the selection of ‘db44’ is that it possesses more resemblance in the signal and mother wavelet function. Most biological signals like EMG possess sharp spikes, which can be analyzed with Daubechies wavelets since these wavelets experience asymmetric spikes. Thus, ‘db44’ seems to have a similar function to EMG amid the 324 mother wavelets [35]. The time domain descriptors are derived from the relation obtained by Parseval’s theorem and are given in the Appendix.
Let L denote the filter coefficients obtained by wavelet decomposing the input EMG signal to a decomposition level of 2. The coefficients in general can be expressed as:
$L=\left[\begin{array}{ll}l_1 & l_2 \\ l_3 & l_4\end{array}\right]$ (5)
The modified TD-PSD is obtained as follows:
$r_1=R_1 * L ; r_2=R_2 * L ; r_3=R_3 * L$ (6)
$r_4=R_4 * L ; r_5=R_5 * L ; r_6=R_6 * L$ (7)
$F_4=$ concatenate $\left[r_1 ; r_2 ; r_3 ; r_4 ; r_5 ; r_6\right]$ (8)
Figure 1. Proposed feature extraction and classification
2.3 Feature sets for investigation
The following feature sets were also taken from the literature to compare the effectiveness of the proposed feature.
AR-HTD [36]: This is the combination of 6th-order autoregressive coefficients and Hudgins Time Domain features MAV, IAV, RMS, WL, SSC, and ZC.
AR-RMS [37]: Combination of autoregressive coefficients of 6th order and RMS value.
MWTD [38]: The multisignal wavelet transform decomposition coefficients can be extracted from the wavelet transform coefficients of each node, including energy, variance, standard deviation, waveform length, and entropy. The entire set of feature sets for investigation is mentioned in Table 2.
Table 2. Features used for classification
|
Features under Investigation |
|
|
AR-HTD |
F1 |
|
AR-RMS |
F2 |
|
MWTD |
F3 |
|
Modified TD-PSD |
F4 |
|
Conditional spectral moments |
F5 |
2.4 Classification of EMG signals
The proposed feature was investigated with four different classifiers: SVM, KNN, Decision Tree Classifier, and Decision Tree Ensemble with Bagging. The pseudocode for the feature extraction and classification is given in Table 3. SVM represents a category of supervised machine learning techniques employed in tasks involving classification and regression [18]. They are well-suited for classifying hand gestures with EMG signals. The objective of this classification algorithm is to find an optimal hyperplane within the feature space, effectively distinguishing various classes of hand movements. Training the SVM model involves utilizing a labeled dataset of EMG signals, each associated with a specific hand gesture, which enables the SVM to identify the most suitable hyperplane for separating the distinct hand gesture classes [19]. In KNN, data points with similar features tend to have the same class labels. When classifying a new data point, KNN looks at the class labels of its nearest neighbors in feature space and assigns the majority class as the predicted label. When classifying a new EMG signal corresponding to a hand gesture, KNN calculates the distance between the feature vector and all feature vectors in the training dataset. The decision tree algorithm navigates through the tree structure, comparing the feature values of the EMG signal to the criteria at each node. The path followed through the tree leads to a leaf node, which represents the predicted hand gesture class. If the data is high-dimensional and if the single classifier fails to perform well, ensemble classifiers provide better results. Decision trees with ensemble bagging classifiers combine the principles of decision trees and ensemble learning, which involves aggregating the predictions of multiple individual models to improve accuracy and robustness. The feasibility of the bagging and boosting ensemble classifiers for basic hand movement recognition is evaluated using EMG signals captured during grasping actions with different objects for the six hand movements [39].
Table 3. Pseudo code for the proposed feature extraction
|
Pseudocode: sEMG signal classification scheme with the proposed feature extraction |
|
Input: n×m sEMG signals xn×m(i), each with class labels 1. for each n = 1 to N, m = 1 to M. 2. for winsize = 250ms, wininc = 75ms. 3. Compute f1=m0, f2=m2, f3=m4, f4=Sparseness, f5=IRF, f6=WL. 4. Wavelet decompose xn×m(i), with 'db44' & extract the filter coefficients by decomposing xn×m(i): L= Wavelet filter coefficients 5. R1=Reshape(f1), R2=Reshape(f2), R3=Reshape(f3), R4=Reshape(f4), R5=Reshape(f5), R6=Reshape(f6). [features of size (1×M) are reshaped into the shape a×b, such that multiplication of a and b gives the channel M. 6. Convolution: R1*L, R2*L, R3*L, R4*L, R5*L, R6*L 7. Flatten and concatenate à Feature F4. 8. Compute the first four conditional spectral moments using Eqs. (1), (2), (3), and (4), feature F5. 9. Classify features with SVM, KNN, Decision Tree, and Ensemble Bagging. |
3.1 EMG data pre-processing model training
This research was conducted on an HP DESKTOP-F26D10N running Windows 10 Pro 64-bit, equipped with an AMD A8-5500B APU featuring Radeon HD Graphics with a 3.20 GHz processor and 8.00 GB of installed RAM. The data pre-processing, feature extraction, and classification were done in MATLAB 2022 software to obtain both the training and testing datasets. Each dataset undergoes different preprocessing procedures. The Ninapro DB1 is sampled at 100 Hz, and wavelet denoising using the Daubechies wavelet of order 44 ['db44'] wavelet is applied. NinaPro DB2 was sampled at 2 kHz. It undergoes low-pass filtering with a 500 Hz cutoff, line noise filtering at 50 Hz, and wavelet denoising using the 'db44' wavelet since the ‘db44’ family of wavelets possesses similar characteristics as the EMG signal. Ten-fold cross-validation is performed to obtain a robust estimate of the model performance and also for better generalization capability. All the classifiers were trained with 70% cross-validation and 30% test sets for all the exercises. Both the training and testing datasets are standardized, ensuring that the resulting data exhibits a mean of zero and a variance of one. Let $T P$ and $T N$ denotes the True positive and True Negative values, $F P$ and $F N$ denotes the False Positive and False Negative values, the classification accuracy is given by:
Accuracy $=\frac{T P+T N}{T P+T N+F P+F N}$ (9)
3.2 Results of Ninapro DB1 and DB2
The proposed features (F4 and F5) are evaluated on the two benchmark datasets, Ninapro DB1 and DB2. Ten-fold cross-validation is used, in which the data is randomly divided into ten equal parts, ensuring that each class is represented proportionally as it appears in the entire dataset. The Ninapro DB1 includes 52 hand gestures, which are categorized into 3 exercises, namely E1 (12 classes), E2 (17 classes), and E3 (23 classes), respectively. Ninapro DB2 contains 49 hand gestures, which are categorized into 3 exercises: E1 (17 classes), E2 (23 classes), and E3 (9 classes). Two experiments were conducted on the aforementioned datasets:
(1) Model evaluation for 12, 17, and 23 classes: The choice of window length and overlap impacts the temporal resolution of feature extraction, influencing the ability to capture detailed temporal aspects. Hence, evaluating the proposed features F4 and F5 with different window lengths and overlaps [(250 ms, 75 ms) and (200 ms, 50 ms)] for the classifiers SVM, KNN, DT, and Bagged Ensemble DT and exploring the best feature-classifier combination. The analysis is conducted separately on three exercises within each dataset to reveal the implications of classification accuracy across different numbers of gestures.
(2) Model evaluation for 52 classes: evaluating the proposed features for all the hand gestures in Ninapro DB1 and DB2.
3.2.1 Model evaluation for 12, 17, and 23 classes
The classification accuracy of Ninapro DB1 for window lengths of 250 ms and 200 ms and overlap of 75 ms and 50 ms is tabulated in Table 4 for all the features mentioned in Table 2. The features F4 and F5 provide better classification accuracy with a bagged ensemble classifier for the window length and overlap of 250 ms and 75 ms, respectively. Similar results are found for all the features. The feature sets F1, F2, and F3 give higher classification accuracy with a bagged ensemble classifier for window length and overlap of 250 ms and 75 ms, respectively. The accuracy of the KNN classifier is higher than that of the SVM for features in F1 and F2. A statistical significance test was conducted for all feature combinations and classifiers and found that the bagged ensemble classifier with features F4 and F5 significantly outperformed all other classifiers (p<0.01).
Table 4. Classification accuracies of Ninapro DB1for 12, 17, 23 classes
|
Feature |
Exercise |
(Win Len, Overlap) |
Classification Accuracy (%) |
|||
|
SVM |
KNN |
DT |
Ensem. Bagg. |
|||
|
F1 |
E1 E2 E3 |
(250ms,75ms) |
69.6 66 65.2 |
68.3 67.8 65.8 |
74.6 72.9 72.2 |
82.5 80.4 77.5 |
|
F1 |
E1 E2 E3 |
(200ms,50ms) |
66.3 65.8 63.7 |
68.1 66.3 65.2 |
73.9 72 71.6 |
81.4 80.1 76.4 |
|
F2 |
E1 E2 E3 |
(250ms,75ms) |
76.7 74.8 73.5 |
82.3 81.4 80.7 |
80.1 78.6 78.1 |
84.1 82.9 81.4 |
|
F2 |
E1 E2 E3 |
(200ms,50ms) |
76.1 73.9 73.2 |
81.8 81.3 79.1 |
79 78.1 76.4 |
83.6 81.8 80.3 |
|
F3 |
E1 E2 E3 |
(250ms,75ms) |
67.3 65.4 64 |
65.9 64.8 64.2 |
76.2 74.8 73.3 |
83.2 83 81.2 |
|
F3 |
E1 E2 E3 |
(200ms,50ms) |
66.8 63.9 63.7 |
65.3 64.2 63.8 |
75.8 74.2 72.8 |
83.1 82.6 80.1 |
|
F4 |
E1 E2 E3 |
(250ms,75ms) |
88.7 88.1 85.4 |
87.8 84.1 83.3 |
80.4 78.1 86.3 |
89.3 87.1 87.2 |
|
F4 |
E1 E2 E3 |
(200ms,50ms) |
82.6 81.8 79 |
87.5 83 82.6 |
80 77.8 76 |
88.1 86.3 85.3 |
|
F5 |
E1 E2 E3 |
(250ms,75ms) |
79.3 77.1 76.3 |
83.1 81.8 80.9 |
85.8 84.1 82.1 |
95.2 94.6 93.2 |
|
F5 |
E1 E2 E3 |
(200ms,50ms) |
79 76.8 75.1 |
81.8 80.1 80.3 |
84.3 83.6 81.8 |
94.8 93.7 91.8 |
Table 5. Classification accuracies of Ninapro DB2 for 12, 17, 23 classes
|
Feature |
Exercise |
(Win Len, Overlap) |
Classification Accuracy (%) |
|||
|
SVM |
KNN |
DT |
Ensem. Bagg. |
|||
|
F1 |
E1 E2 E3 |
(250ms,75ms) |
66.3 64.8 64.2 |
70.4 68.6 68.3 |
72.4 71.6 68.3 |
79.7 78.6 76.3 |
|
F1 |
E1 E2 E3 |
(200ms,50ms) |
65.9 63.6 62.1 |
70.1 68.3 67.1 |
71.8 71.1 70.3 |
78.4 77.1 76.1 |
|
F2 |
E1 E2 E3 |
(250ms,75ms) |
73.2 71.4 71.1 |
80.2 78.6 79 |
77.4 77.1 74.3 |
84.9 84.2 82.6 |
|
F2 |
E1 E2 E3 |
(200ms,50ms) |
72.1 70.2 70 |
79.1 78.1 78.3 |
76.3 76.1 73.3 |
84.3 84.1 81.3 |
|
F3 |
E1 E2 E3 |
(250ms,75ms) |
68.5 67.1 66.8 |
65 64.1 63.2 |
73.1 72.6 70.3 |
75.9 75.1 74.6 |
|
F3 |
E1 E2 E3 |
(200ms,50ms) |
67.7 66.3 65.9 |
64.2 63.6 62.2 |
72.1 71.9 70.1 |
74.8 74.9 73.2 |
|
F4 |
E1 E2 E3 |
(250ms,75ms) |
82.9 81.3 80.8 |
85.8 84.6 84.1 |
81.4 78.3 78.1 |
87.2 85.6 85.5 |
|
F4 |
E1 E2 E3 |
(200ms,50ms) |
81.7 80.7 80.2 |
84.7 83.5 83.9 |
80.2 77.9 7.5 |
86.7 85.5 85.3 |
|
F5 |
E1 E2 E3 |
(250ms,75ms) |
79.5 77.6 76.2 |
83.4 82.1 80.9 |
82.4 80.3 80.1 |
91.9 91.1 90.3 |
|
F5 |
E1 E2 E3 |
(200ms,50ms) |
79.5 77 76.1 |
83 81.9 80.7 |
82.1 80.1 79.4 |
91.3 90.9 89.7 |
Figure 2. Comparison chart of classification accuracies for Ninapro DB1 for different feature sets
Figure 3. Comparison chart of classification accuracies for Ninapro DB2 for different feature sets
Figure 4. Confusion matrix for 12 gestures of Ninapro DB1
Figure 5. Confusion matrix for 17 gestures of Ninapro DB1
Figure 6. Confusion matrix for 23 gestures of Ninapro DB1
The classification accuracy of Ninapro DB1 for window lengths of 250 ms and 200 ms and overlap of 75 ms and 50 ms is tabulated in Table 5 for all the features mentioned in Table 2. For Ninapro DB2, the features F4 and F5 give the highest classification accuracy for the bagged tree ensemble classifier compared to the features F1, F2, and F3. The performance of the KNN classifier is better than that of SVM for features F1, F2, F4, and F5. The SVM classifier is better than KNN for feature F3. A statistical significance test was conducted for all the feature-classifier combinations; the bagged ensemble classifier for the features F4 and F5 significantly outperformed (p<0.01) the other feature sets. The classification accuracy of the DT classifier for feature sets F1, F2, and F3 is higher than that of the SVM and KNN classifiers. For F4 and F5, the accuracies of SVM and KNN are higher than DT classifiers. Figures 2 and 3 show the classification accuracies of Ninapro DB1 and DB2.
The confusion matrix of Ninapro DB1 for the three exercises is given in Figures 4-6. This depicts the model performance for the feature F5 classified with the Ensemble Bagging classifier. In exercise E1, gesture number 8 is misclassified along with neighboring gestures. In exercise E2, gestures 17 and 21 are misclassified. In exercise E3, gestures 33 and 41 are misclassified along with neighboring gestures. All other gestures show better performance. The effect of window length on classification accuracy, according to the work done by Menon et al. [40], was investigated by adopting different window sizes for feature extraction. From the results based on the window length and overlap, a window length of 250 ms and 75 ms works better than a window length of 200 ms and 50 ms. The studies focused on classifying finger and wrist movements, which include the most complex muscle activations. To best utilize the processing capability of the prosthetic device, the overlapping windowing approach was proposed, in which the analysis window is increased by the processing delay, which is the time it takes to extract features and to make a decision. Also, this model can be compared with deep learning (DL) models with respect to its classification performance and the ability to extract both spatial and temporal information.
3.2.2 Model evaluation for 52 classes
The classification accuracies of Ninapro DB1 and DB2 for the 52 classes of movements for the window length of 250 ms and overlap of 75 ms are tabulated in Table 6. Using Ninapro DB1 and DB2, the features F4 and F5 show more promising results for the bagged ensemble classifier. The highest classification accuracy is reported for feature F5, followed by feature F4. The performance of SVM is better than the KNN classifier for the features F1, F2, F4, and F5 for Ninapro DB1. For feature F3, the KNN classifier gives better accuracy than SVM. For Ninapro DB2, for all the features, the performance of the SVM classifier is better than that of the KNN classifier.
Table 6. Classification accuracies of Ninapro DB1 and DB2 for 52 classes
|
Feature |
Ninapro DB1 |
Ninapro DB2 |
||||||
|
SVM |
KNN |
DT |
Bagged Ensemble |
SVM |
KNN |
DT |
Bagged Ensemble |
|
|
F1 |
78.5 |
76.3 |
80.9 |
83.4 |
76.4 |
73.9 |
80.3 |
81.4 |
|
F2 |
75.3 |
72.4 |
77.3 |
80.3 |
73.9 |
71.3 |
78.6 |
80.1 |
|
F3 |
76.4 |
79.3 |
81.4 |
84.9 |
77.4 |
72.9 |
80.1 |
82.9 |
|
F4 |
84.5 |
81.3 |
86.3 |
87.1 |
80.3 |
77.3 |
81.7 |
85.3 |
|
F5 |
87.6 |
85.7 |
90.1 |
92.9 |
86.6 |
83.4 |
88.3 |
90.8 |
Table 7. Computation time for different feature extraction schemes
|
Feature |
Computation Time (sec) |
|
F1 |
0.723 |
|
F2 |
0.752 |
|
F3 |
0.536 |
|
F4 |
0.874 |
|
F5 |
0.793 |
3.3 Computation time
The computation time for the feature extraction of the EMG signals is given below in Table 7.
The computation time for extracting F4 and F5 is higher than F1, F2, and F3. Efforts to mitigate this limitation could involve exploring advanced hardware acceleration techniques, parallel processing strategies, or implementing more efficient algorithms for feature extraction. The increased computational requirements may impact the real-time applicability of the system, particularly in situations where low latency is essential. Despite the current computational time constraint, the study lays the foundation for future research endeavors aimed at improving the efficiency of EMG-based hand gesture classification systems. Incorporating multi-modal data, such as combining EMG signals with other sensor data like accelerometers or inertial sensors, can potentially enhance the robustness and accuracy of prosthesis control systems.
3.4 Comparison with the literature
The proposed feature extraction is compared with the literature as given in Table 8. Atzori et al. [23] have proposed an EMG classification with Ninapro DB1 and DB2 and provided an accuracy of 75.32% and 75.27%, respectively. Wei et al. [21] utilized the same dataset for gesture classification and came up with an accuracy of 88.20% and 83.70%. Karnam et al. [13] utilized energy features and attained an accuracy of 88.8% for energy features. Pancholi et al. [22], by utilizing Ninapro DB1, achieved an accuracy of 92.18%. The modified time domain descriptor feature provides classification accuracies of 87.1% and 85.3% for Ninapro DB1 and DB2. The conditional spectral moments provide classification accuracies of 92.9% and 90.8% for Ninapro DB1 and DB2, respectively, which are higher than the literature.
Table 8. Comparison of the proposed features with the literature values
|
Authors |
Database |
Classes |
Accuracy (%) |
|
Atzori et al. [23] |
Ninapro DB1 |
50 |
75.32 |
|
Ninapro DB2 |
49 |
75.27 |
|
|
Wei et al. [21] |
Ninapro DB1 |
52 |
88.20 |
|
Ninapro DB2 |
50 |
83.70 |
|
|
Karnam et al. [13] |
Ninapro DB1 |
52 |
88.8 |
|
Pancholi et al. [22] |
Ninapro DB1 |
53 |
92.18 |
|
Proposed Feature F5 (Ensemble Bagging) |
Ninapro DB1 |
52 |
92.9 |
|
Ninapro DB2 |
50 |
90.8 |
In this work, the feature extraction and classification of myoelectric signals, with a primary focus on enhancing the accuracy of classifying hand gestures with multiple DoF for prosthetic applications, have been presented. The pivotal role of feature extraction in the success of myoelectric pattern recognition has been underscored, and in this work, two features, namely conditional spectral moments and an enhanced time-domain descriptor, have been proposed to achieve improved classification performance. The conditional spectral moments, derived from the time-frequency distribution of EMG signals, effectively captured variations in muscle activity and movement, providing valuable insights for the classification of intricate hand gestures. Furthermore, the enhanced time-domain descriptor obtained through convolution with wavelet filter coefficients successfully extracted both spatial and temporal characteristics of muscle activity, contributing to a more robust representation of the myoelectric signal. Ninapro DB1 and DB2 were used to evaluate the efficacy of the enhanced time domain descriptors and conditional spectral moment features by using the classifiers SVM, KNN, DT, and ensemble bagging. The classification accuracies for the proposed TD-PSD and conditional spectral moment feature have improved significantly, with 87.1% and 92.9%, respectively, for Ninapro DB1 and 85.3% and 90.8% for Ninapro DB2, respectively. These findings showcase the ability of the features to capture spatial and temporal information from the EMG signal and provide multiple DoFs for prosthetic users. By incorporating feature selection and dimensionality reduction, the extensive computation time for feature extraction can be reduced, making the model more practical and scalable. This study can be further extended for more complex movements with multiple degrees of freedom to handle intricate hand gestures and diverse movement patterns. The feature engineering in myoelectric applications thus paves the way for more accurate and effective prosthetic control systems.
|
k |
Frequency index |
|
Δ |
Derivative |
|
db44 |
Daubechies wavelet function |
|
p |
Probability under assumption |
Let $X[k]$ represent the corresponding Discrete Fourier Transform of the EMG input signal. It states that the sum of the square of $x[j]$ equals the sum of the square of $X[k]$. It can be described by the equation:
$\sum_{j=0}^{N-1}\left|x[j]^2\right|=\frac{1}{N} \sum_{k=0}^{N-1}\left|X[k] X^*[k]\right|=\sum_{k=0}^{N-1} P[k]$ (10)
$P[k]$ is the phase-excluded power spectrum derived from the multiplication of $X[k]$ by its conjugate, and ' $k$ ' is the frequency index. The power spectral density cannot be directly accessed from the time domain, both positive and negative frequencies should be dealt with, in significance with the symmetry property, and statistically, all the odd moments become zero. The $\mathrm{n} t h$-order moment $\mathrm{m}$ of $P[k]$ is given by the following equation:
$m_n=\sum_{k=0}^{N-1} k^n P[k]$ (11)
Eq. (10) uses Eq. (11) for n = 0 and non-zero values, it utilizes the time-differentiation property. According to this property, the nth derivative corresponds to multiplying the spectrum by kn for a discrete-time signal in the time domain.
Mathematically, this can be given as:
$F\left[\Delta^n x[j]\right\rceil=k^n X[k]$ (12)
The zero-order moment (power in frequency domain) or the strength of muscle contraction is stated as:
$f_1=m_0=\sqrt{\sum_{j=0}^{N-1} x[j]^2}$ (13)
Accordingly, the second and fourth-order moments can be given by the equations:
$f_2=m_2=\sqrt{\sum_{k=0}^{N-1} k^2 P[k]}=\sqrt{\frac{1}{N} \sum_{j=0}^{N-1} \Delta x[j]^2}$ (14)
A repetition of this yields the fourth-order moment.
$f_3=\sqrt{\sum_{k=0}^{N-1} k^4 P[k]}=\sqrt{\frac{1}{N} \sum_{j=0}^{N-1}\left(\Delta^2 x[j]\right)^2}$ (15)
Sparseness is a feature that states how much energy is packed within a few components. It is represented by:
$f_4=\log \left(\frac{m_0}{\sqrt{\left(m_0-m_2\right)\left(m_0-\left(m_4\right)\right.}}\right)$ (16)
The Irregularity Factor (IF) can be defined from the number of Zero Crossings (ZC) and the Number of Peaks (NP). Both the measures can be expressed merely by their spectral moments. The IF can be demonstrated expressively by:
$f_5=\log \left(\frac{Z C}{N P}\right)=\log \left[\frac{\sqrt{\frac{m_2}{m_0}}}{\sqrt{\frac{m_2}{m_2}}}\right]=\log \left(\frac{m_2}{\sqrt{m_0 m_4}}\right)$ (17)
The Waveform Length Ratio (WL) feature can be obtained by summing the absolute value of the EMG signal derivatives. It can be expressed mathematically as:
$f_6=\log \left(\frac{\sum_{j=0}^1|\Delta x|}{\sum_{j=0}^1\left|\Delta^2 x\right|}\right)$ (18)
[1] Rodríguez-Tapia, B., Soto, I., Martínez, D.M., Arballo, N.C. (2020). Myoelectric interfaces and related applications: Current state of EMG signal processing - A systematic review. IEEE Access, 8: 7792-7805. https://doi.org/10.1109/ACCESS.2019.2963881
[2] Parajuli, N., Sreenivasan, N., Bifulco, P., et al. (2019). Real-time EMG based pattern recognition control for hand prostheses: A review on existing methods, challenges and future implementation. Sensors, 19(20): 4596. https://doi.org/10.3390/s19204596
[3] Rafiee, J., Rafiee, M.A., Yavari, F., Schoen, M.P. (2011). Feature extraction of forearm EMG signals for prosthetics. Expert Systems with Applications, 38(4): 4058-4067. https://doi.org/10.1016/j.eswa.2010.09.068
[4] Khushaba, R.N., Al-Ani, A., Al-Timemy, A., Al-Jumaily, A. (2016). A fusion of time-domain descriptors for improved myoelectric hand control. In 2016 IEEE Symposium Series on Computational Intelligence, Athens, Greece, pp. 1-6. https://doi.org/10.1109/SSCI.2016.7850064
[5] Al-Timemy, A.H., Khushaba, R.N., Bugmann, G., Escudero, J. (2015). Improving the performance against force variation of EMG controlled multifunctional upper-limb prostheses for transradial amputees. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 24(6): 650-661. https://doi.org/10.1109/TNSRE.2015.2445634
[6] Kuzborskij, I., Gijsberts, A., Caputo, B. (2012). On the challenge of classifying 52 hand movements from surface electromyography. In 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, USA, pp. 4931-4937. https://doi.org/10.1109/EMBC.2012.6347099
[7] Waris, A., Niazi, I.K., Jamil, M., Englehart, K., Jensen, W., Kamavuako, E.N. (2018). Multiday evaluation of techniques for EMG-based classification of hand motions. IEEE Journal of Biomedical and Health Informatics, 23(4): 1526-1534. https://doi.org/10.1109/JBHI.2018.2864335
[8] Hudgins, B., Parker, P., Scott, R.N. (1993). A new strategy for multifunction myoelectric control. IEEE Transactions on Biomedical Engineering, 40(1): 82-94. https://doi.org/10.1109/10.204774
[9] Phinyomark, A., Scheme, E. (2018). An investigation of temporally inspired time domain features for electromyographic pattern recognition. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Honolulu, HI, USA, pp. 5236-5240. https://doi.org/10.1109/EMBC.2018.8513427
[10] Al-Timemy, A.H., Bugmann, G., Escudero, J., Outram, N. (2013). Classification of finger movements for the dexterous hand prosthesis control with surface electromyography. IEEE Journal of Biomedical and Health Informatics, 17(3): 608-618. https://doi.org/10.1109/jbhi.2013.2249590
[11] Chu, J.U., Moon, I., Lee, Y.J., Kim, S.K., Mun, M.S. (2007). A supervised feature-projection-based real-time EMG pattern recognition for multifunction myoelectric hand control. IEEE/ASME Transactions on Mechatronics, 12(3): 282-290. https://doi.org/10.1109/TMECH.2007.897262
[12] Tuncer, T., Dogan, S., Subasi, A. (2020). Surface EMG signal classification using ternary pattern and discrete wavelet transform based feature extraction for hand movement recognition. Biomedical Signal Processing and Control, 58: 101872. https://doi.org/10.1016/j.bspc.2020.101872
[13] Karnam, N.K., Turlapaty, A.C., Dubey, S.R., Gokaraju, B. (2021). Classification of sEMG signals of hand gestures based on energy features. Biomedical Signal Processing and Control, 70: 102948. https://doi.org/10.1016/j.bspc.2021.102948
[14] Xiao, F., Yang, D., Lv, Z., Guo, X., Liu, Z., Wang, Y. (2020). Classification of hand movements using variational mode decomposition and composite permutation entropy index with surface electromyogram signals. Future Generation Computer Systems, 110: 1023-1036. https://doi.org/10.1016/j.future.2019.11.025
[15] Jabbari, M., Khushaba, R., Nazarpour, K. (2021). Spatio-temporal warping for myoelectric control: An offline, feasibility study. Journal of Neural Engineering, 18(6): 066028. https://doi.org/10.1088/1741-2552/ac387f
[16] Alkan, A., Günay, M. (2012). Identification of EMG signals using discriminant analysis and SVM classifier. Expert Systems with Applications, 39(1): 44-47. https://doi.org/10.1016/j.eswa.2011.06.043
[17] Savalia, S., Emamian, V. (2018). Cardiac arrhythmia classification by multi-layer perceptron and convolution neural networks. Bioengineering, 5(2): 35. https://doi.org/10.3390/bioengineering5020035
[18] Paul, Y., Goyal, V., Jaswal, R.A. (2017). Comparative analysis between SVM & KNN classifier for EMG signal classification on elementary time domain features. In 2017 4th International Conference on Signal Processing, Computing and Control (ISPCC), Solan, India, pp. 169-175. https://doi.org/10.1109/ISPCC.2017.8269670
[19] Yaman, E., Subasi, A. (2019). Comparison of bagging and boosting ensemble machine learning methods for automated EMG signal classification. BioMed research International, 2019: 9152506. https://doi.org/10.1155/2019/9152506
[20] Khushaba, R.N., Al-Timemy, A.H., Samuel, O.W., Scheme, E.J. (2022). Myoelectric control with fixed convolution-based time-domain feature extraction: Exploring the spatio–temporal interaction. IEEE Transactions on Human-Machine Systems, 52(6): 1247-1257. https://doi.org/10.1109/THMS.2022.3146053
[21] Wei, W., Dai, Q., Wong, Y., Hu, Y., Kankanhalli, M., Geng, W. (2019). Surface-electromyography-based gesture recognition by multi-view deep learning. IEEE Transactions on Biomedical Engineering, 66(10): 2964-2973. https://doi.org/10.1109/TBME.2019.2899222
[22] Pancholi, S., Joshi, A.M., Joshi, D. (2022). DLPR: Deep learning-based enhanced pattern recognition frame-work for improved myoelectric prosthesis control. IEEE Transactions on Medical Robotics and Bionics, 4(4): 991-999. https://doi.org/10.1109/TMRB.2022.3216957
[23] Atzori, M., Cognolato, M., Müller, H. (2016). Deep learning with convolutional neural networks applied to electromyography data: A resource for the classification of movements for prosthetic hands. Frontiers in Neurorobotics, 10: 9. https://doi.org/10.3389/fnbot.2016.00009
[24] Tsinganos, P., Cornelis, B., Cornelis, J., Jansen, B., Skodras, A. (2019). Improved gesture recognition based on sEMG signals and TCN. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing, Brighton, UK, pp. 1169-1173. https://doi.org/10.1109/ICASSP.2019.8683239
[25] Côté-Allard, U., Fall, C.L., Drouin, A., et al. (2019). Deep learning for electromyographic hand gesture signal classification using transfer learning. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 27(4): 760-771. https://doi.org/10.1109/TNSRE.2019.2896269
[26] Young, A.J., Smith, L.H., Rouse, E.J., Hargrove, L.J. (2012). Classification of simultaneous movements using surface EMG pattern recognition. IEEE Transactions on Biomedical Engineering, 60(5): 1250-1258. https://doi.org/10.1109/TBME.2012.2232293
[27] Scheme, E., Englehart, K. (2011). Electromyogram pattern recognition for control of powered upper-limb prostheses: State of the art and challenges for clinical use. Journal of Rehabilitation Research & Development, 48(6): 643-659. https://doi.org/10.1682/jrrd.2010.09.0177
[28] Turlapaty, A.C., Gokaraju, B. (2019). Feature analysis for classification of physical actions using surface EMG data. IEEE Sensors Journal, 19(24): 12196-12204. https://doi.org/10.1109/JSEN.2019.2937979
[29] Du, S., Vuskovic, M. (2004). Temporal vs. spectral approach to feature extraction from prehensile EMG signals. In Proceedings of the 2004 IEEE International Conference on Information Reuse and Integration, Las Vegas, NV, USA, pp. 344-350. https://doi.org/10.1109/IRI.2004.1431485
[30] Côté-Allard, U., Gagnon-Turcotte, G., Laviolette, F., Gosselin, B. (2019). A low-cost, wireless, 3-D-printed custom armband for sEMG hand gesture recognition. Sensors, 19(12): 2811. https://doi.org/10.3390/s19122811
[31] Côté-Allard, U., Campbell, E., Phinyomark, A., Laviolette, F., Gosselin, B., Scheme, E. (2020). Interpreting deep learning features for myoelectric control: A comparison with handcrafted features. Frontiers in Bioengineering and Biotechnology, 8: 158. https://doi.org/10.3389/fbioe.2020.00158
[32] Loughlin, P.J. (2001). What are the time-frequency moments of a signal? IAdvanced Signal Processing Algorithms, Architectures, and Implementations XI, 4474: 35-44. https://doi.org/10.1117/12.448676
[33] Atzori, M., Gijsberts, A., Heynen, S., et al. (2012). Building the Ninapro database: A resource for the biorobotics community. In 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics, Rome, Italy, pp. 1258-1265. https://doi.org/10.1109/BioRob.2012.6290287
[34] Atzori, M., Gijsberts, A., Castellini, C., et al. (2014). Electromyography data for non-invasive naturally-controlled robotic hand prostheses. Scientific Data, 1(1): 140035. https://doi.org/10.1038/sdata.2014.53
[35] Rafiee, J., Rafiee, M.A., Prause, N., Schoen, M.P. (2011). Wavelet basis functions in biomedical signal processing. Expert Systems with Applications, 38(5): 6190-6201. https://doi.org/10.1016/j.eswa.2010.11.050
[36] Triwiyanto, T., Wahyunggoro, O., Nugroho, H.A., Herianto, H. (2017). An investigation into time domain features of surface electromyography to estimate the elbow joint angle. Advances in Electrical and Electronic Engineering, 15(3): 448-458. https://doi.org/10.15598/aeee.v15i3.2177
[37] Phinyomark, A., Khushaba, R., Scheme, E. (2018). Feature extraction and selection for myoelectric control based on wearable EMG sensors. Sensors, 18(5): 1615. https://doi.org/10.3390/s18051615
[38] Khushaba, R. (2023). Feature Extraction Using Multisignal Wavelet Transform Decom. https://github.com/RamiKhushaba/getmswtfeat, accessed on July, 2023
[39] Subasi, A., Qaisar, S.M. (2022). Surface EMG signal classification using TQWT, Bagging and Boosting for hand movement recognition. Journal of Ambient Intelligence and Humanized Computing, 13(7): 3539-3554. https://doi.org/10.1007/s12652-020-01980-6
[40] Menon, R., Di Caterina, G., Lakany, H., Petropoulakis, L., Conway, B.A., Soraghan, J.J. (2017). Study on interaction between temporal and spatial information in classification of EMG signals for myoelectric prostheses. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(10): 1832-1842. https://doi.org/10.1109/TNSRE.2017.2687761
