Dexterous hand gestures recognition based on low-density sEMG signals for upper-limb forearm amputees

Abstract

Introduction

Intuitive prosthesis control is one of the most important challenges in order to reduce the user effort in learning how to use an artificial hand. This work presents the development of a novel method for pattern recognition of sEMG signals able to discriminate, in a very accurate way, dexterous hand and fingers movements using a reduced number of electrodes, which implies more confidence and usability for amputees.

Methods

The system was evaluated for ten forearm amputees and the results were compared with the performance of able-bodied subjects. Multiple sEMG features based on fractal analysis (detrended fluctuation analysis and Higuchi’s fractal dimension) combined with traditional magnitude-based features were analyzed. Genetic algorithms and sequential forward selection were used to select the best set of features. Support vector machine (SVM), K-nearest neighbors (KNN) and linear discriminant analysis (LDA) were analyzed to classify individual finger flexion, hand gestures and different grasps using four electrodes, performing contractions in a natural way to accomplish these tasks. Statistical significance was computed for all the methods using different set of features, for both groups of subjects (able-bodied and amputees).

Results

The results showed average accuracy up to 99.2% for able-bodied subjects and 98.94% for amputees using SVM, followed very closely by KNN. However, KNN also produces a good performance, as it has a lower computational complexity, which implies an advantage for real-time applications.

Conclusion

The results show that the method proposed is promising for accurately controlling dexterous prosthetic hands, providing more functionality and better acceptance for amputees.

Keywords
Electromyography; Upper-limb prosthesis; Low-density surface electromyography; Dexterous hand gestures; Pattern recognition

Introduction

An ideal upper-limb prosthesis should be recognized as a natural part of the amputee body, supplying motor and sensorial limb abilities (Englehart and Hudgins, 2003). Nevertheless, one of the major problems is the user’s acceptance after starting the training process to use prostheses. Some of the common factors of their rejection are their facilities and comfort, exterior appearance, but most of all, its functionality (Peerdeman et al., 2011). An evaluation of the important daily live activities for prosthetic hand users was presented by Peerdeman et al. (2011). Opening/closing zipper, making the bed, grasping a glass, catching a ball, and using knife and fork were activities considered as relevant. As a result, grasp tasks were found to be more important than wrist movements, being lateral, cylindrical and tripod grasps the most important ones (Sensinger et al., 2009). Other study with prosthesis users (Zecca et al., 2002) reports that 100% of interviewed would like to point the finger, 90% wanted to have individual fingers control and 70% considered useful to have wrist flexion/extension. However, most studies have focused on recognizing power functions and wrist movements, while dexterous movements of prostheses have not been widely addressed, being a lack to improve their functionality. As a result, there is a need for a more functional and reliable control system, and using a minimum number of surface Electromyographic (sEMG) electrodes for upper-limb amputees.

Prosthesis control can be divided into two subsystems: high–level control (HLC) and low–level control (LLC). HLC interprets the subject’s intentions gathered from patterns extracted from sEMG signals, whereas LLC takes the output of the HLC as a set point to control angles of joint prosthesis. The real challenge for researchers is in HLC, in which intuitive prosthetic control is one of the most important challenges, in order to reduce the user’s learning effort for the artificial hand adaptation. In this aspect, myoelectric control may be considered quite suitable as it allows a more intuitive user prosthesis interface in a more natural way. However, the still unskillful control, lack of feedback and training of these prostheses make them unacceptable to several users.

A great variety of methods for feature extraction in time and frequency domains have been explored to recognize sEMG patterns (Zecca et al., 2002). Several works have used magnitude-based features to feed classifiers to recognize hand motor tasks involving elbow, forearm, wrist and open/close hand movements (Guo et al., 2015; Oskoei and Hu, 2008; Phinyomark et al., 2009). Other systems have got user’s commands for a limited number of hand and individual finger gestures (Naik et al., 2010, 2009; Peleg et al., 2002; Tsenov et al., 2006), and even for combined finger movements (Khushaba et al., 2012). However, such attempts did not included more complex dexterous movements, in which statistical-based features are not sufficiently reliable due to the weak signals from these movements. Previous works did consider sEMG signals from dexterous movements, but only for grasp gestures, aiming to improve the functionality of the prosthetic control (Chu and Lee, 2009; Hargrove et al., 2009; Khezri and Jahed, 2007; Tommasi et al., 2013; Wang et al., 2013). In fact, the non-linear relationship between force and electric activity of muscles at low-levels of contraction (Naik et al., 2010) makes much more difficult the sEMG signals analysis. Techniques based on fractals dimensions (FD) have been used to estimate the non-linear properties of sEMG, which present sensibility at frequency and magnitude to the strength of muscle contraction (Arjunan and Kumar, 2007). Recently, a combination of Higuchi’s fractal dimension (HFD) and detrended fluctuation analysis (DFA) were more employed on sEMG signals (Guo et al., 2015), in order to measure the non-linear property in sEMG signals, with the advantages of features from time and frequency domains. Another factor that bother the prosthesis users is the high number of electrodes, as the training with many input channels is a long and hard process, resulting in their decision to use only a limited and very simple prosthesis (2-3 degrees of freedom). Moreover, prosthesis with electrode array are complex, in addition to the fact that differences in electrode placements lead to variations in sEMG signals and their spectrum (Kumar et al., 2013). Some works have sought for systems with low-density (less than six channels) sEMG signals (Arjunan and Kumar, 2010; Castro et al., 2015; Phinyomark et al., 2012b), which reduce problems as electrode fixation and computational demand. However, the accuracy reported by these researchers to recognize dexterous movements are still poor and their experiments were only conducted in able-bodied subjects.

Some studies with amputees using few number of electrodes have been conducted in order to fulfil this gap, such as done in Al-Timemy et al. (2013), Cipriani et al. (2011), Li et al. (2011), Kumar et al. (2013) and Tenore et al. (2009). In particular, in Kumar et al. (2013) a method based on wavelet maxima density was proposed as a non-linear parameter to extract relevant from sEMG signals using only one channel, but no grasp gestures were considered. Grasp gestures were did consider in Li et al. (2011), but using high-density sEMG signals (twelve electrodes). Nevertheless, in all these previous works conducted with amputees, a common lack of experiments with hand, wrist and individual finger gestures can be noticed, and few of these works have even included dexterous movements, and only for high-density. In fact, to the best of our knowledge, no studies with low-density for dexterous movements on amputees can be found in the literature. Table 1 shows very complete details about each one of these works, which are classified according to the year of the research, number of channels, number of gestures, kind of tasks, signal features, classifiers, window length for sEMG analysis and duration of the window overlapping.

Table 1
Relevant researches for hand gesture recognition using sEMG signals.

Authors	Year	Ch	G	Tasks	Features	Feature selection	Classifier	W/O [ms]	Sub.
Peleg†	2002	2	5	F1-F5	DFT, AR, Binary electrode activation	GA	KNN	1,2	4 C
Englehart	2003	4	4	WF, WE, RD, UD	ZC, SSC, WL, MAV			256/16	12 C
Tsenov†	2006	2	4	F1, F2, F3, HC	MAV,VAR, WL, Norm, ZC, Absolute Max and Min, Max-Min, Med		ANN (MLP, RBF, LVQ)		1 C
Khezri† ‡	2007	2	6	HO, HC, F1, TP, WF, WE	MAV, SSC, AR, DWT		Neuro-fuzzy system	200/50	4C
Oskoei and Hu	2008	4	5	Isotonic: HF, HE, hand straight, hand abduction and adduction	MAV, RMS, WL, VAR, ZC, SSC, WAMP, MAV1, MAV2, PS, AR-2 and AR-6, MNF, MNF		SVM, LDA, MLP		11 C
Phinyomark	2009			WF, WE, HC, HO, FP, FS	TD and FD (16)		LDA		3 C
Tenore†	2009	19	12	FFI, FEI, F345	MAV, VAR, WL, WAMP	PCA	MLP	200/25	5 C 1 A
Chu and Lee‡	2009	4	10	WF, WE, radial and ulnar flexion of the wrist, HO,CG, LT,RS	AR-4, ZC, WL, SSC, MAV		GMM		10 C
Naik†	2010	2	3	F3, F4, F5	ICA, RMS		ANOVA		7 C
Cipriani†	2011	8	7	F1,F2,F345,F2345,TO,TR,LT	MAV		KNN	50/50	5 C 5 A
Li‡	2011	12	11	EFE,WF, WE, WP, WS, HO,TP, Power Grip, Tool Grip, RS	MAV, ZC, WL, SSC		LDA		5 C 8 A
Phinyomark	2012a	1	2	FP, FS, WF, WE, HO, HC	DFA and TD		LDA		20 C
Khushaba†	2012	2	10	F1-F5, F12, F13, F14, F15, HC	SSC, ZC, WL, HTD, Sample Skewness, AR	LDA	SVM-libsvm and KNN	100	8 C
Phinyomark	2012b	5	8	FP, FS, HO, HC, WE, WF, RD, UD	TD and FD (37)		LDA		20 C
Al-Timemy†	2013	6	12	F1-F5, E1-E5, TA, F12, F234, F1234, RS	TD-AR	PCA OFNDA	LDA and SVM		10 C 6 A
Kumar†	2013	1	4	F1, F2, F3, F4, RS	Wavelet Maxima Density		T-SVM	300	11 C 1 A
Tommasi† ‡	2013	10	6	TP, LT, TR, PW, PEG, prismatic 4 fingers grasp	MAV		LS-SVM		27 C
Wang‡	2013	2	8	CG, HG, LT, PT, SG, TR, TP, RS	MAV, VAR, AR-4, ZC, MNF, MDF	LDA	LDA	256	6 C
Castro‡	2015	5	10	TP, TR, HC, HO, RS	PSD-Av		FLD		4 C
Guo	2015	6	8	EFE, FP, FS, WF, WE, Wab, Wad	WP, DFA and Muscle Models		SVM and ANN		7 C

• ^†
  works including finger movements;
• ^‡
  works including grasp gestures;
• Ch: number of channels; G: number of gestures; W/O: window length/overlapping; Sub: number of subjects; C: control group; A: amputee; F1-F5: flexion of finger 1 to finger 5 and combinations; E1-E5: extension of finger 1 to finger 5; WF: wrist flexion; WE: wrist extension; RD: radial deviation; UD: ulnar deviation; HC: hand close; HO: Hand open; HF: hand flexion; HE: hand extension; FP: forearm pronation; FS: forearm supination; FFI: finger flexion individually; FEI: finger extension individually; RS: rest state; TO: thumb opposition; TP: tip pinch; LT: lateral grasp; CG: cylindrical grasp; TR: tripod; EFE: elbow flexion and extension; WP: wrist pronation; WS: wrist supination; TA: thumb abduction; PW: power grasp; PEG: parallel extension grasp; HG: hook grasp; PT: point; SG: spherical grasp; Wab: wrist abduction; Wad: wrist adduction; TD: time domain; FD: frequency domain; PSAD-Av: Power Spectral Density Average; DFT: discrete Fourier transform; DWT: discrete wavelet transform; WAMP: amplitude Willison; ICA: independent component analysis; HTD: Hjorth Time Domain; MAV: mean absolute value; MAV1: modified mean absolute value 1; MAV2: modified mean absolute value 2; VAR: variance of sEMG; RMS: root mean square; WL: waveform length; ZC: zero crossing; SSC: slope sign change; AR: autoregressive model; MNF: mean frequency; MDF: median frequency; PKF: peak frequency; MNP: mean power; TTP: total power; PSR: power spectrum ratio; DFA: detrended fluctuation analysis; HFD: Higuchi fractal dimension; PS: power spectrum; GA: genetic algorithm; PCA: principal component analysis; OFNDA: orthogonal fuzzy neighborhood discriminant analysis; ANN: artificial neural network; MLP: multilayer perceptron; LVQ: learning vector quantization; RBF: radial basis function; LDA: linear discriminant analysis; KNN: K-nearest neighbors; SVM: support vector machine; T-SVM: twin-SVM; LS-SVM: least-squares SVM; FLD: Fisher’s linear discriminant; GMM: Gaussian mixture model; ANOVA: analysis of variance.

In summary, although these studies have identified different hand gestures and finger movements, a common aspect of them is the non-inclusion of forearm amputees, which is the focus of our study. In our research, we propose a system to recognize accurate dexterous gestures from amputees using low-density sEMG signals (four electrodes), in order to improve the functionality of upper-limb prostheses. Individual finger movements, hand gestures and grasp tasks are characterized using fractal-based analysis combined with a suitable election of others well-known features on time and frequency domains. The extent of dimension reduction is also investigated, considering seventeen features from the sEMG signals, and, in addition, the efficiency of three supervised classifiers, in off-line mode, is compared as well. From our study, a new method composed of a unique feature set and one classifier is introduced here, with the aim of obtaining a reliable control, to provide more functionality for forearm amputees in a more natural way to control a multifunction myoelectric prosthesis.

Methods

Four electrodes were used in this study to improve the functionality of upper-limb prostheses. The method is based on a three stages structure, such as feature extraction, feature selection, and pattern classification.

The first stage (“Feature Extraction”) is used to extract the sEMG patterns. Individual finger and hand movements are characterized using fractal-based analysis combined with others well-known features on time and frequency domains. Here, the sEMG signals are considered as a stationary process considering short time segments between 200 and 500 ms. This technique leads to generate a stream of sEMG patterns, in which short data segments are processed using independents windows. Choosing window length is crucial for on-line implementations. For this, an overlapping window was used, which allows an increase of the number of experimental trials, increasing their statistical dependence (Merletti and Parker, 2004).

The second stage (“Feature Selection”) is used to investigate the extent of dimension reduction, considering seventeen features from the sEMG, which is done because there is a strong correlation between the input data sets, so that the same information is repeated several times. Two techniques for dimensionality reduction were compared to obtain the suitable method, which consist of selecting an appropriate subset of the input data, discarding the redundant or low contribution features. For this, error of classification has been used as the criteria for minimization. An iterative process for analyzing all possible subsets is also carried out to select the best subset of features.

The third stage (“Classification”) is used to compare the efficiency of three supervised classifiers, which were validated in off-line execution.

Finally, the proposed system, composed of a unique feature set and one classifier, was validated with a group of amputees and compared with a control group.

Subjects

This study was conducted on a control group and a group of amputees. sEMG signals were recorded from the dominant forearm of ten able-bodied subjects (five males and five females), aged 22-35 years with no history of neurological or neuromuscular disorders, and ten forearm amputee subjects from the Rehabilitation Physical Center of Espírito Santo State (CREFES/Brazil). All amputees had traumatic amputation. The amputees (five females and five males), aged 19-64, are described in both, Table 2 and Figure 1a. The amputee subjects were previously evaluated by a physiotherapist, with an assessment including aspects as participant identification and physical examination (anamnesis, inspection, palpation, range of motion and sensitivity). All subjects did not have any experience of attending this kind of research study before. The inclusion criteria adopted in this research were as follows: no evidence, in their medical history, of peripheral neuropathy, diseases of the central nervous system and restricted mobility. All participants were informed about the objectives and methodology of the study, through oral presentation. After knowing the detailed procedures, the participants signed the free consent form, according to the ethical principles of the Federal University of Espirito Santo (UFES/Brazil). The study was approved by the Human Ethics Committee of UFES (protocol number 214/10).

Table 2
Participant demographics. Level of amputation is indicated.

Subject	Gender/age	Missing Hand	Time since amputation	Prosthesis used	Level of amputation
A1	Female/45	Right	4 years	Esthetic	WD
A2	Male/64	Left	42 year	Esthetic	WD
A3	Female/48	Right	1 year	Non	WD
A4	Female/23	Right	4 years	Non	TR
A5	Female/48	Right	2 years	Non	WD
A6	Female/50	Left	25 years	Non	TR‡
A7	Male/34	Both†	2 years	Non	WD
A8	Male/21	Left	2 years	Non	WD
A9	Male/27	Both†	1 year	Non	TR
A10	Male/24	Right	1 year	Non	TR

• ^†
  Bilateral amputation, not same level on both sides.
• ^‡
  Cause of amputation due to poor circulation.
• WD: wrist disarticulation; TR: transradial.

Figure 1
(a) Group of amputees who participated voluntarily of the experiments; (b) Different kinds of individual finger movements and grasp gestures considered in this study (rest state picture is not included); (c) Forearm muscles by channel adopted in the experimental protocol and the position of the electrodes. Source: adapted from E-Hand.com (2017).

Equipment and electrode placement

sEMG data were acquired using reusable bipolar active electrodes (PL091060A - 60Hz) manufactured by Touch Bionics, with inbuilt 60 Hz notch filter, pre-amplification and conditioning circuits, with adjustable gain. The sEMG signals were sampled (1 kHz) via an NI USB-6009 data acquisition system. In all experiments, it was necessary to ensure that the pressure of the electrode’s contact surfaces on the participant’s skin was evenly dispersed and that the electrodes were placed on skin areas with similar characteristics. All the electrodes were placed according to SENIAM recommendations (Hermens et al., 2000). Prior to place the electrodes over the muscle, the skin was previously cleaned with 70% alcohol, and conductive gel was used before attaching the electrodes, in order to reduce skin impedance. Four electrodes were placed on the selected muscles, according to their relation with the hand movements: flexor pollicis longus (FPL) (channel 1), flexor digitorum superficialis (FDS) (channel 2), wrist flexors (WF) (flexor carpi radialis and flexor carpi ulnaris muscles, channel 3) and wrist extensors (WE) (extensor carpi radialis longus, extensor carpi radialis brevis and extensor carpi ulnaris muscles, channel 4). The electrode placement procedures were the same for able-bodied subjects and amputees. A minimal inter-electrode distance of 2 cm was used for each subject.

Experimental protocol

Subjects performed each motor task (shown in Figure 1b) separately. Thirteen gestures were considered in this study, arranged into three task categories: the category A (CA) for individual fingers, which includes thumb flexion (F1), index flexion (F2), middle flexion (F3), ring flexion (F4), little flexion (F5), hand close (HC) and hand open (HO); the category B (CB), for hand grasp, which was arranged within the following taxonomy: into power, intermediate and precision grasps. As power grasps, two kind of full hand wrap grasps were considered: large diameter (LD: diameter = 90 cm, height = 110 cm) and medium diameter (MD: diameter = 50 cm, height = 140 cm). The lateral grasp (LT) was included as intermediated grasp. For the precision grasps, tripod grasp (TR) and tip pinch (TP) were considered. Finally, category C (CC), which includes all the thirteen gestures. The rest state (RT) was included in all categories as a motor task.

For the experiments, the subjects were seated in a chair with both arms on a table. Prior to the recording, the subjects were encouraged to familiarize themselves with the experimental protocol. The amputee subjects were asked to produce muscle contractions while they imagined specific movements with their phantom stump. At the same time, they performed a mirrored bilateral movement with the intact limb to facilitate the contraction of the affected side. During the experiments, the subjects were instructed by both visual and oral cues to elicit a contraction from the rest state and hold that task for 6 s, followed by a background activity (rest state) of 4 s, switching between isometric contraction and relaxation. Each hand gesture was repeated five times consecutively, with a resting period of 3 minutes between each gesture, in order to avoid fatigue. Each performed repetition and background period are referred to in this paper as “trial”. Within each trial, the contraction period was split roughly into a phase of onset and a subsequent steady-state phase. Moreover, to enhance generalization ability due to the fluctuation of sEMG signal, the experiments were repeated on three different days.

Data window selection

All the raw signals were preprocessed to remove unwanted DC level. Afterwards, the steady-state sEMG signal (isometric task) was extracted from each trial to avoid a transitional stage (i.e. during movement changes), taking one second after the presentation of the cue, until one second before finishing the motor task, and returning to the rest state. This process is made manually because of the low amplitude of the background activity, which is similar to the isometric activity itself and noise magnitude. Before feature extraction, the sEMG signals were windowed, and from each window a control command is conveyed. An evaluation of the effects of the window length and its overlapping on the classification accuracies from all subjects was also performed in this research, which was performed by comparison of the classification accuracies using all features included in this research. Average results for all classifiers implemented in this study were computed. All these parameters agrees with the real-time constraint required by human being, whose response time should be less than 300 ms, in order to not introduce a perceivable delay by the user (Englehart and Hudgins, 2003).

Feature extraction

Seventeen features were considered in this study, which are based on time domain (TD), frequency domain (FD) (Phinyomark et al., 2013; Zecca et al., 2002) and non-linear analysis related to Fractals (Esteller et al., 2001; Phinyomark et al., 2012b), as shown in Table 3. The mathematical definitions of DFA and HFD features are broadly described in Esteller et al. (2001) and Phinyomark et al. (2012b), respectively.

Table 3
Mathematical definition of features selected for this study, split in groups according its domain.

Domain	Feature	Parameters	d/c	Mathematical definition
Time domain	Mean Absolute Value (MAV)	-	1	$\text{MAV=}\frac{\text{1}}{\text{N}}{\displaystyle \sum _{n=1}^N\left|X_{\text{n}}\right|}$
	Modified Mean Absolute Value 1 (MAV1)	-	1	$\text{MAV1=}\frac{\text{1}}{\text{N}}{\displaystyle \sum _{n=1}^N\left|X_{\text{n}}\right|}$ $w_n\text{=}\{\begin{array}{c}\text{1}\text{,}if\text{0}\text{.25}N\le n\le \text{0}\text{.75}N\\ \text{0}\text{.5}\text{,}otherwise\end{array}$
	Modified Mean Absolute Value 2 (MAV2)	-	1	$\text{MAV2=}\frac{\text{1}}{\text{N}}{\displaystyle \sum _{n=1}^N\left|X_{\text{n}}\right|}$ $w_n\text{=}\{\begin{array}{c}\text{1}\text{,}if\text{0}\text{.25}N\le n\le \text{0}\text{.75}N\\ \text{4n/}N\text{,}if\text{0}\text{.25}N\text{<}n\\ \text{4(1-n}\text{/}N\text{)}\text{,}if\text{0}\text{.75}N\text{>}n\end{array}$
	Variance of sEMG (VAR)	-	1	$\text{VAR=}\frac{\text{1}}{\text{N-1}}{\displaystyle \sum _{n=1}^N{X}_{\text{n}}{}^{\text{2}}}$
	Root Mean Square (RMS)	-	1	$\text{RMS=}\sqrt{\frac{\text{1}}{\text{N}}{\displaystyle \sum _{n=1}^N{X}_{\text{n}}{}^{\text{2}}}}$
	Waveform Length (WL)	-	1	$\text{WL=}{\displaystyle \sum _{n=1}^{N-1}\left|X_{\text{n+1}}\text{-}{X}_{\text{n}}\right|}$
	Zero Crossing (ZC)	thld = 0.0005	1	$\text{ZC=}{\displaystyle \sum _{n=1}^{N-1}\left[sgn\left({\text{X}}_{\text{n}}{\text{×X}}_{\text{n+1}}\right)\cap \left|{\text{X}}_{\text{n}}{\text{-X}}_{\text{n+1}}\right|\ge \text{thld}\right]}$ $\text{sgn}\left(x\right)\text{=}\{\begin{array}{c}\text{1}\text{,}ifx\ge thld\\ \text{0}\text{,}otherwise\end{array}$
	Slope Sign Change (SSC)	thld = 0.001	1	$SSC={\displaystyle \sum }_{n=1}^{N-1}f\left[\left(X_n-X_{n-1}\right)\times \left(X_n-X_{n+1}\right)\right]$ $f\left(x\right)=\{\begin{array}{c}\mathrm{1,}ifx\ge thld\\ \mathrm{0,}otherwise\end{array}$
	Autoregressive Model (AR)	p = 4	4	$x_i={\displaystyle \sum }_{p=1}^P{a}_p{x}_{i-p}+w_i$
Frequency domain	Mean Frequency (MNF)	-	1	$MNF={\displaystyle \sum }_{j=1}^M{f}_j{P}_j/{\displaystyle \sum }_{j=1}^M{P}_j$
	Median Frequency (MDF)	-	1	${\displaystyle \sum }_{j=1}^{MDF}{P}_j={\displaystyle \sum }_{j=MDF}^M{P}_j=\frac{1}{2}{\displaystyle \sum }_{j=1}^M{P}_j$
	Peak Frequency (PKF)	-	1	$PKF=\max \left(P_j\right),j=\mathrm{1,...,}M$
	Mean Power (MNP)	-	1	$MNF={\displaystyle \sum }_{j=1}^M{P}_j/M$
	Total Power (TTP)	-	1	$TTP={\displaystyle \sum }_{j=1}^M{P}_j$
	Power Spectrum Ratio (PSR)	r = 20	1	$PSR={\displaystyle \sum }_{j=f_0-r}^{f_0+r}{P}_j/{\displaystyle \sum }_{j=-\infty }^{\infty }{P}_j$
Fractal features	Detrended Fluctuation Analysis (DFA)	L = 10	1	(Phinyomark et al., 2012b)
	Higuchi Fractal Dimension (HFD)	Kmax = 10	1	(Esteller et al., 2001)

• d/c: dimensions per channel; Xn represents the nth sample of the sEMG signal in a window segment; N denotes the number of sample of the sEMG signal; wn is the continuous weighting window function; thld denotes a threshold used to avoid low-voltage fluctuations or background noises; r is the order of AR; Pj is the sEMG power spectrum at the frequency j; M is the length of the sEMG power spectrum; f0 is a feature value of PKF; r is the integral limit of the ratio (r = 20); Po is nearby the maximum value of the sEMG power spectrum; P is the whole energy of the sEMG power spectrum in a range of 10 to 500 Hz. L: number of equal-size boxes; Kmax: maximum normalization factor. MAV: mean absolute value; MAV1: modified mean absolute value 1; MAV2: modified mean absolute value 2; VAR: variance of sEMG; RMS: root mean square; WL: waveform length; ZC: zero crossing; SSC: slope sign change; AR: autoregressive model; MNF: mean frequency; MDF: median frequency; PKF: peak frequency; MNP: mean power; TTP: total power; PSR: power spectrum ratio; DFA: detrended fluctuation analysis; HFD: Higuchi fractal dimension.

It is worth mentioning that the fractal dimension estimates the fractional dimension of the waveform signal in the time domain, which is considered as a geometric figure, quite useful for transient detection. In this aspect, detrended fluctuation analysis (DFA) is one of the most frequently used fractal time-series algorithms, which explores the non-stationary properties of sEMG signals with computational simplicity. DFA is a modified root mean square that provides a self-similarity parameter representing the fractal dimension. This scaling exponent indicates the presence of fractal scaling in a detrended time series of the RMS fluctuation in a succession of random division of the integrated sEMG signal on the time domain (Phinyomark et al., 2012b). DFA offers advantage over methods based on wavelet transformations in the time-scale domain (Phinyomark et al., 2012b). On the other hand, HFD (Esteller et al., 2001) is one of the most used fractal dimension feature, as it has shown better performance than other fractal methods (Esteller et al., 2001), and has also shown good performance in the classification of EMG signals.

Each feature was normalized individually based on the average and standard deviation values. For each data window, the features extracted from all channels were concatenated, which yielded a twenty-dimensional feature vector per channel. It is important to remark that AR coefficients are considered here as four-dimension feature for each channel.

Feature selection

It is known that multiple feature sets are more feasible to accomplish a high accuracy for the classifier. However, despite the analysis of all aforementioned features may provide redundant information, too much features will increase the computation cost. Hence, it is imperative to use techniques for feature reduction for optimal subsets in a point of view of class separability. Two methods were considered in this study: genetic algorithms (GA) and sequential forward selection (SFS). A GA approach (Huang and Wang, 2006) for feature selection is here used as an alternative to the conventional heuristic method SFS (Theodoridis and Koutroumbas, 2008). In the case of SFS method, the classification accuracy is used as selection criteria, thus, an exhaustive analysis was conducted. Thus, GA obtains the optimal subset after a series of iterations, being efficient with large search spaces and less chance to get local optimal solution than other algorithms. In addition, a fitness function assesses the mutual information between features and the output based on the entropy criteria. This approach is based on the feature space distribution, which minimizes the within-cluster scatter and maximizes the between-cluster separation, such as proposed by Huang and Wang (2006).

Both methods are carried out following two approaches: in the first one, features from the four channels are considered as a whole dataset using all-channel analysis (GA and SFS). In the second approach, features from each channel are selected using an individual-channel analysis (denoted as GA-CH and SFS-CH). All methods were performed by each subject for both control and amputee groups and for the three task categories, yielding an own feature subset in each case. Data classification was also performed, in order to determine the performance of each method.

Classification

Linear discriminant analysis (LDA), K-nearest neighbors (KNN) and multi-class support vector machine (SVM) (one-against-one approach) were employed to recognize the sEMG signal patterns. These classifiers were selected due to their high performance in classification problems and low computational complexity (Chowdhury et al., 2013), being recommended as robust classifiers in several studies (Chowdhury et al., 2013; Cipriani et al., 2011; Guo et al., 2015; Khushaba et al., 2012; Oskoei and Hu, 2008; Phinyomark et al., 2012a; Wang et al., 2013). In addition, five-fold cross validation with all trials of the experiments was used to assess the performance of the classifiers. This process was repeated three times and the average accuracy was calculated. The validation was performed off-line, and training and validation were analyzed for each subject individually. Finally, the outcome accuracy is smoothed using a majority vote (MV) post-processing technique (Khushaba et al., 2012), which consider the previous two decisions to the given point window of the decision stream. This technique prevents overwhelming the prosthetic controller with varying misclassification decisions, thus enhancing the classifier performance by eliminating spurious misclassification (Englehart and Hudgins, 2003).

Experiments

Two experiments were conducted in this study. In the first one, after the feature extraction, an analysis for the feature selection is carried out for each case and each task category, each feature selection approach, and each group of voluntaries, in order to determine an overall optimal feature subset. Features are ranked for each case separately, according to the number of times a feature is included in an optimal subset, in ascendant order, from the more selected to the less selected. As a result, the feature subsets obtained for each method and each task category were compared. In the second experiment, an overall method was adopted, selecting the optimal feature subset and the classification method, to recognize patterns related to all aforementioned task groups.

Statistical evaluation

Accuracy (Acc), mean percentage error (MPE), specificity (Sp) and Kappa’s coefficient (k) were used to evaluate the performance of each classifier. The Kappa coefficient is a parameter proposed by Cohen (Japkowicz and Shah, 2011), which represents the concordance between the targets and the prediction values. Values between 0.61 and 0.80 indicate a substantial agreement, while values greater than 0.81 indicate an almost perfect agreement. Further, in order to have a very complete analysis of the classifier performance, a confusion matrix was calculated to obtain the average accuracy for all classes.

For the analysis of the results, and taking into account the low number of observations and their unknown distribution, non-parametric approaches were used, which are strongly suggested in the literature (Cipriani et al., 2011) due to not require the assumption of normality. Statistical differences among experimental results were also evaluated, firstly using the Wilcoxon rank-sum test to compare two groups with unpaired data, and the Friedman test for simultaneous comparison of more than two groups. Post hoc pairwise comparisons using Wilcoxon rank-sum test with a Bonferroni correction factor were conducted, in which a level of ρ < 0.05 was selected as the threshold for statistical significance. The outcome of these tests were interpreted in this research to establish if there was a statistically significance difference in accuracy for each category of gestures between different subjects.

Results

In the first part of the experiments, the effects of the window length and its overlapping were evaluated. The window length was tested between 100, 200, 300, 400 and 500 ms, while the overlapping was tested between 20, 40, 60, 80 and 100 ms. Results showed a direct correlation between the classification accuracy and size of the window length: the more length of the window the more accuracy. However, there was no significant enhancement for window lengths larger than 300 ms. As well, the classification accuracy was found to increase with a progressive reduction of the overlapping time. It is actually expected that a larger window improves the accuracy, but it increases the response time. On the other hand, a larger overlap reduces the effects of the delay, but it increases the computational cost. Thus, a trade-off between these two parameters was taken into account, being chosen in this study a window length of 300 ms and an overlapping of 50 ms.

The selection of feature parameters was optimized iteratively by trial and error, seeking to improve the classification accuracy. For ZC and SSC, the thresholds values used were 0.0005 and 0.001, respectively. Other parameters were chosen according to the literature, such as: for AR (order = 4), DFA (L = 10), PSR (N = 20) (Phinyomark et al., 2013); and HFD (Kmax = 10) (Esteller et al., 2001). Additionally, the classifier’s parameters were optimized as: for KNN (K = 9) and for SVM (C = 0.01 and polynomial kernel with order 3).

A relation between average classification and number of features was also studied. Classification accuracy was found to increase in most of cases when including more features. However, there was no substantially improvement with more than six features, which agrees with the findings of Phinyomark et al. (2012a). Thus, the feature selection was adopted to obtain a suitable subset of six features to form the sEMG patterns.

Feature selection analysis

In the first experiment, the feature selection methods were evaluated for each task category. The features were ranked according to the number of times they were selected in the optimal subsets for all subjects. Figure 2a shows the ranking results where darker boxes indicate the more selected features (as the best) while clearer boxes indicate a lower selection (worst). In relation to GA and GA-CH methods, the ZC and DFA features were the most frequently selected in the optimal subsets, followed by SSC, AR, HFD and PSR, as shown in Figure 2a. Specifically, ZC and DFA were selected in all task categories for all amputee subjects. Moreover, time domain features were little considered as relevant, with MAV1 as the most relevant feature from this set. Similarly, PKF obtained from frequency domain was considered in lesser extent. The analysis with AG-CH, using the individual-channel approach, showed similarity in the selected features between channels. Furthermore, from AG and AG-CH, the features were found similar between task categories and subjects. On the other hand, in relation to SFS and SFS-CH methods, AR was the most frequently feature selected, followed by HFD and MAV1. Different features from time and frequency domains were included in the subsets for both control and amputee groups. However, outcomes using SFS and SFS-CH approaches showed strong differences in the features selected, between categories and subjects.

Figure 2
Results of the feature selection experiment for each one of the task categories. (a) Representation of the selected frequency feature, for the control and amputee groups. (b) Error variance distribution of methods for amputees. (c) Classification error for control group and amputees using the proposed feature set with SVM. CA: category A; CB: category B; CC: category C; GA: genetic algorithm; SFS: sequential forward selection; GA-CH: genetic algorithm by individual-channel; SFS-CH: sequential forward selection by individual-channel; MAV: mean absolute value; MAV1: modified mean absolute value 1; MAV2: modified mean absolute value 2; VAR: variance of sEMG; RMS: root mean square; WL: waveform length; ZC: zero crossing; SSC: slope sign change; AR: autoregressive model; MNF: mean frequency; MDF: median frequency; PKF: peak frequency; MNP: mean power; TTP: total power; PSR: power spectrum ratio; DFA: detrended fluctuation analysis; HFD: Higuchi fractal dimension; SVM: support vector machine; LDA: linear discriminant analysis; KNN: K-nearest neighbors.

All the feature subsets were tested in the chosen classifiers, in order to compare the performance of the methods to select the most suitable approach. In general, the comparison of error variance distribution of the methods had similar mean and standard deviation values for all task categories, as shown in Figure 2b, in which the results are related with all categories. Due to the similar performance of both control and amputee groups, only results of amputees have been discussed here. KNN and LDA classifiers also had similar error variance distributions among the methods, unlike SVM classifier, which showed a different behavior. From the results, SFS had the best classification performance with SVM (MPE < 2.8%), followed closely by GA-CH and GA methods with similar performances (MPE < 6.1%) and no significant differences (ρ > 0.954). In contrast, SFS showed the lowest performance using KNN and LDA, having a significant difference when compared with SFS-CH (ρ < 0.009). The SFS-CH method achieved the best performance for KNN and LDA classifiers, followed closely by GA-CH. In addition, the performance of SFS-CH was found to be significantly better when was compared with GA and SFS (ρ < 0.028). In contrast, SFS-CH method showed the lowest performance among other methods when using SVM. On the other hand, from Figure 2b, GA and GA-CH approaches not presented significant differences (ρ > 0.618) using KNN. In addition, improved results were found using the features from GA-CH with KNN (MPE < 9.6%) and LDA (MPE < 25.1%).

Error variance descripted in the box plots in Figure 2b shows high dispersion in different task categories for SFS (i.e. classification of CB and CC using KNN and LDA) and for SFS-CH (i.e. CA, CB and CC using SVM). The results indicate that features of SFS, GA and GA-CH with the SVM classifier provide lower classification error rates for both control and amputee groups. From these findings, the ranking of the selected features from all cases was employed to obtain an optimal feature subset for each method. From GA and GA-CH, the ranking resulted in the same feature sets, which were DFA, HFD, AR, ZC, SSC and PSR. Likewise, from SFS, the features obtained were DFA, HFD, AR, ZC, MAV1 and MNF. For SFS-CH, the features were different for each category, in which the ranking was not significant and therefore, it was no suitable to be considered. Afterwards, an evaluation of performances with the subsets obtained from the raking for each method was performed having no significant differences between SFS and GA (ρ < 0.918). As a result, taking into account all of the above, the six features selected by GA (DFA, HFD, AR, ZC, SSC and PSR) were chosen as the proposed feature set to be used for the analyses conducted in this study.

Proposed pattern recognition system

In the second experiment, the proposed feature set was evaluated on the KNN, LDA and SVM classifiers. For all task categories, SVM showed the best performance, followed by KNN and LDA, respectively. Moreover, SVM had a significant difference with LDA (ρ < 0.024) for all cases, but it did not have a significance difference with KNN (ρ > 0.062), except for the control group for gestures of the CB (ρ < 0.032). Figure 2c shows the classification error for both control and amputee groups using the SVM classifier. From the results with amputees, the average accuracy on CA, CB and CC categories using SVM were 98.9 ± 0.8%, 96.9 ± 1.0% and 97.2 ± 0.9%, respectively, with the grasp gestures (CB) obtaining the lowest performance. For all categories, the Kappa and specificity values were higher than 0.94 and 99.0%, respectively. The accuracy among subjects ranged from 99.8% (A7 for CA) to 90.8% (A3 for CB). In all experiments, the highest performances per subject were for the subjects A7, A10 and A8, sorted by performance, respectively. For category C, which includes thirteen gestures, the best accuracies were achieved by subject A7 (99.5 ± 1.0%), followed closely by A10 (97.0 ± 2.4%). On the other hand, the worst result was for subject A3, for CB (90.8 ± 3.6%), while for CA and CC categories, subjects A5 and A6 showed the worst performance (although, above 92.4%). Results using KNN showed average accuracies above 94.6% (CB), while for LDA the performance was above 71.9% (CC). From the results for the control group, accuracies ranged from 99.9% to 96.8%. The average accuracy on CA, CB and CC categories using SVM were 99.2 ± 0.6%, 97.6 ± 0.8% and 98.8 ± 0.7%, respectively, with specificity (Sp > 99.6%) and Kappa (K > 0.98). Table 4 summarizes the average results for the groups of control and amputees.

Table 4
Average classification accuracy (%), Specificity (%) and Kappa’s Coefficient of three task categories, for control and amputees. Table includes the results for SVM, LDA and KNN classifiers.

Cat.	Subjects	SVM			LDA			KNN
		Acc	Sp	k	Acc	Sp	k	Acc	Sp	k
CA	Control	99.17±0.6	99.94±0.0	0.99±0.0	95.06±2.50	99.3±0.3	0.92±0.0	99.55±0.2	99.6±0.2	0.96±0.0
	Amputees	98.94±0.8	99.59±0.3	0.97±0.0	80.21±10.88	96.8±1.9	0.78±0.2	96.98±1.9	97.6±0.8	0.92±0.1
CB	Control	97.64±0.8	99.61±0.3	0.98±0.0	89.35±3.48	96.7±0.8	0.84±0.0	98.07±1.4	97.5±1.1	0.93±0.1
	Amputees	96.94±1.0	99.03±0.4	0.94±0.0	76.06±9.76	94.9±1.4	0.75±0.1	94.58±2.2	96.3±1.4	0.84±0.1
CC	Control	98.77±0.7	99.90±0.1	0.99±0.0	88.19±3.26	96.7±0.8	0.88±0.0	97.89±0.7	97.5±1.1	0.97±0.1
	Amputees	97.19±0.9	99.63±0.2	0.95±0.0	71.91±10.78	94.9±1.4	0.75±0.1	95.36±2.0	96.3±1.4	0.89±0.1

• SVM: support vector machine; LDA: linear discriminant analysis; KNN: K-nearest neighbors; Acc: accuracy; Sp: specificity; k: Kappa’s coefficient; Cat: category; CA: category A; CB: category B; CC: category C.

The confusion matrices for each task category provide the average error classification for all amputees, as shown in Figure 3. The confusion matrix for CA shows that finger recognitions was mainly confused with flexions of the near fingers. On the other hand, the little finger (F5) was found to be easily confused with most of the gestures. Furthermore, hand close (HC) was confused with all fingers (F1-F5). For CB, the confusion matrix shows that most gestures were confused among them. Specifically, confusion matrices for the subjects show that tripod (TP) is easily mistaken with all other classes. However, the highest mistaken was found between the full hand wrap grasps (for 2.9% of times LD was confused as MD). Finally, the results for CC resemble with previous observations, in which TP was found to be the most difficult gesture to be recognized. It was also found from the results that gestures belonging to CA were rarely confused with CB (below 0.1% of mistaken).

Figure 3
Confusion matrix with average misclassification for amputees. CA: category A; CB: category B; CC: category C; RT rest state; F1: thumb flexion; F2: index flexion; F3: middle flexion; F4: ring flexion; F5: little flexion; HC: hand close; HO: hand open; LD: large diameter, MD: medium diameter; LT: lateral grasp; TR: tripod grasp; TP: tip pinch.

Discussion

The objective of this study was to propose a method able to recognize patterns from dexterous gestures from amputees based on low-density sEMG signals. A comparison was presented among different feature combinations to define the optimal set which improved the performance results. The results indicate that feature selection using SFS-based methods was highly variable between subjects, while GA-based methods provided more homogeneous feature subsets, showing a trend to select the same features. Moreover, classification performance for SFS and SFS-CH methods showed variation according to the algorithm used for classification. In terms of the nature of the features, GA method selected in most of cases DFA and HFD features, which are suitable to characterize the complexity of the sEMG signals. On the other hand, AR feature was also considered with high relevance in the proposed feature set, which is consistent with literature, as shown in Table 1. Other features from the time domain such as SSC and ZC were proposed in this research, which provide information about frequency properties of the signal. Similar results were achieved by SFS method, which suggest a correlation between both methods. However, SFS included the MNF feature in the frequency domain, which involves a transformation. Moreover, SFS uses the classification accuracy as criteria for selection, which makes it dependent of the selection of the classifier and the time consumption for training and validation. Otherwise, GA is based on the entropy of the feature distribution, which provide high-quality results for the feature selection and avoid local solutions, as was here reported. In terms of computational complexity, GA provides a suitable method for feature selection in comparison with SFS methods. As a result of the factors mentioned above, the selection of the outcome of GA were proposed to provide a better characterization of dexterous patterns from the sEMG signal. It should also be noted that extraction of same features from all channels is more convenient for simpler implementations. However, this study showed that it is possible to obtain better results with a single-channel approach, but it would require a proposed method by each subject. Thus, a suitable scheme to get the best results is based on the best performance achieved with GA + SVM. It is worth mentioning that KNN also could be considered as a good classifier, due to present a performance close to SVM and have a lower computational complexity.

In relation to the number of classes, CA includes eight, CB includes six and CC has thirteen. Notice that a high number of classes increase the complexity of the recognition. A comparison of the overall results showed that grasp gestures had a lower performance, even when the scheme includes fewer classes. The CB gestures had lower accuracy than gestures of CA for all amputees except for subject A5, with difference between categories from 0.21% to 5.66%. Classification accuracy was calculated by post-processing EMG recordings (off-line). Although each subject’s performance was different (i.e. subjects A7, A10 and A8 achieved generally better performance than the other ones), most of them presented clear abilities for gestures recognizing. Specifically, the subject A3 showed difficulties when performing grasp gestures, which reflected in the results. However, the result for individual finger recognition for the same subject was similar to other participants (96.4 ± 1.3%). In all cases, it was found a lower accuracy for the amputees in comparison with control subjects, which may be due to several reasons, such as disuse of the muscles or damage of the remaining muscles (Kumar et al., 2013). All task category analyses lead to understand the abilities from amputees to send commands to control hand gestures. This study about dexterous gestures included the analysis of the ability to perform individual fingers movements to understand the abilities of amputees to perform the grasps. Grasp gestures recognition seems to be more difficult than individual finger, mainly due to the simultaneous use of more fingers. The category including all the gestures showed the possibility for identification of both individual finger gestures and grasps with high accuracy.

In addition, it is known from the literature that the relation between strength of contraction and sEMG amplitude is non-linear for muscle contractions on dexterous movements, making it difficult the differentiation of muscular activities with the rest state at these conditions (Arjunan and Kumar, 2010; Arjunan et al., 2015). On the other hand, sEMG signals from dexterous movements have a poor SNR (Signal to Noise Ratio), while higher contractions produce a bigger SNR, making the features based on amplitude feasible only to better performance. Also, in the experiments with amputees, it was found greater difficulty to accomplish lower contractions during the performance of gestures in comparison with able-bodied subjects, mainly because the amputation effect. On the other hand, spasms and difficulty for contraction of selective muscles while conducting dexterous tasks for a long time was also reported by the amputees, mainly A3 and A5, although we think these issues can be overcome with a more frequent use of the muscles. Also, we think that the level of amputation has influence in the results, specifically for subjects A4, A6 and A9, who also had some changes on the insertion point of the muscles due to the amputation height. In relation to the amputee’s characteristics, the age range of the amputee group was not restricted because of the difficulty to find volunteers with the inclusion criteria adopted. On the other hand, the range stablished for the control group was set to be smaller than for the amputees. However, age was not considered in our study as a factor to be compared between both groups. Meanwhile, not significant differences between accuracies and age (ρ < 0.365), for all movements on amputee group, were found.

For all experiments, the participants were required to concentrate whilst the tasks were carried out. We attempted to avoid muscle fatigue by allowing enough time between repetitions during the tests, and no more than one hour was considered for tests, in order to avoid mental stress. This is due to the protocol used, in which all the participants held the arms on the table, but a final application would not have this limitation.

Despite the system was validated in off-line mode, the required time for recording and processing the raw sEMG signals is lower than 300 ms, which agrees with the criteria reported in Englehart et al. (2001) to be used in real time application. A comparison of our technique with previous works can be unbalanced because of the difference in number of electrodes and muscles selected, number of classes, whether amputees were included in the study and the kind of gestures used in relation to the level of dexterity. However, it is possible to obtain a ratio (R), which is proposed in this research, to get the relation between the number of classes and the number of electrodes, meaning that the higher is value of R, the better is the method. Table 5 shows a summary of previous works, using the value of R for comparison with our work. In this sense, our work presents value of R equal 2 (for CA) and 1.5 (for CB).

Table 5
Comparison of previous research with this current work.

Authors	N. Ch	N. Cl	Kind of gestures	Subjects	Acc [%]	Ratio R
Peleg et al. (2002)	2	5	Fingers gestures	4 C	93	2.50
Tsenov et al. (2006)	2	4	Finger and hand gestures	1 C	98	2.00
	4	4				1.00
Khezri and Jahed (2007)	2	6	Hand and grasp gestures	4 C	87.3	3.00
Oskoei and Hu (2008)	4	5	Wrist gestures	11 C	97	1.25
Tenore et al. (2009)	19	12	Finger gestures	5 C and 1 A	88	0.63
	32	12	Finger gestures	5 C and 1 A	94	0.38
Chu and Lee (2009)	4	10	Wrist and grasp gestures	11 C	97	2.50
Cipriani et al. (2011)	8	7	Finger gestures	5 C and 5 A	48 to 98	0.88
Li et al. (2011)	12	11	Wrist and grasp gestures	5 C 8 A	71.3	0.92
Phinyomark et al. (2012b)	1	2	Forearm, wrist and hand gestures	20 C	78 to 91	2.00
Al-Timemy et al. (2013)	6	15	Finger gestures	10 C	89	2.50
	6	12	Finger gestures	6 A	79	2.00
Wang et al. (2013)	2	8	Grasp gestures	6 C	98	4.00
Castro et al. (2015)	5	6	Finger gestures	4 C	97	1.20
	5	10	Finger and grasp gestures	4 C	80	2.00
This study	4	8	Finger and hand gestures	10 C and 10 A	99	2.00
	4	6	Grasp gestures	10 C and 10 A	97	1.50
	4	13	Finger and grasp gestures	10 C and 10 A	97	3.25

• It were included the number of channels (N.Ch); number of classes (N.Cl); the kind of gestures, as if it included gestures related to finger, hand, writs, forearm and grasps; subjects, whether Control (C) or amputee (A); Accuracy (Acc); and finally the ratio R here proposed, which is the relation of N.Cl/N.Ch.

Our research represents a contribution in the study of the non-linear techniques to characterize sEMG signals for accurately recognize dexterous movements. The validation of the proposed method to recognize all thirteen gestures considered with high accurate represents also a contribution to the field. Additionally, the use of low-density sEMG signals represents an important advantage for the acceptance of prostheses by amputees, according with Khushaba et al. (2012), who state that reduction in the number of electrodes, without compromising the classification accuracy, would significantly simplify the requirements for controlling state of the art prostheses. From the studies which considered dexterous gestures, Chu and Lee (2009) only included two grasp gestures (cylindrical and lateral grasps). On the other hand, Khezri and Jahed (2007) identified six gestures combined hand and wrist gestures, and only one grasp gesture is considered in comparison with five gestures reported in our study. They used two channels and performed experiments in real-time. It is worth mentioning the study of Li et al. (2011), which recognizes four different grasps among eleven gestures with 71.3% of accuracy, validated on amputees. However, their work included high-density sEMG signals (twelve channels). Moreover, Al-Timemy et al. (2013) included twelve different based-finger gestures with six amputees and six sEMG channels, with 89% of accuracy. However, their study did not include grasp gestures, a drawback that our research has overcome. On the other hand, Castro et al. (2015) included finger and grasp gestures with five channels, with 80% of accuracy and only was validated on able-bodied subjects. Additionally, our study included the most important gestures from a user-centered perspective according to Peerdeman et al. (2011), which improves significantly the functionality of the prostheses for daily live activities. Moreover, results presented in this research can be considered quite relevant, due to the validation of techniques with ten amputees, even in off-line mode. For our knowledge, no studies for recognition of dexterous movements in more than eight amputees were found in literature. It is worth mentioning that these on-line results are out of focus of this study. It is expected that on-line and real-time experiments result in decreased performance, which could be overcome with a period of training for a better adaptation to the prosthesis.

The problem addressed here has not been fully resolved in the literature yet, being that these results can be considered both interesting and challenging, since in the state of the art there is a lack of tests with amputees which will be, at the end, the real users. This work presents an original contribution on both pattern recognition and application in amputees, having an outperformed results compared to others that evaluate only non-amputee volunteers. A new method is here proposed to identify individual finger flexion, open/close hand and grasps movements for hand prosthesis, using weak signals and sEMG low density. The system is divided into three categories for the study: individual finger movements, open/close hand and grasp gestures. A set of feature combining non-linear techniques and statistical parameters proportional to the sEMG amplitude were also proposed to be used as input to the classifiers. In terms of classification, LDA had poor results, with the best results obtained by SVM, followed closely by KNN. However, KNN is faster than SVM, which implies an advantage over SVM for real time applications, taking into account the not significant difference on statistic significant tests. These results are encouraging for the development of real-time control strategies based on the use of small number of sEMG channels to accurately control dexterous prosthetic hands. In comparison with others works in the literature, the method proposed in this research reached the highest average accuracy (98.9%) and the highest value of R (3.25) in works including amputees, which means that this method is reliable and efficient. The validation of the method here proposed was performed in off-line mode. In the future, we will test the validity of the proposed method in on-line and real-time applications, towards a single-channel for the recognition of dexterous gestures on amputees.

Acknowledgements

This work has been supported by CAPES and FAPES/Brazil (Project Number 007/2014: Use of robotics and assistive technology for children and adults with disabilities).

• How to cite this article: Mayor JJV, Costa RM, Frizera A No, Bastos TF. Dexterous hand gestures recognition based on low-density sEMG signals for upper-limb forearm amputees. Res Biomed Eng. 2017; 33(3):. DOI: 10.1590/2446-4740.08516

References

• Al-Timemy AH, Bugmann G, Escudero J, Outram N. Classification of finger movements for the dexterous hand prosthesis control with surface electromyography. IEEE Journal of Biomedical and Health Informatics. 2013; 17(3):608-18. PMid:24592463. http://dx.doi.org/10.1109/JBHI.2013.2249590
  » http://dx.doi.org/10.1109/JBHI.2013.2249590
• Arjunan SP, Kumar DK, Bueno L, Villarejo JJ, Bastos TF. Upper limb prosthesis devices. In: Bastos TF, Kumar DK, Arjunan SP. Devices for mobility and manipulation for people with reduced abilities. Boca Raton: CRC Press; 2015. p. 121-7.
• Arjunan SP, Kumar DK. Decoding subtle forearm flexions using fractal features of surface electromyogram from single and multiple sensors. Journal of Neuroengineering and Rehabilitation. 2010; 7(1):53. PMid:20964863. http://dx.doi.org/10.1186/1743-0003-7-53
  » http://dx.doi.org/10.1186/1743-0003-7-53
• Arjunan SP, Kumar DK. Fractal theory based Non-linear analysis of sEMG. In: 3rd International Conference on Intelligent Sensors, Sensor Networks and Information; 2007 Dec 3-6; Melbourne, Australia. Melbourne: IEEE; 2007. p. 545-8.
• Castro MCF, Arjunan SP, Kumar DK. Selection of suitable hand gestures for reliable myoelectric human computer interface. Biomedical Engineering Online. 2015; 14(1):30. PMid:25889735. http://dx.doi.org/10.1186/s12938-015-0025-5
  » http://dx.doi.org/10.1186/s12938-015-0025-5
• Chowdhury RH, Reaz MBI, Ali MABM, Bakar AAA, Chellappan K, Chang TG. Surface electromyography signal processing and classification techniques. Sensors (Basel). 2013; 13(9):12431-66. PMid:24048337. http://dx.doi.org/10.3390/s130912431
  » http://dx.doi.org/10.3390/s130912431
• Chu JU, Lee YJ. Conjugate-prior-penalized learning of gaussian mixture models for multifunction myoelectric hand control. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2009; 17(3):287-97. PMid:19228565. http://dx.doi.org/10.1109/TNSRE.2009.2015177
  » http://dx.doi.org/10.1109/TNSRE.2009.2015177
• Cipriani C, Antfolk C, Controzzi M, Lundborg G, Rosen B, Carrozza MC, Sebelius F. Online myoelectric control of a dexterous hand prosthesis by transradial amputees. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2011; 19(3):260-70. PMid:21292599. http://dx.doi.org/10.1109/TNSRE.2011.2108667
  » http://dx.doi.org/10.1109/TNSRE.2011.2108667
• E-Hand.com. Anatomy of the hand [internet]. American Society for Surgery of the Hand; 2017 [cited 2017 Jan 28]. Available from: http://www.eatonhand.com/hom/hom033.htm
  » http://www.eatonhand.com/hom/hom033.htm
• Englehart K, Hudgins B, Parker PA. A wavelet-based continuous classification scheme for multifunction myoelectric control. IEEE Transactions on Biomedical Engineering. 2001; 48(3):302-11.
• Englehart K, Hudgins B. A robust, real-time control scheme for multifunction myoelectric control. IEEE Transactions on Biomedical Engineering. 2003; 50(7):848-54.
• Esteller R, Vachtsevanos G, Echauz J, Litt BA. Comparison of waveform fractal dimension algorithms. IEEE Transactions on Circuits and Systems. 2001; 48(2):177-83. http://dx.doi.org/10.1109/81.904882
  » http://dx.doi.org/10.1109/81.904882
• Guo S, Pang M, Gao B, Hirata H, Ishihara H. Comparison of sEMG-Based Feature Extraction and Motion Classification Methods for Upper-Limb Movement. Sensors (Basel). 2015; 15(4):9022-38. PMid:25894941. http://dx.doi.org/10.3390/s150409022
  » http://dx.doi.org/10.3390/s150409022
• Hargrove LJ, Li G, Englehart KB, Hudgins BS. Principal components analysis preprocessing for improved classification accuracies in pattern-recognition-based myoelectric control. IEEE Transactions on Biomedical Engineering. 2009; 56(5):1407-14.
• Hermens HJ, Freriks B, Disselhorst-Klug C, Rau G. Development of recommendations for SEMG sensors and sensor placement procedures. Journal of Electromyography and Kinesiology. 2000; 10(5):361-74. PMid:11018445. http://dx.doi.org/10.1016/S1050-6411(00)00027-4
  » http://dx.doi.org/10.1016/S1050-6411(00)00027-4
• Huang C, Wang C. A GA-based feature selection, parameters optimization for support vector machines. Expert Systems with Applications. 2006; 31(2):231-40. http://dx.doi.org/10.1016/j.eswa.2005.09.024
  » http://dx.doi.org/10.1016/j.eswa.2005.09.024
• Japkowicz N, Shah M. Evaluation learning algorithms a classification perspective. New York: Cambridge University Press; 2011.
• Khezri M, Jahed M. Real-time intelligent pattern recognition algorithm for surface EMG signals. Biomedical Engineering Online. 2007; 6(1):45. PMid:18053184. http://dx.doi.org/10.1186/1475-925X-6-45
  » http://dx.doi.org/10.1186/1475-925X-6-45
• Khushaba RN, Kodagoda S, Takruri M, Dissanayake G. Toward improved control of prosthetic fingers using surface electromyogram (EMG) signals. Expert Systems with Applications. 2012; 39(12):10731-8. http://dx.doi.org/10.1016/j.eswa.2012.02.192
  » http://dx.doi.org/10.1016/j.eswa.2012.02.192
• Kumar DK, Arjunan SP, Singh VP. Towards identification of finger flexions using single channel surface electromyography - able bodied and amputee subjects. Journal of Neuroengineering and Rehabilitation. 2013; 10(1):50. PMid:23758881. http://dx.doi.org/10.1186/1743-0003-10-50
  » http://dx.doi.org/10.1186/1743-0003-10-50
• Li G, Li Y, Yu L, Geng Y. Conditioning and sampling issues of EMG signals in motion recognition of multifunctional myoelectric prostheses. Annals of Biomedical Engineering. 2011; 39(6):1779-87. PMid:21293972. http://dx.doi.org/10.1007/s10439-011-0265-x
  » http://dx.doi.org/10.1007/s10439-011-0265-x
• Merletti R, Parker PJ. Electromyography: physiology, engineering and non-ivasive applications. New York: John Wiley; 2004.
• Naik GR, Kumar DK, Arjunan S. Use of sEMG in identification of low level muscle activities: features based on ICA and Fractal dimension. In: EMBC 2009: Proceedings of the 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society: Engineering the Future of Biomedicine; 2009 Sept 3-6; Minneapolis, USA. Minneapolis: IEEE; 2009. p. 364-7.
• Naik GR, Kumar DK, Arjunan SP. Pattern classification of myo-electrical signal during different Maximum Voluntary Contractions: a study using BSS techniques. Measurement Science Review. 2010; 10(1):1-6. http://dx.doi.org/10.2478/v10048-010-0001-y
  » http://dx.doi.org/10.2478/v10048-010-0001-y
• Oskoei MA, Hu H. Support vector machine-based classification scheme for myoelectric control applied to upper limb. IEEE Transactions on Biomedical Engineering. 2008; 55(8):1956-65.
• Peerdeman B, Boere D, Witteveen H, Huis R, Hermens H, Stramigioli S, et al. Myoelectric forearm prostheses: State of the art from a user-centered perspective. Journal of Rehabilitation Research and Development. 2011; 48(6):719-37. PMid:21938658. http://dx.doi.org/10.1682/JRRD.2010.08.0161
  » http://dx.doi.org/10.1682/JRRD.2010.08.0161
• Peleg D, Braiman E, Yom-Tov E, Inbar GF. Classification of finger activation for use in a robotic prosthesis arm. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2002; 10(4):290-3. PMid:12611366. http://dx.doi.org/10.1109/TNSRE.2002.806831
  » http://dx.doi.org/10.1109/TNSRE.2002.806831
• Phinyomark A, Limsakul C, Phukpattaranont P. A novel feature extraction for robust EMG pattern recognition. Journal of Computers. 2009; 1(1):71-80.
• Phinyomark A, Phukpattaranont P, Limsakul C. Feature reduction and selection for EMG signal classification. Expert Systems with Applications. 2012a; 39(8):7420-31. http://dx.doi.org/10.1016/j.eswa.2012.01.102
  » http://dx.doi.org/10.1016/j.eswa.2012.01.102
• Phinyomark A, Phukpattaranont P, Limsakul C. Fractal analysis features for weak and single-channel upper-limb EMG signals. Expert Systems with Applications. 2012b; 39(12):11156-63. http://dx.doi.org/10.1016/j.eswa.2012.03.039
  » http://dx.doi.org/10.1016/j.eswa.2012.03.039
• Phinyomark A, Quaine F, Charbonnier S, Serviere C, Tarpin-Bernard F, Laurillau Y. EMG feature evaluation for improving myoelectric pattern recognition robustness. Expert Systems with Applications. 2013; 40(12):4832-40. http://dx.doi.org/10.1016/j.eswa.2013.02.023
  » http://dx.doi.org/10.1016/j.eswa.2013.02.023
• Sensinger JW, Lock BA, Kuiken TA. Adaptive pattern recognition of myoelectric signals: exploration of conceptual framework and practical algorithms. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2009; 17(3):270-8. PMid:19497834. http://dx.doi.org/10.1109/TNSRE.2009.2023282
  » http://dx.doi.org/10.1109/TNSRE.2009.2023282
• Tenore FVG, Ramos A, Fahmy A, Acharya S, Etienne-Cummings R, Thakor NV. Decoding of individuated finger movements using surface electromyography. IEEE Transactions on Biomedical Engineering. 2009; 56(5):1427-34.
• Theodoridis S, Koutroumbas K. Pattern recognition. 4th ed. Cambridge: Academic Press; 2008.
• Tommasi T, Orabona F, Castellini C, Caputo B. Improving control of dexterous hand prostheses using adaptive learning. IEEE Transactions on Robotics. 2013; 29(1):207-19. http://dx.doi.org/10.1109/TRO.2012.2226386
  » http://dx.doi.org/10.1109/TRO.2012.2226386
• Tsenov G, Zeghbib AH, Palis F, Shoylev N, Mladenov V. Neural networks for online classification of hand and finger movements using surface EMG signals. In: NEUREL 2006: 8th Seminar on Neural Network Applications in Electrical Engineering; 2006 Sept 25-27; Belgrade, Serbia & Montenegro. Serbia: IEEE; 2006. p. 167-71.
• Wang N, Chen Y, Zhang X. The recognition of multi-finger prehensile postures using LDA. Biomed Signal Proces. 2013; 8(6):706-12. http://dx.doi.org/10.1016/j.bspc.2013.06.006
  » http://dx.doi.org/10.1016/j.bspc.2013.06.006
• Zecca M, Micera S, Carrozza MC, Dario P. Control of multifunctional prosthetic hands by processing the electromyographic signal. Critical Reviews in Biomedical Engineering. 2002; 30(4-6):459-85. PMid:12739757. http://dx.doi.org/10.1615/CritRevBiomedEng.v30.i456.80
  » http://dx.doi.org/10.1615/CritRevBiomedEng.v30.i456.80

Publication Dates

• Publication in this collection
  17 Aug 2017
• Date of issue
  Jul-Sep 2017

History

• Received
  28 Jan 2017
• Accepted
  18 July 2017