Dexterous hand gestures recognition based on low-density sEMG signals for upper-limb forearm amputees

Mayor, John Jairo Villarejo; Costa, Regina Mamede; Frizera, Anselmo; Bastos, Teodiano Freire

doi:10.1590/2446-4740.08516

Acessibilidade / Reportar erro

Brasil

Research on Biomedical Engineering

Español English

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

Original Article • Res. Biomed. Eng. 33 (3) • Jul-Sep 2017 • https://doi.org/10.1590/2446-4740.08516 copy

Dexterous hand gestures recognition based on low-density sEMG signals for upper-limb forearm amputees

Authorship SCIMAGO INSTITUTIONS RANKINGS

Abstract

Introduction

Intuitive prosthesis control is one of the most important challenges in order to reduce the user effort in learning how to use an artificial hand. This work presents the development of a novel method for pattern recognition of sEMG signals able to discriminate, in a very accurate way, dexterous hand and fingers movements using a reduced number of electrodes, which implies more confidence and usability for amputees.

Methods

The system was evaluated for ten forearm amputees and the results were compared with the performance of able-bodied subjects. Multiple sEMG features based on fractal analysis (detrended fluctuation analysis and Higuchi’s fractal dimension) combined with traditional magnitude-based features were analyzed. Genetic algorithms and sequential forward selection were used to select the best set of features. Support vector machine (SVM), K-nearest neighbors (KNN) and linear discriminant analysis (LDA) were analyzed to classify individual finger flexion, hand gestures and different grasps using four electrodes, performing contractions in a natural way to accomplish these tasks. Statistical significance was computed for all the methods using different set of features, for both groups of subjects (able-bodied and amputees).

Results

The results showed average accuracy up to 99.2% for able-bodied subjects and 98.94% for amputees using SVM, followed very closely by KNN. However, KNN also produces a good performance, as it has a lower computational complexity, which implies an advantage for real-time applications.

Conclusion

The results show that the method proposed is promising for accurately controlling dexterous prosthetic hands, providing more functionality and better acceptance for amputees.

Keywords
Electromyography; Upper-limb prosthesis; Low-density surface electromyography; Dexterous hand gestures; Pattern recognition

Authors	Year	Ch	G	Tasks	Features	Feature selection	Classifier	W/O [ms]	Sub.
Peleg^† † works including finger movements;	2002	2	5	F1-F5	DFT, AR, Binary electrode activation	GA	KNN	1,2	4 C
Englehart	2003	4	4	WF, WE, RD, UD	ZC, SSC, WL, MAV			256/16	12 C
Tsenov^† † works including finger movements;	2006	2	4	F1, F2, F3, HC	MAV,VAR, WL, Norm, ZC, Absolute Max and Min, Max-Min, Med		ANN (MLP, RBF, LVQ)		1 C
Khezri^† † works including finger movements; ^‡ ‡ works including grasp gestures;	2007	2	6	HO, HC, F1, TP, WF, WE	MAV, SSC, AR, DWT		Neuro-fuzzy system	200/50	4C
Oskoei and Hu	2008	4	5	Isotonic: HF, HE, hand straight, hand abduction and adduction	MAV, RMS, WL, VAR, ZC, SSC, WAMP, MAV1, MAV2, PS, AR-2 and AR-6, MNF, MNF		SVM, LDA, MLP		11 C
Phinyomark	2009			WF, WE, HC, HO, FP, FS	TD and FD (16)		LDA		3 C
Tenore^† † works including finger movements;	2009	19	12	FFI, FEI, F345	MAV, VAR, WL, WAMP	PCA	MLP	200/25	5 C 1 A
Chu and Lee^‡ ‡ works including grasp gestures;	2009	4	10	WF, WE, radial and ulnar flexion of the wrist, HO,CG, LT,RS	AR-4, ZC, WL, SSC, MAV		GMM		10 C
Naik^† † works including finger movements;	2010	2	3	F3, F4, F5	ICA, RMS		ANOVA		7 C
Cipriani^† † works including finger movements;	2011	8	7	F1,F2,F345,F2345,TO,TR,LT	MAV		KNN	50/50	5 C 5 A
Li^‡ ‡ works including grasp gestures;	2011	12	11	EFE,WF, WE, WP, WS, HO,TP, Power Grip, Tool Grip, RS	MAV, ZC, WL, SSC		LDA		5 C 8 A
Phinyomark	2012a	1	2	FP, FS, WF, WE, HO, HC	DFA and TD		LDA		20 C
Khushaba^† † works including finger movements;	2012	2	10	F1-F5, F12, F13, F14, F15, HC	SSC, ZC, WL, HTD, Sample Skewness, AR	LDA	SVM-libsvm and KNN	100	8 C
Phinyomark	2012b	5	8	FP, FS, HO, HC, WE, WF, RD, UD	TD and FD (37)		LDA		20 C
Al-Timemy^† † works including finger movements;	2013	6	12	F1-F5, E1-E5, TA, F12, F234, F1234, RS	TD-AR	PCA OFNDA	LDA and SVM		10 C 6 A
Kumar^† † works including finger movements;	2013	1	4	F1, F2, F3, F4, RS	Wavelet Maxima Density		T-SVM	300	11 C 1 A
Tommasi^† † works including finger movements; ^‡ ‡ works including grasp gestures;	2013	10	6	TP, LT, TR, PW, PEG, prismatic 4 fingers grasp	MAV		LS-SVM		27 C
Wang^‡ ‡ works including grasp gestures;	2013	2	8	CG, HG, LT, PT, SG, TR, TP, RS	MAV, VAR, AR-4, ZC, MNF, MDF	LDA	LDA	256	6 C
Castro^‡ ‡ works including grasp gestures;	2015	5	10	TP, TR, HC, HO, RS	PSD-Av		FLD		4 C
Guo	2015	6	8	EFE, FP, FS, WF, WE, Wab, Wad	WP, DFA and Muscle Models		SVM and ANN		7 C

Subject	Gender/age	Missing Hand	Time since amputation	Prosthesis used	Level of amputation
A1	Female/45	Right	4 years	Esthetic	WD
A2	Male/64	Left	42 year	Esthetic	WD
A3	Female/48	Right	1 year	Non	WD
A4	Female/23	Right	4 years	Non	TR
A5	Female/48	Right	2 years	Non	WD
A6	Female/50	Left	25 years	Non	TR^‡ ‡ Cause of amputation due to poor circulation.
A7	Male/34	Both^† † Bilateral amputation, not same level on both sides.	2 years	Non	WD
A8	Male/21	Left	2 years	Non	WD
A9	Male/27	Both^† † Bilateral amputation, not same level on both sides.	1 year	Non	TR
A10	Male/24	Right	1 year	Non	TR

Domain	Feature	Parameters	d/c	Mathematical definition
Time domain	Mean Absolute Value (MAV)	-	1	$MAV= \frac{1}{N} \sum_{n = 1}^{N} \| X_{n} \|$
	Modified Mean Absolute Value 1 (MAV1)	-	1	$MAV1= \frac{1}{N} \sum_{n = 1}^{N} \| X_{n} \|$ $w_{n} = {\begin{matrix} 1, i f 0 .25 N \leq n \leq 0 .75 N \\ 0 .5, o t h e r w i s e \end{matrix}$
	Modified Mean Absolute Value 2 (MAV2)	-	1	$MAV2= \frac{1}{N} \sum_{n = 1}^{N} \| X_{n} \|$ $w_{n} = {\begin{matrix} 1, i f 0 .25 N \leq n \leq 0 .75 N \\ 4n/ N, i f 0 .25 N < n \\ 4(1-n / N), i f 0 .75 N > n \end{matrix}$
	Variance of sEMG (VAR)	-	1	$VAR= \frac{1}{N-1} \sum_{n = 1}^{N} X_{n}^{2}$
	Root Mean Square (RMS)	-	1	$RMS= \sqrt{\frac{1}{N} \sum_{n = 1}^{N} X_{n}^{2}}$
	Waveform Length (WL)	-	1	$WL= \sum_{n = 1}^{N - 1} \| X_{n+1} - X_{n} \|$
	Zero Crossing (ZC)	thld = 0.0005	1	$ZC= \sum_{n = 1}^{N - 1} [s g n (X_{n} {×X}_{n+1}) \cap \| X_{n} {-X}_{n+1} \| \geq thld]$ $sgn (x) = {\begin{matrix} 1, i f x \geq t h l d \\ 0, o t h e r w i s e \end{matrix}$
	Slope Sign Change (SSC)	thld = 0.001	1	$S S C = \sum_{n = 1}^{N - 1} f [(X_{n} - X_{n - 1}) \times (X_{n} - X_{n + 1})]$ $f (x) = {\begin{matrix} 1, i f x \geq t h l d \\ 0, o t h e r w i s e \end{matrix}$
	Autoregressive Model (AR)	p = 4	4	$x_{i} = \sum_{p = 1}^{P} a_{p} x_{i - p} + w_{i}$
Frequency domain	Mean Frequency (MNF)	-	1	$M N F = \sum_{j = 1}^{M} f_{j} P_{j} / \sum_{j = 1}^{M} P_{j}$
	Median Frequency (MDF)	-	1	$\sum_{j = 1}^{M D F} P_{j} = \sum_{j = M D F}^{M} P_{j} = \frac{1}{2} \sum_{j = 1}^{M} P_{j}$
	Peak Frequency (PKF)	-	1	$P K F = \max (P_{j}), j = 1,..., M$
	Mean Power (MNP)	-	1	$M N F = \sum_{j = 1}^{M} P_{j} / M$
	Total Power (TTP)	-	1	$T T P = \sum_{j = 1}^{M} P_{j}$
	Power Spectrum Ratio (PSR)	r = 20	1	$P S R = \sum_{j = f_{0} - r}^{f_{0} + r} P_{j} / \sum_{j = - \infty}^{\infty} P_{j}$
Fractal features	Detrended Fluctuation Analysis (DFA)	L = 10	1	(Phinyomark et al., 2012bPhinyomark A, Phukpattaranont P, Limsakul C. Fractal analysis features for weak and single-channel upper-limb EMG signals. Expert Systems with Applications. 2012b; 39(12):11156-63. http://dx.doi.org/10.1016/j.eswa.2012.03.039. http://dx.doi.org/10.1016/j.eswa.2012.03... )
Fractal features	Higuchi Fractal Dimension (HFD)	Kmax = 10	1	(Esteller et al., 2001Esteller R, Vachtsevanos G, Echauz J, Litt BA. Comparison of waveform fractal dimension algorithms. IEEE Transactions on Circuits and Systems. 2001; 48(2):177-83. http://dx.doi.org/10.1109/81.904882. http://dx.doi.org/10.1109/81.904882... )

Cat.	Subjects	SVM			LDA			KNN
Cat.	Subjects	Acc	Sp	k	Acc	Sp	k	Acc	Sp	k
CA	Control	99.17±0.6	99.94±0.0	0.99±0.0	95.06±2.50	99.3±0.3	0.92±0.0	99.55±0.2	99.6±0.2	0.96±0.0
CA	Amputees	98.94±0.8	99.59±0.3	0.97±0.0	80.21±10.88	96.8±1.9	0.78±0.2	96.98±1.9	97.6±0.8	0.92±0.1
CB	Control	97.64±0.8	99.61±0.3	0.98±0.0	89.35±3.48	96.7±0.8	0.84±0.0	98.07±1.4	97.5±1.1	0.93±0.1
CB	Amputees	96.94±1.0	99.03±0.4	0.94±0.0	76.06±9.76	94.9±1.4	0.75±0.1	94.58±2.2	96.3±1.4	0.84±0.1
CC	Control	98.77±0.7	99.90±0.1	0.99±0.0	88.19±3.26	96.7±0.8	0.88±0.0	97.89±0.7	97.5±1.1	0.97±0.1
CC	Amputees	97.19±0.9	99.63±0.2	0.95±0.0	71.91±10.78	94.9±1.4	0.75±0.1	95.36±2.0	96.3±1.4	0.89±0.1

Authors	N. Ch	N. Cl	Kind of gestures	Subjects	Acc [%]	Ratio R
Peleg et al. (2002)Peleg D, Braiman E, Yom-Tov E, Inbar GF. Classification of finger activation for use in a robotic prosthesis arm. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2002; 10(4):290-3. PMid:12611366. http://dx.doi.org/10.1109/TNSRE.2002.806831. http://dx.doi.org/10.1109/TNSRE.2002.806...	2	5	Fingers gestures	4 C	93	2.50
Tsenov et al. (2006)Tsenov G, Zeghbib AH, Palis F, Shoylev N, Mladenov V. Neural networks for online classification of hand and finger movements using surface EMG signals. In: NEUREL 2006: 8th Seminar on Neural Network Applications in Electrical Engineering; 2006 Sept 25-27; Belgrade, Serbia & Montenegro. Serbia: IEEE; 2006. p. 167-71.	2	4	Finger and hand gestures	1 C	98	2.00
	4	4		1 C		1.00
Khezri and Jahed (2007)Khezri M, Jahed M. Real-time intelligent pattern recognition algorithm for surface EMG signals. Biomedical Engineering Online. 2007; 6(1):45. PMid:18053184. http://dx.doi.org/10.1186/1475-925X-6-45. http://dx.doi.org/10.1186/1475-925X-6-45...	2	6	Hand and grasp gestures	4 C	87.3	3.00
Oskoei and Hu (2008)Oskoei MA, Hu H. Support vector machine-based classification scheme for myoelectric control applied to upper limb. IEEE Transactions on Biomedical Engineering. 2008; 55(8):1956-65.	4	5	Wrist gestures	11 C	97	1.25
Tenore et al. (2009)Tenore FVG, Ramos A, Fahmy A, Acharya S, Etienne-Cummings R, Thakor NV. Decoding of individuated finger movements using surface electromyography. IEEE Transactions on Biomedical Engineering. 2009; 56(5):1427-34.	19	12	Finger gestures	5 C and 1 A	88	0.63
	32	12	Finger gestures	5 C and 1 A	94	0.38
Chu and Lee (2009)Chu JU, Lee YJ. Conjugate-prior-penalized learning of gaussian mixture models for multifunction myoelectric hand control. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2009; 17(3):287-97. PMid:19228565. http://dx.doi.org/10.1109/TNSRE.2009.2015177. http://dx.doi.org/10.1109/TNSRE.2009.201...	4	10	Wrist and grasp gestures	11 C	97	2.50
Cipriani et al. (2011)Cipriani C, Antfolk C, Controzzi M, Lundborg G, Rosen B, Carrozza MC, Sebelius F. Online myoelectric control of a dexterous hand prosthesis by transradial amputees. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2011; 19(3):260-70. PMid:21292599. http://dx.doi.org/10.1109/TNSRE.2011.2108667. http://dx.doi.org/10.1109/TNSRE.2011.210...	8	7	Finger gestures	5 C and 5 A	48 to 98	0.88
Li et al. (2011)Li G, Li Y, Yu L, Geng Y. Conditioning and sampling issues of EMG signals in motion recognition of multifunctional myoelectric prostheses. Annals of Biomedical Engineering. 2011; 39(6):1779-87. PMid:21293972. http://dx.doi.org/10.1007/s10439-011-0265-x. http://dx.doi.org/10.1007/s10439-011-026...	12	11	Wrist and grasp gestures	5 C 8 A	71.3	0.92
Phinyomark et al. (2012b)Phinyomark A, Phukpattaranont P, Limsakul C. Fractal analysis features for weak and single-channel upper-limb EMG signals. Expert Systems with Applications. 2012b; 39(12):11156-63. http://dx.doi.org/10.1016/j.eswa.2012.03.039. http://dx.doi.org/10.1016/j.eswa.2012.03...	1	2	Forearm, wrist and hand gestures	20 C	78 to 91	2.00
Al-Timemy et al. (2013)Al-Timemy AH, Bugmann G, Escudero J, Outram N. Classification of finger movements for the dexterous hand prosthesis control with surface electromyography. IEEE Journal of Biomedical and Health Informatics. 2013; 17(3):608-18. PMid:24592463. http://dx.doi.org/10.1109/JBHI.2013.2249590. http://dx.doi.org/10.1109/JBHI.2013.2249...	6	15	Finger gestures	10 C	89	2.50
	6	12	Finger gestures	6 A	79	2.00
Wang et al. (2013)Wang N, Chen Y, Zhang X. The recognition of multi-finger prehensile postures using LDA. Biomed Signal Proces. 2013; 8(6):706-12. http://dx.doi.org/10.1016/j.bspc.2013.06.006. http://dx.doi.org/10.1016/j.bspc.2013.06...	2	8	Grasp gestures	6 C	98	4.00
Castro et al. (2015)Castro MCF, Arjunan SP, Kumar DK. Selection of suitable hand gestures for reliable myoelectric human computer interface. Biomedical Engineering Online. 2015; 14(1):30. PMid:25889735. http://dx.doi.org/10.1186/s12938-015-0025-5. http://dx.doi.org/10.1186/s12938-015-002...	5	6	Finger gestures	4 C	97	1.20
	5	10	Finger and grasp gestures	4 C	80	2.00
This study	4	8	Finger and hand gestures	10 C and 10 A	99	2.00
	4	6	Grasp gestures	10 C and 10 A	97	1.50
	4	13	Finger and grasp gestures	10 C and 10 A	97	3.25

Sociedade Brasileira de Engenharia Biomédica Centro de Tecnologia, bloco H, sala 327 - Cidade Universitária, 21941-914 Rio de Janeiro RJ Brasil, Tel./Fax: (55 21)2562-8591 - Rio de Janeiro - RJ - Brazil
E-mail: rbe@rbejournal.org

Acompanhe os números deste periódico no seu leitor de RSS

[1] *Corresponding author: Postgraduate Program in Electrical Engineering, Technological Center, Federal University of Espírito Santo – UFES, Avenida Fernando Ferrari, 514, Goiabeiras, CEP 29075-910, Vitória, ES, Brazil. E-mail: jvimayor@gmail.com