Synergistic control of forearm based on accelerometer data and artificial neural networks

In the present study, we modeled a reaching task as a two-link mechanism. The upper arm and forearm motion trajectories during vertical arm movements were estimated from the measured angular accelerations with dual-axis accelerometers. A data set of reaching synergies from able-bodied individuals was used to train a radial basis function artificial neural network with upper arm/ forearm tangential angular accelerations. The trained radial basis function artificial neural network for the specific movements predicted forearm motion from new upper arm trajectories with high correlation (mean, 0.9149-0.941). For all other movements, prediction was low (range, 0.0316-0.8302). Results suggest that the proposed algorithm is successful in generalization over similar motions and subjects. Such networks may be used as a high-level controller that could predict forearm kinematics from voluntary movements of the upper arm. This methodology is suitable for restoring the upper limb functions of individuals with motor disabilities of the forearm, but not of the upper arm. The developed control paradigm is applicable to upper-limb orthotic systems employing functional electrical stimulation. The proposed approach is of great significance particularly for humans with spinal cord injuries in a free-living environment. The implication of a measurement system with dual-axis accelerometers, developed for this study, is further seen in the evaluation of movement during the course of rehabilitation. For this purpose, training-related changes in synergies apparent from movement kinematics during rehabilitation would characterize the extent and the course of recovery. As such, a simple system using this methodology is of particular importance for stroke patients. The results underlie the important issue of upper-limb coordination.


Introduction
The recovery of the functional use of upper limbs by patients after spinal cord injuries (SCI) or stroke is one of the main goals of rehabilitation.Classical treatments primarily rely on the use of physiotherapy, which depends on the trained therapists and their past experience.Evidence shows that additional early exercise training in SCI and stroke patients may be beneficial (1,2).Independent and repetitive exercises could directly strengthen arms and legs, and may help patients recover more quickly (3,4).
Patients with SCI at the C5/C6 level retain normal control of their upper arm movements, they have some control of their elbow flexion, wrist flexion/extension and supination/pronation; however, they lack volitional control of elbow extension and fingers.Comparable limitations can be found among patients with motor impairments resulting from other neurological disorders such as stroke.In recent years, several neural prostheses (NP) have been developed and tested as orthoses or as therapeutic systems for hemiplegic and tetraplegic subjects aiming to improve the function of the upper extremities.Several systems with surface and implanted electrodes have been used for restoring motor functions by use of functional electrical stimulation (FES) (5)(6)(7) and by applying synergistic control mechanisms (8)(9)(10)(11)(12)(13)(14).The use of NP has demonstrated that targeted groups of subjects could significantly benefit from FES that is integrated in goal-directed movements (15).The controller for such stimulation needs a source of command signals indicating what the user is attempting to do.Various sources exist such as electromyogram (16), electroencephalogram and/or brain-computer interface (17,18), or body segment motion (8) to control FES.Several groups (14,(19)(20)(21)(22) have attempted to use the motion of one segment of the upper limb to control the other (i.e., the upper arm to control forearm).
In a previous study (21), we proposed this simple strategy to guide reaching in quadriplegic humans lacking elbow extension and flexion.An experimental study of reaching in the horizontal plane by able-bodied humans allowed us to reduce the dimensionality of the control vector from two to a single variable.This was accomplished by detailed analysis of the synergy between shoulder and elbow joint angles.The same experimental setup in quadriplegics with retained shoulder movement showed that natural synergism was preserved even though the motor and sensory components were reduced or absent.Based on these findings, we proposed that an assistive system for the elbow may be volitionally controlled only from ipsilateral shoulder movements.For this we applied an inductive learning technique for determining synergies between movements of the shoulder and elbow joints while reaching (14).Once the task and preferred strategy for movement are selected, the voluntary control drives the proximal segment (upper arm) and the synergistic (artificial) control drives the distal segment (forearm).We expanded this approach to an automatic method for synthesizing the control for NP that could also augment pronation/supination in persons with quadriplegia (19).With a well-chosen reduced set of data from the upper arm and the forearm, the group (20) succeeded improving the control algorithm designed as the command signal for powered prostheses and FES in patients with transhumeral amputations and C5/C6 quadriplegia, respectively.They planned to deploy this algorithm in FES control systems to predict the movement of the paralyzed joint from the movement of the joints still under voluntary control.However, to the best of our knowledge, no results have been reported in the literature.Tresadern et al. (22) was the first group to use machine learning for upper limb FES triggering based on forearm accelerations.A neural network that they trained on various datasets generalized well over test sessions and (to a lesser extent) patients.Generalization over motion was demonstrated for similar motions and less for dissimilar ones.
In the present study, we evaluated the possible use of upper arm accelerations for synergistic control of the forearm.This approach was selected on the basis of our previous research (13)(14)(15)19,21,23,24).We have demonstrated that upper arm and forearm movements are strongly coupled in the space formed by angular accelerations of the shoulder and elbow joints (19,25).To observe segment movements, micro-electro-mechanical systems-based accelerometers may be used (26)(27)(28); however, thus far, accelerometers were never used efficiently for the assessment of joint angles with sufficient precision in upper extremities.We therefore used tangential angular accelerations instead.For this study we selected to use artificial neural networks (ANN), a currently popular method to generate control signals from multiple, fuzzy sensory information (29)(30)(31)(32).
It has been established from motor control studies that the biomechanical structure of the arm is used in different ways to provide movements in different directions.The control of arm joints varies during hand movements in different directions despite the common movement characteristics of the hand (bell shape of velocity, close to straight path).Most published studies (6,15,22,(33)(34)(35) report results while analyzing hand movements in horizontal workspace, commonly the surface of the table in front of the subject.This can restrict the performance of the activities of daily living (ADL).Upper limb movements in the vertical direction belong to a free-living environment (36) that implies a need for the development of a control method that will provide upper limb vertical movement.In this study, we propose an approach to control forearm motion in a selected class of ADL that require a relatively large arm elevation.Such arm motion is of interest always when the hand and arm are initially in a relaxed position at the side of the body.The same holds true when the hand is placed on a table, but somewhat less arm elevation is necessary to perform ADL.These situations occur frequently (e.g., using a telephone, personal hygiene and grooming, switching light on/off on the wall, taking an object from a table and placing it on a shelf above the table).Thus, we evaluated reaching movements between different height levels in a para-sagittal plane (36).
Traditional assessment methods lack an objective stan-dardized analysis for evaluating a patient's performance and an estimate of the effectiveness of a therapy.To address this problem, trajectories during the rehabilitation course after a stroke have to be quantified, and hence appropriate instruments for quantitative measurements are desirable to capture motion trajectories.The method that we designed for the control of NP may be applicable for the quantitative assessment of the discrepancy from normal movement.

Subjects
Seven healthy adults (3 male and 4 female) with a mean age of 29 ± 7.8 years (range 23-41 years) and a mean height of 179 ± 11.8 cm (range 167-194 cm) participated in the study.All subjects were right-handed by selfreport and were either students or employees at the School of Electrical Engineering, University of Belgrade.
All subjects were naive as to the experimental purpose and gave informed consent to participate in the study.The experimental procedures were approved by the Institutional Ethical Review Board of the University of Belgrade and were in accordance with the declaration of Helsinki.

Experiment
The testing environment was standardized as much as possible.During testing, a healthy subject sat in front of a table with his/her hand and arm hanging in a relaxed position at the side of the body (Figure 1).The table was at such a height as to permit the subject to comfortably place his/her hand on the table without moving the torso.The chair was armless and its height was adjusted so that the subject's feet could be flat on the floor.
Participants performed reaching tasks to three targets located in a para-sagittal plane passing through the shoulder.The first three tasks were initiated from the relaxed arm in a hanging position, while the remaining two were started with the hand placed on the table and the arm relaxed.
For the first three tasks, the subjects were instructed to reach: a) forward and place their hand on the surface of the table, b) upward and touch the ear with their hand, and c) upward to attain their highest hand position.After completing each task the subjects were instructed to retract the arm back to the starting position.For the two additional tasks, the subjects were instructed to perform the same motion as described above in b) and c), however, by starting with their hand on the table.The subjects were told not to move the trunk.These were the only instructions given to subjects.Their pace of movement and their strat-egy to perform each of the tasks were arbitrary.Note that this experimental paradigm was designed to require relatively large arm elevation.Movements involved shoulder adduction and/or flexion and elbow extension during reaching or shoulder abduction and/or extension and elbow flexion while retracting the hand depending on subjects' own strategy.
Each task was performed five times consecutively within one cycle and each cycle was repeated five times.Therefore, 125 movements were recorded per subject (5 tasks x 5 trials x 5 cycles).To avoid fatigue and lack of concentration, the subjects were instructed to pause for a few minutes after each cycle.

Instrumentation
Four dual-axis ADXL203 (Analog Devices, Norwood, MA, USA) accelerometers with appropriate signal conditioning circuitry were secured to the ends of two lightweight Figure 1.Experimental design.Five tasks (T j , j = 1-5) were performed using target hand locations at three different height levels: table, ear and the highest hand position (dashed lines).The first three tasks (T 1 , T 2 , T 3 ) were initiated from the relaxed hanging position (first initial position -thick line), while the remaining two (T 4 , T 5 ) were started with the hand placed at a relaxed position on the table (second initial position -thick line).All hand movements were performed in para-sagittal plane passing through the shoulder (reference system xOy).One bar with two dual-axis ADXL203 accelerometers at proximal and distal positions was attached per upper arm and forearm.
www.bjournal.com.brbars.The two bars were attached along the upper arm and forearm using Velcro straps (Figure 1).The accelerometers were positioned so that one direction of the sensor pointed along the length of the bar, while the other direction pointed perpendicular to it.Both sets of data were considered.The data used in this study were collected from the measurements in the direction perpendicular to the axes of upper arm and forearm only.The leads from the four accelerometers were fastened to the upper arm and forearm, leaving enough slack so as not to hinder motion.
The voltages from the accelerometers (eight signals in total) were recorded by a laptop computer using the National Instruments DAQCard-6062E acquisition card.The software used for acquisition was developed using LabView 7.1.

Data collection
Data from all eight recorded channels were sampled at 50 samples per second.The signals from the channels, which were recording accelerations perpendicular to the long axis of the upper arm and forearm, were considered as tangential.Accelerations along the long axis were defined as radial (not analyzed in this paper).This resulted in pairs of relative radial and tangential accelerations associated with the forearm and upper arm in local coordinate frames (Figure 1) for proximal (p) and distal (d) sensor placements: (Eq. 1) where, r p and r d are proximal and distal distances to the accelerometers from an arbitrary reference point, ω 2 is the square of the angular velocity, α is the absolute angular acceleration of the bar, and are the absolute accelerations experienced by the sensors, and is the unknown acceleration of the reference point.The vectors and are unity vectors denoting the radial and tangential directions, respectively, for the segment in the relative segment reference system.As stated above, we present only the results obtained from the tangential components.

Data processing
All data processing stored during acquisition was within MatLab (ver.7.3.0.267 (2006b), MathWorks, Inc., USA).The difference between the voltages from two outputs of sensors (p and d ) attached to the same segment (Equations 1 and 2) yielded four acceleration components, two for the forearm and two for the upper arm: where D is the distance between the sensors.
The radial component of the acceleration is proportional to the square of the absolute angular velocity while the tangential component is proportional to the absolute angular acceleration.The coefficient of proportionality for both components is the distance between the two accompanied sensors (D, identical for the upper arm and forearm).
Signals representing movements to targets from tangential components were extracted and used for further analysis.Data were low-pass filtered with a 4th order Butterworth filter using a cutoff frequency of 2.5 Hz.
Reaching synergies were formed as a relationship between the forearm and upper arm tangential angular accelerations.Their changes were plotted for movements across the para-sagittal plane.The phase plots with reaching synergies from all the movements were also inspected visually.

Neural network design
The subjects' data were arbitrarily divided into two groups.One group had five subjects whose data were used for training (5 of 7), while the other group had 2 subjects whose data were used for validation (2 of 7).Most of the data (75%) from the first group were used for training while the rest of it was used for validation of the generalization.
ANN was implemented using the Neural Network Toolbox in MatLab.We selected a radial basis function ANN (RBF ANN) with an input layer using the radial basis function and hidden layers of pure linear functions.The networks were trained using the newrb function where the constants (goal and spread ) were empirically evaluated for each ANN.
Although it is possible to use raw signals as input to the ANN, we used filtered signals.Signals used in this study were tangential angular accelerations.The ANNs were trained by using upper arm data as input and forearm data as output signals.Five different RBF ANNs were trained, in which each network related to a different task.Pearson's correlation coefficients (k ij , i,j = 1-5) between desired (actual) trajectory and predicted with RBF ANN were computed.One coefficient was estimated for each RBF ANN i to each task j.For a good correlation the threshold was set at 0.9.
Our hypothesis was that for specific movements, the trained ANN would predict output with a high correlation (>0.9) when compared to the desired output each time the network was exposed to a new input.For all other movements, the trained ANN will predict output with a low correlation coefficient (<0.9).We visually inspected all phase plots and found that as subjects moved to different areas in the workspace, the shape and orientation of reaching synergies varied substantially.Other variations were also noticed.For example, subject 3 performed all of the trials of task 2 in a similar way (Figure 4A) but not of task 1 (Figure 4B).Subjects 1 and 2 performed task 1 in a similar way (Figure 4C), but subjects 3 and 4 performed task 2 differently (Figure 4D).
After visual inspection of all phase plots, signals were ready for ANN training.A total of 75% of data sets from the training group (5 subjects) were used to train RBF ANNs.Upper arm and forearm tangential angular accelerations for one task were the inputs and outputs to the network, respectively.One RBF ANN i corresponded to one task T i (i = 1-5).
For the remaining 25% of data sets, forearm angular accelerations were predicted by RBF ANN and compared to the actual (desired) accelerations.Figure 5 illustrates the computed correlation coefficients that were obtained with varying degrees of success.The left panels show ANN outputs with high correlation in the case of testing with training data (Figure 5A) and validation data (Figure 5C).On the contrary, the panels on the right are for the cases with less success in testing with training data (Figure 5B) and validation data (Figure 5D).
In the overall correlation coefficient analysis, we calculated the mean (k) and standard deviation (SD) for the 25% of data from the training group that were not used for training (Table 1).High mean values for correlation coefficients (0.9303 < k i,i < 0.9709, i = 1-5) and low standard deviations (0.0247 < SD < 0.0449) were obtained for all   The overall classification accuracy was tested in cross-correlation analysis between each ANN and each task.Data are reported as a summary means of all data for 5 trained networks and for 5 tasks.Mean values in bold type are for validation group from Table 1.
comparisons.The highest mean value and the lowest SD were obtained for ANN1.The opposite was found for ANN5.Minimal and maximal correlation coefficients ranged from 0.8414 (ANN3) to 0.8951 (ANN1) and from 0.9825 (ANN5) to 0.9982 (ANN2), respectively.The same computations were analyzed for the data from the validation group (2 subjects; Table 1).In comparison to the training group all of the mean values for the validation group were slightly lower, but always greater than 0.9.The mean values of the correlation coefficients ranged from 0.9149 to 0.9497 and the SD ranged from 0.0434 to 0.0610.Minimal varied from 0.7533 (ANN1) to 0.8322 (ANN3) and maximal varied from 0.9854 (ANN2) to 0.9936 (ANN3).
Finally, to test the overall classification accuracy, the forearm tangential angular accelerations predicted by one RBF ANN were compared to the actual (desired) for all tasks (RBF ANN i vs T j , where i,j = 1-5).The correlation coefficients were analyzed for validation data only.The classification results for all five of the RBF ANNs for each task are reported in Table 2.As already described in the previous paragraph, high correlation coefficients (0.9149 < k i,i < 0.9497) were obtained (values in Table 2 in bold forming a diagonal) for all RBF ANN i vs T i comparisons.As expected, the remaining correlation coefficients (k i,j , i ≠ j) were lower than 0.9.They ranged from k 5,1 = 0.0316 for ANN5 applied to data collected in task 1 to k 2,3 = 0.8302 for ANN2 used for the data measured during task 3.
Performance differences among five of the ANNs were variable for each task.We calculated the critical differences between the mean correlation coefficients for the same task.They ranged from 0.04 for task 2 between ANN2 and ANN3 to 0.47 for task 5 between ANN5 and ANN2.For Tasks 1, 3, and 4, the critical differences were 0.24, 0.11, and 0.26, respectively.

Discussion
We propose a method for the control of neural prostheses that can be used with available systems that apply either surface or implantable interfaces to sensory-motor systems.The control strategy was based on mimicking an output space model of natural control, determined from motor synergies of reaching movements of healthy humans.It is specifically designed for neuro-rehabilitation. Hand movement was performed in the sagittal plane comprising elbow and shoulder joints.We constrained our setup to a sitting position and considered five different horizontal levels.The starting point and movement direction were controlled, rather than selected by the subjects.
We evaluated five different tasks using various shoul-der motions, ranging from substantial to those with less amplitude.The selected tasks required the coordination of both shoulder and elbow joint movements (35).The influence of the shoulder on the elbow motion is much more powerful than vice versa (34).This is due to specific inertial properties of the upper arm and forearm, as well as to the more proximal position of the upper arm in the chain of segments comprising the arm.These movements are of particular interest for the rehabilitation of patients with a paralyzed/paretic arm and hand as they belong to the repertoire of ADL.Selected reaching tasks led to combining "elbow extension and shoulder adduction and/or flexion" or "elbow flexion and shoulder abduction and/or extension".We selected a radial basis function network as the universal function approximator (37), since it is able to model any continuous function between the inputs and outputs for a given number of sufficient training samples and unlimited hidden units.Our findings indicate clear differences in the correlation coefficients for ANN1, ANN4 and ANN5 from other ANNs; however, better results are desired for ANN2 and ANN3.
An objective measure of the discrepancy of hand movements is clinically important for assessment and diagnosis, as well as for rehabilitation outcome to allow for evidence-based practice (1,2).The method described in this study was primarily designed for the active-assistance control of the forearm (3,4).It may also be applicable for the quantitative assessment of the deviation from normal movement.Trained RBF ANN with data from healthy subjects may be used for movement evaluation of individuals whose data were not used for the training.The relationship between the correlation coefficient k (Tables 1 and 2) and clinical scales (e.g., Fugel-Mayer scores, Aschworth grades) could be evaluated in another study with stroke survivors.As such, coefficient k could be used for estimation of preserved motor skills based on kinematics (38,39).
We modeled a reaching task as a two-link mechanism.The upper arm and forearm motion trajectories during vertical arm movements were estimated from the measured tangential angular accelerations, and their correlation was documented by using RBF ANNs.Our results suggest that the proposed method successfully discriminates between different motions.This approach to control forearm motion from upper arm kinematics is of great significance particularly for humans with SCI in a free-living environment.
The measurement system of using dual-axis accelerometers, developed for this study, could also be used for an objective assessment of upper limb activity.For this purpose, training-related changes in synergies apparent from movement kinematics during rehabilitation would characterize the extent and the course of recovery.As such, a simple system using this methodology is of particular importance for stroke patients.
Limitations of this study include the use of a small number of participants and the use of only non-impaired participants.The selection of movements evaluated in this study was based on our personal opinion of functionally useful actions.Other heights for initial and target hand positions would extend the repertoire of activities that could be achieved.Future studies should incorporate full body reaching tasks (40).

Figure 2 .
Figure 2.An example of the raw data: signals in tangential direction from sensors attached to upper arm proximally (full line) and distally (dashed line) and from sensors attached to forearm proximally (dotted line) and distally (dot-dashed line).Eight accelerations were measured during a movement; four used for this study are shown.After completing each task, the subjects were instructed to retract the arm back to the starting position.Only trajectories from initial to target position were plotted.Note that movement duration was approximately 1.3 s.

Figure 3 .
Figure 3. A, One representative example of the processed data: filtered differences of tangential components of angular accelerations (αD) are calculated according to Equation 3 (see text).The data are divided by the distance D between accelerometers and used as input and output, respectively, for radial basis function artificial neural network training and validation.Full line is for upper arm and dashed line for forearm.B, Examples of phase plots formed by angular acceleration of upper arm (UA) and forearm (FA).Five different tasks (T i , i = 1-5) performed by one subject are shown.One trial for each of five tasks is presented.Notice different shape and size of reaching synergy patterns.

ResultsA
typical arm movement to a target for task 2 is shown in Figure 2. Movement duration ranged from 1.1 to 1.5 s.All signals were processed according to Equation 3.After filtering, representative signals are shown in Figure 3A.Phase plots formed by tangential angular acceleration data from upper arm and forearm reaching synergies for one subject and one trial per task for five different tasks are illustrated in Figure 3B.

Figure 4 .
Figure 4. Phase plots formed by angular accelerations (α) of upper arm (UA) and forearm (FA).Various examples of acquired data are shown: A, One subject (#3) repeating task 2 similarly in four trials.B, The same subject (#3) repeating task 1 differently in four trials.C, Task 1 performed by two subjects (#1 and #2) in a similar way; full line is for one trial of subject #1 and dashed line is for one trial of subject #2.D, Task 2 performed by two subjects (#3 and #4) in different ways; full line is for one trial of subject #3 and dashed line is for one trial of subject #4.

Figure 5 .
Figure 5.The output (angular acceleration (α) in rad/s 2 vs number of samples) predicted from radial basis function artificial neural network (full line) and the desired (actual) trajectory (dashed line).Correlation coefficients (k) reported in the upper right-hand corners for each desired/predicted pair.Left panels are examples of high correlation (A and C) and right panels are for low correlation (B and D).Upper two panels (A and B) are for the data not used for network training in the training group (25% of data from 5 of 7 subjects) and lower two panels (C and D) are for validation data (from 2 of 7 subjects).FA = forearm.

Table 1 .
Person's correlation coefficients (k i,i , i = 1-5) between desired (actual) and predicted by radial basis function (RBF) artificial neural network (ANN) forearm angular accelerations computed for training and validation data.The ANNs were trained by using upper arm data as input and forearm data as output signals.Five different RBF ANNs were trained in which each network related to a different task.Data are reported as means ± SD for the summary of all data for five trained networks, each for one task.Values in bold indicate the best ANN performance.