Psychometric properties of the portuguese version of the Jebsen-Taylor test for adults with mild hemiparesis

Avaliação das propriedades pscicométricas da versão em português do teste de Jebsen Taylor para adultos com hemiparesia leve

Psychometric properties of the portuguese version of the Jebsen-Taylor test for adults with mild hemiparesis

Avaliação das propriedades pscicométricas da versão em português do teste de Jebsen Taylor para adultos com hemiparesia leve

Karina N. Ferreiro^I; Renata L. dos Santos^I; Adriana B. Conforto^I,II

^ILaboratório de Neuroestimulação, Divisão de Clínica Neurológica, Hospital das Clínicas, Faculdade de Medicina, Universidade de São Paulo, São Paulo (SP), Brazil

^IIInstituto Israelita de Ensino e Pesquisa, Hospital Israelita Albert Einstein, São Paulo (SP), Brazil

^{Correspondence to}

ABSTRACT

OBJECTIVES: To evaluate the psychometric properties of the Portuguese version of the Jebsen-Taylor Test (JTT) in patients with stroke.

METHODS: Forty participants who suffered a stroke in the cerebral hemisphere were videotaped while performing the JTT. Scores were defined by the time taken to perform the tasks, and two physical therapists evaluated the performance of the participants. Intra- and inter-rater reliability was defined by intraclass correlation coefficients (ICC) through videotape analysis. Cronbach's alpha and Pearson's correlation coefficient (r) were used to measure the internal consistency of the scale. Confidence intervals (CI) were calculated, and the influence of handedness and educational level on the JTT scores was evaluated.

RESULTS: Inter-rater (ICC = 1.0; CI, 1.0-1.0) and intra-rater reliabilities (ICC=0.997; CI, 0.995-0.998) were excellent. Regarding internal consistency, Cronbach's α was 0.924. The item "writing a sentence" was less consistent than the other items (Cronbach's alpha=0.884). Pearson's r (item score - total score) was lower for the item "small objects" (r=0.657). There was no significant influence of handedness or educational level on the JTT scores.

CONCLUSIONS: Videotaping test performances can be a useful tool in multicenter studies if inter-rater reliability is appropriate. The inter- and intra-rater reliabilities of the Portuguese version of the JTT were excellent in patients with stroke. The JTT can be a valuable tool for evaluating dexterity in research protocols aiming at efficacy of rehabilitation interventions.

Key words: Stroke; reproducibility of results; disability evaluation; motor skills; rehabilitation; activities of daily living.

RESUMO

OBJETIVOS: Avaliar as propriedades psicométricas da versão em Português do teste de Jebsen-Taylor (TJT) em pacientes com acidente vascular encefálico (AVE).

MÉTODOS: Quarenta pacientes com AVEs em hemisférios cerebrais foram filmados enquanto realizaram o TJT. A pontuação no teste é definida pelo tempo de execução de tarefas motoras. Duas fisioterapeutas avaliaram o desempenho dos pacientes. Por meio das análises dos vídeos, foram determinadas as confiabilidades intra e interexaminador, pelos coeficientes de correlação intraclasse (CCI). O alfa de Crobach e o coeficiente de correlação de Pearson (r) foram utilizados para medir a consistência interna da escala. Foram avaliados os efeitos de dominância manual e escolaridade sobre a pontuação no TJT.

RESULTADOS: Houve excelentes correlações interexaminador (CCI=1,0; intervalo de confiança, 1,0-1,0) e intraexaminador (CCI=0,997; intervalo de confiança, 0,995-0,998). Na avaliação da consistência interna, o alfa de Cronbach total foi 0,924. O item "escrever uma frase" teve consistência menor que os demais itens (Cronbach's alpha=0,884). O coeficiente de correlação de Pearson (item - total da escala) foi mais baixo para o item "objetos pequenos" (r=0,657). Não houve efeitos significativos de dominância manual ou escolaridade, sobre a pontuação no TJT.

CONCLUSÕES: As confiabilidades interexaminador e intraexaminador foram excelentes, assim como a consistência interna da versão em Português do TJT em pacientes com AVE, avaliada por meio de vídeos. Essas são informações importantes para o planejamento de protocolos de reabilitação voltados para a melhora da função do membro superior em pacientes com AVE.

Palavras-chave: acidente vascular encefálico; reprodutibilidade dos resultados; avaliação da deficiência; destreza motora; reabilitação; atividades cotidianas.

Introduction

Stroke is a leading cause of death in Brazil¹, and the most frequent neurological impairment is hemiparesis², which can lead to a decrease in function and ability in activities of daily living (ADLs). In general, the functional recovery of the upper limbs is slower and less efficient than lower limb recovery³. It has been estimated that 45% of stroke patients do not recover upper limb function². Hand function has a significant impact on disability in patients with stroke^2,4, therefore tools to evaluate upper limb function are highly relevant to stroke rehabilitation.

Grasping, holding and manipulating objects are ADLs that may be affected after stroke⁵. The Jebsen Taylor Test⁶ (JTT) (Figure 1), described in 1969, evaluates these activities and consists of seven tasks: writing a sentence, card turning, small common objects, simulated feeding, stacking checkers, moving large light objects and large heavy objects. The JTT has good test-retest reliability⁶, good concurrent validity with other tests of upper limb dexterity⁷, and it has been used in many studies on the effects of somatosensory or cortical stimulation on upper extremity function in patients with stroke and spinal cord injury^8-12.

The aim of the present study was to evaluate the following psychometric properties: inter-rater reliability, intra-rater reliability and internal consistency of the videotaped Portuguese version of the JTT in patients with hemiparesis following stroke. In addition, we investigated the influence of handedness and educational level on test performance.

Methods

Patients were recruited from the Cerebrovascular Diseases Clinic and the Neurology Emergency Department of our institution. The inclusion criteria were: hemiparesis following a single, ischemic stroke confirmed by computed tomography or magnetic resonance imaging; a minimum of 30 days since stroke; age between 18 and 80 years; ability to understand instructions and perform all JTT tasks. The exclusion criteria were: multiple brain lesions; severe joint deformity; severe heart or lung disease, advanced cancer. Forty-eight patients fulfilled the inclusion criteria, and eight patients were excluded due to inability to perform all JTT tasks. Therefore, forty patients were included in the study. Age, gender, years of education (<5 years or >5 years), time since stroke and handedness (according to the Oldfield Inventory¹³) were recorded.

The JTT instructions were translated and adapted to the Portuguese language. Back-translation was performed by an expert in English¹⁴. We manufactured a board with the characteristics described by Jebsen et al.⁶: 105.4 cm long, 28.6 cm wide and 1.1 cm thick. The front edge (1.1 cm thick) of the board was marked at 10.1 cm intervals. A vertical barrier (50.8 cm long, 5.08 cm wide and 1.3 cm thick) was glued to the board 11.7 cm from the right end and 15.2 cm from the front of the board. The front of the vertical barrier was marked at 5.1 cm intervals beginning 2.5 cm from each end for referencing in object placement.

Cards were placed 5.1 cm apart and 12.7 cm from the front edge of the desk. For the "small objects" task, a can was placed 12.7 cm from the front edge of the desk. Six small objects were placed in a horizontal row to the left of the can: two paper clips positioned vertically, two regular-size bottle caps with the inside of the cap facing up and two US one-cent coins. The paper clips were placed at the far left and the coins next to the can. The distance between objects was 5.1 cm.

For the "simulated feeding" task, five beans were placed on the board in front of the participant, 12.7 cm from the front edge of the desk, positioned to the left of the center, parallel to and touching the vertical barrier, and 5.1 cm apart. A can was placed in front of the board. For the "stacking checkers" task, four checkers (3.2 cm diameter) were placed on the table in contact with the front of the board, at a distance of 12.7 cm from the front edge of the desk. For the "moving objects" task, five light cans (height, 9.5 cm; diameter, 7.5 cm) were placed in front of the board, 5.1 cm apart. In addition, five heavy cans (weighing 1 pound) were positioned as described above.

The commands given to the patients to complete the tasks were the same as those described in the original article, and all tests were videotaped. Rater 1 (R.L.S.) evaluated the participants and reevaluated the tapes on separate occasions, blinded to the performance time measured in the first evaluation. Rater 2 (K.N.F) evaluated the videos and was blind to the performance time measured by Rater 1. Intra- and inter-rater reliabilities were evaluated by intraclass correlation coefficients (ICC)¹⁵. The internal consistency¹⁵ of the scale was checked with Pearson's¹⁶ r (total score versus each item and total score versus total score minus each item), and Cronbach's α¹⁷ (total score versus total score minus each item). The raters also recorded the number of mistakes performed during the JTT: misspelled words; changes in the strategy to turn the cards; dropped small objects, beans, checkers, or cans.

The JTT scores for the paretic, dominant hand were compared to the scores for the paretic, non-dominant hand using the Mann-Whitney test. This test was also used to compare scores between participants with higher (>4 years) and lower levels (≤4 years) of education. The Wilcoxon test was used to compare the number of mistakes recorded by Raters 1 and 2. The Mann-Whitney test was used to evaluate differences in JTT scores between participants with lesions on the right and left hemispheres. A p-value of 0.05 was considered statistically significant. The experimental protocol was approved by the Ethics Committee of Hospital das Clínicas/ Universidade de São Paulo, São Paulo (SP), Brazil (protocol numbers 1049/04 and 279/05). All patients gave their written informed consent.

Results

The mean age (±SD) was 52.5±16.1 years, and 42.5% of the participants were male. Fifty percent had 5 to 16 years of education. The mean interval between stroke onset and testing was 214 (±141.9) days; 57.2% of the participants had hemiparesis on the dominant side, and 92.5% were right-handed according to the Oldfield Inventory¹³.

The ICC was 0.997 (0.995-0.998) for intra-rater reliability and 1.0 (1.0-1.0) for inter-rater reliability. Inter-rater reliability for each of the tasks is shown in Table 1. Pearson's correlation coefficients and Cronbach's alpha were used to assess internal consistency and are shown in Table 2. The internal consistency of the test was good (Cronbach's α=0.924). Regarding each item, "writing a sentence" was less consistent than the other tasks (Cronbach's α=0.884, total versus total minus item). Pearson's r was lower for the task "picking up small objects" (r=0.657).

There were no significant differences in JTT scores between the participants with different levels of education (p=0.291). In addition, there were no significant differences (p=0.277) in the JTT scores measured in the paretic, dominant hand (128.6±80.5 seconds) and those measured in the paretic, non-dominant hand (139.3±41.1 seconds). Table 3 shows individual scores for each task. The median number of mistakes in the JTT was 1 (range, 0-10). There were no significant differences (p=0.531) in the number of mistakes, as evaluated by each of the raters (Rater 1: median 1, range 0-8; Rater 2: median 1, range 0-10).

Discussion

Our results showed excellent intra- and inter-rater reliabilities of the JTT scale in stroke patients by videotape analysis. The psychometric properties of the scale were similar to those reported by the videotape analysis of the Wolf Motor Function (WMFT)¹⁸ and the Action Research Arm Test (ARAT)¹⁹. The WMFT is a detailed, validated laboratory-based scale for evaluation of upper-limb performance that has been widely used, particularly in clinical trials of constraint-induced therapy^9,20-23. The test can be videotaped for off-line scoring. Concerns have been raised regarding the limitations of the WMFT in functional evaluation^23,24. The tasks in this test are not directly related to ADLs, and the scores are based on the time taken to complete predefined tasks and on the evaluation of coordination and fluidity, as well as clinically relevant characteristics of movement¹⁸. The ARAT is composed of 19 tasks that include grasping, gripping, pinching and gross movement subscales. Rating is based on the ability to properly perform the tasks. The test can be administered relatively quickly, can be videotaped and is sensitive to clinically meaningful changes in upper limb ability^19,25. Standardized instructions for administration have been published²⁶. The ARAT tasks are less related to ADLs than the JTT tasks.

The Arm Motor Ability Test (AMAT)²⁷ is composed of ten tasks for the upper limb that resemble ADLs, and the inter-rater reliability has been shown to be appropriate. The test is time consuming, which may be an obstacle to widespread use. The original version of the Test d'Évaluation des Membres Supérieurs de Personnes Âgées (TEMPA)²⁸ is comprised of nine tasks (5 bimanual and 4 unimanual) and evaluates quantitative and qualitative aspects of motor performance. The ICC for inter-rater reliability of the translated version of the TEMPA (with 8 tasks) was 0.94. This is lower than the coefficient obtained in the present study. AMAT and TEMPA assess many bimanual activities, whereas JTT evaluates unimanual tasks. Although the number of tasks in the adapted version of TEMPA and the JTT is quite similar, the TEMPA can be more time-consuming because it includes more complex tasks and because the functional rating and the quality of the performed tasks are also analyzed.

The intra-rater reliability of the JTT was excellent, and the inter-rater reliability was higher than the inter-rater reliability reported by the videotape analysis of the Wolf Motor Function¹⁸ (WMFT; ICC=0.97) and the Action Research Arm Test¹⁹ (ARAT; ICC=0.98). The internal consistency of the JTT scale was similar to that of the WMFT scale ( α=0.92). However, the items "writing" and "picking up small objects" were slightly less consistent than the others. The relatively low level of education and the fact that most patients had lesions in the left hemisphere (frequently associated with language disorders) might explain the results obtained in the item "writing a sentence". The likely explanation for this finding is that this item reflects not only dexterity but also language abilities. With regard to the item "picking up small objects", the performance of more precise pinch movements, requiring higher levels of dexterity, may have contributed to the lower consistency of this item, compared to the others.

There were no significant differences between the scores obtained in the item "writing a sentence" for the participants with lower and higher levels of education or in the participants performing the task with the paretic, dominant hand compared to those performing the test with the paretic, non-dominant hand. The sample size may have limited the statistical power to make these comparisons. Alternatively, the effect of impairment due to stroke on performance in this item may have had a greater magnitude than the effects of handedness or education.

A limitation of this study is that the inter- and intra-rater reliability was based on the evaluation of videotapes by the raters. Rater 1 evaluated the participants in person and, for intra-rater reliability, watched the videotapes blinded to the previously scored results. Rater 2 evaluated only the videotapes. It is theoretically possible that reliability would be lower if the participants were not instructed in a standard manner, or if the scores were based on a less objective evaluation than on time to complete the tasks. The need to provide the same written instructions to patients and to follow the recommendations to position the participant and the objects cannot be overemphasized. Still, our results show that, if patients are taped while performing the JTT, reliable scores can be given by different raters and by the same rater on separate occasions. This information is important to plan rehabilitation trials.

In other scales that are widely used in neurology studies, videotapes are often employed to train raters, as well as to check intra- and inter-rater reliability, and the results are sometimes not as good as those reported here. For instance, videotapes have been used in the certification of a widely used scale of neurological impairment in patients with stroke, the NIH stroke scale²⁹ (NIHSS). The NIHSS is often used in research protocols and in clinical practice because the score in this scale is part of the criteria to perform intravenous thrombolysis in acute stroke patients³⁰. Nevertheless, NIHSS overall scoring by analysis of videotapes by a large diverse sample of physicians has been shown to be inconsistent³¹. In contrast, inter-rater reliability based on the videotape analysis of the WMFT¹⁸ has been reported to be as high as that reported in the present study.

The objects used in the test can be easily purchased, and the implementation of the JTT is inexpensive. The instructions are simple, straightforward, and expertise on administering the test is not time-consuming. Another important advantage of the JTT scale is the evaluation of movements related to ADLs, even though the WMFT includes some tasks that are commonly performed in daily living (such as turning a key in the lock, lifting a basket), other tasks (such as elbow extension, moving the hand to a box) are more laboratory-based^23,24. In addition, the WMFT is more time-consuming than the JTT.

The JTT has some limitations. The test rates speed, but does not rate different strategies of task performance. Different compensation mechanisms to position the upper limb during the JTT will not be reflected on the scores. Hence, it is important to provide appropriate instructions before starting the test and to ask patients not to change their strategy while being tested or, in clinical trials that use the JTT score as an endpoint, not to change strategies in follow-up evaluations. Furthermore, patients with moderate to severe functional impairment are often not testable with the JTT. Therefore, the test is appropriate to evaluate dexterity in patients with slight to moderate hand disability.

The main advantage of the JTT is to provide an objective measure of hand function, employing functionally relevant tasks, with good intra- and inter-rater reliability. We conclude that the Brazilian Portuguese version of the JTT is an important scale in protocols that investigate the efficacy of rehabilitation interventions on upper extremity function in hemiparetic stroke patients.

Acknowledgments

Researchers received funding from the Fogarty International Center, the National Institutes of Health (1R21 TW006706) and Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP) (07/53123-1).

Received: 02/03/2009

Revised: 19/08/2009

Accepted: 21/08/2009

• ¹
  Ministério da Saúde – Datasus [homepage na Internet]. Mortalidade - Brasil. Óbitos por ocorrência por sexo segundo causa CID-BR-10. Estatísticas vitais-mortalidade e nascidos vivos/mortalidade geral/ abrangência geográfica: Brasil por região e unidade da Federação. Brasília: Ministério da Saúde – Datasus; c2008 [atualizada em 2008; acesso em 25 Jul 2008]. Disponível em: http://www.datasus.gov.br/
• 2. Gresham GE, Duncan PW, Stason WB, Adams HP, Adelman AM, Alexander DN, et al. Clinical practice guideline number 16: post-stroke rehabilitation. Rockville: US Department of Health and Human Services, Public Health Service, Agency for Health Care Plicy and Research; 1995.
• 3. Desrosiers J, Malouin F, Richards C, Bourbonnais D, Rochette A, Bravo G. Comparison of changes in upper and lower extremity impairments and disabilities after stroke. Int J Rehabil Res. 2003;26(2):109-16.
• 4. Kwakkel GK, Kollen BJ, Van der Grond J, Prevo AJ. Probability of regaining dexterity in the flaccid upper limb: impact of severity of paresis and time since onset in acute stroke. Stroke. 2003;34(9):2181-6.
• 5. Alon G, Levitt AF, McCarthy PA. Functional electrical stimulation enhancement of upper extremity functional recovery during stroke rehabilitation: a pilot study. Neurorehabil Neural Repair. 2007;21(3):207-15.
• 6. Jebsen RH, Taylor N, Trieschmann RB, Trotter MJ, Howard LA. An objective and standardized test of hand function. Arch Phys Med Rehabil. 1969;50(6):311-9.
• 7. Bovend'Eerdt TJ, Dawes H, Johansen-Berg H, Wade DT. Evaluation of the modified Jebsen test of hand function and the University of Maryland arm questionnaire for stroke. Clin Rehabil. 2004;18(2):195-202.
• 8. Kimberley TJ, Lewis SM, Auerbach EJ, Dorsey LL, Lojovich JM, Carey JR. Electrical stimulation driving functional improvements and cortical changes in subjects with stroke. Exp Brain Res. 2004;154(4):450-60.
• 9. Beekhuizen KS, Field-Fote EC. Massed practice versus massed practice with stimulation: effects on upper extremity function and cortical plasticity in individuals with incomplete cervical spinal cord injury. Neurorehabil Neural Repair. 2005;19(1):33-45.
• 10. Conforto AB, Cohen LG, dos Santos RL, Scaff M, Marie SK. Effects of somatosensory stimulation on motor function in chronic cortico-subcortical strokes. J Neurol. 2007;254(3):333-9.
• 11. Wu CW, Seo HJ, Cohen LG. Influence of electric somatosensory stimulation on paretic-hand function in chronic stroke. Arch Phys Med Rehabil. 2006;87(3):351-7.
• 12. Hummel F, Celnik P, Giraux P, Floel A, Wu WH, Gerloff C, et al. Effects of non-invasive cortical stimulation on skilled motor function in chronic stroke. Brain. 2005;128(Pt 3):490-9.
• 13. Oldfield RC. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia. 1971;9(1):97:113.
• 14. Guillemin F. Cross-cultural adaptation and validation of health status measures. Scand J Rheumatol. 1995;24(2):61-3.
• 15. Lopez MN, Charter RA, Mostafavi B, Nibut LP, Smith WE. Psychometric properties of the folstein mini-mental state examination. Assessment. 2005;12(2):137-44.
• 16. Kline RB. Principles and practice of structural equation modeling. New York: Guilford Press; 1998.
• 17. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(13):297-334.
• 18. Morris DM, Uswatte G, Crago JE, Cook EW3rd, Taub E. The reliability of the wolf motor function for assessing upper extremity function after stroke. Arch Phys Med Rehabil. 2001;82: 750-5.
• 19. Hsieh CL, Hsueh IP, Chiang FM, Lin PH. Inter-rater reliability and validity of the action research arm test in stroke. Age Ageing. 1998;27(2):107-13.
• 20. Wolf SL, Catlin PA, Ellis M, Archer AL, Morgan B, Piacentino A. Assessing wolf motor function test as outcome measure for research in patients after stroke. Stroke. 2001;32(7):1635-9.
• 21. Koski L, Lin JC, Wu AD, Winstein CJ. Reliability of intracortical and corticomotor excitability estimates obtained from the upper extremities in chronic stroke. Neurosci Res. 2007;58(1): 19-31.
• 22. Wittenberg GF, Chen R, Ishii K, Bushara KO, Eckloff S, Croarkin E, et al. Constraint-induced therapy in stroke: magnetic-stimulation motor maps and cerebral activation. Neurorehabil Neural Repair. 2003;17(1):48-57.
• 23. Miltner WH, Bauder H, Sommer M, Dettmers C, Taub E. Effects of constraint-induced movement therapy on patients with chronic motor deficits after stroke: a replication. Stroke. 1999;30(3): 586-92.
• 24. Dromerick AW, Lang CE, Birkenmeier R, Hahn MG, Sahrmann SA, Edwards DF. Relationships between upper-limb functional limitation and self-reported disability 3 months after stroke. J Rehabil Res Dev. 2006;43(3):401-8.
• 25. McDonnell M. Action research arm test. Aust J Physiother. 2008;54(3):220.
• 26. Yozbatiran N, Der-Yeghiaian L, Cramer SC. A standardized approach to performing the action research arm test. Neurorehabil Neural Repair. 2008;22(1):78-90.
• 27. Morlin ACG, Delattre AM, Cacho EWA, Oberg TD, Oliveira R. Concordância e tradução para o português do teste de habilidade motora do membro superior - THMMS. Revista Neurociências. 2006;14(2):6-9.
• 28. Michaelsen SM, Natalio MA, Silva AG, Pagnussat AS. Confiabilidade da tradução e adaptação do test d'évaluation des membres supérieurs de personnes âgées (TEMPA) para o português e validação para adultos com hemiparesia. Rev Bras Fisioter. 2008;12(6):511-9.
• 29. Josephson SA, Hills NK, Johnston SC. NIH stroke scale reliability in ratings from a large sample of clinicians. Cerebrovasc Dis. 2006;22(5-6):389-95.
• 30. Tissue plasminogen activator for acute ischemic stroke. The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. N Engl J Med. 1995;333(24):1581-7.
• 31. Caneda MAG, Fernandes JG, Almeida AG, Mugnol FE. Confiabilidade de escalas de comprometimento neurológico em pacientes com acidente vascular cerebral. Arq Neuropsiquiatr. 2006;64(3-A):690-7.

Publication Dates

• Publication in this collection
  27 Aug 2010
• Date of issue
  Oct 2010

History

• Reviewed
  19 Aug 2009
• Received
  02 Mar 2009
• Accepted
  21 Aug 2009