Reliability, comprehension and acceptability of the Portuguese version of the Motor Assessment Scale in stroke patients

Abstracts

CONTEXTUALIZAÇÃO: Avaliações motoras são extensivamente utilizadas na fisioterapia. A Motor Assessment Scale (MAS) é uma avaliação da função motora que tem se mostrado válida e confiável para pacientes pós acidente vascular encefálico (AVE). Entretanto, não foram encontrados estudos sobre a sua confiabilidade no Brasil. OBJETIVOS: Avaliar a confiabilidade inter e intra-avaliador da versão em português da MAS, aplicada em pacientes pós-AVE, e observar sua compreensão e aceitação por fisioterapeutas brasileiros sem treinamento prévio ao uso deste instrumento. MÉTODOS: Para verificar a confiabilidade interavaliador, 23 fisioterapeutas pontuaram, por meio de um vídeo, as habilidades funcionais de seis pacientes avaliados pela versão em português da MAS. A confiabilidade intra-avaliador foi obtida pela avaliação do vídeo de 15 pacientes por sete fisioterapeutas em duas ocasiões, com intervalo de três semanas. Ao final, os fisioterapeutas responderam a um questionário para avaliação da compreensão e aceitação do instrumento. Foram utilizados o Coeficiente de Correlação Intraclasse e Kruskall Wallis para análise entre avaliadores, e ICC e Wilcoxon para a intra-avaliador. RESULTADOS: Foi encontrada uma alta confiabilidade intra (ICC entre 0,80 a 0,97) e inter-avaliador (ICC entre 0,93 e 1,00). Houve excelente compreensão e aceitação da escala. Entretanto, houve dúvidas em determinados itens, quanto à pontuação dos pacientes avaliados. CONCLUSÕES: A versão em português da MAS mostrou-se confiável e obteve excelente aceitação pelos fisioterapeutas. Porém, devido às dúvidas na pontuação, sugere-se um treinamento prévio à utilização clínica da escala. A partir destes dados, sugere-se um futuro estudo sobre validação da versão em português deste instrumento de avaliação.

acidente cerebrovascular; fisioterapia; avaliação; reprodutibilidade dos testes

ORIGINAL ARTICLE

Reliability, comprehension and acceptability of the Portuguese version of the Motor Assessment Scale in stroke patients

Ana L. F. Conte^I; Paula P. Ferrari^I; Thais B. Carvalho^II; Patrícia C. A. Relvas^II; Rita C. M. Neves^III; Sérgio F. Rosa^III

^IChild Physical Therapy, Disabled Children's Aid Association (Associação de Assistência à Criança Deficiente - AACD), São Paulo (SP), Brazil

^IIHydrotherapy, AACD

^IIIAdult Physical Therapy, AACD

Correspondence to

ABSTRACT

BACKGROUND: Motor assessments are frequently applied by physical therapists. The Motor Assessment Scale (MAS) evaluates motor function and has been shown to be a reliable and valid instrument for stroke patients. However, no previous study has examined its reliability in Brazil.

OBJECTIVES: The aim of this study was to determine the inter- and intra-rater reliability of the Portuguese version of the MAS in chronic stroke patients and to observe its comprehension and acceptability by Brazilian physical therapists without prior training on its use.

METHODS: For inter-rater analysis, 23 physical therapists scored the functional ability of six video-recorded stroke patients during assessment with the Portuguese version of the MAS; intra-rater reliability was determined by the assessment of 15 video-recorded stroke patients by seven physical therapists, on two separate occasions, three weeks apart. At the end of the study, the physical therapists answered a questionnaire in order to assess the comprehension and acceptability of the instrument. Statistical analysis was performed using the Intraclass Correlation Coefficient (ICC) and the Kruskall-Wallis non-parametric test to compare inter-rater data and using the ICC and the non-parametric Wilcoxon test to compare intra-rater data.

RESULTS: High inter- (ICC range 0.93-1.00) and intra-rater (ICC range 0.80-0.97) reliability was found. There was excellent comprehension and acceptability of the scale. However, in some items there were questions on how to score the patients.

CONCLUSIONS: The Portuguese version of the MAS was shown to be a reliable assessment instrument and had excellent acceptability by the physical therapists. However, due to the questions on scoring, we suggest that the physical therapists receive training on how to apply the scale. Further study is recommended on the validation of the translated version of this assessment instrument.

Key words: stroke; physical therapy; assessment; reproducibility of results.

Introduction

Stroke can be defined as a sudden interruption in vascular flow in the brain. It can be ischemic or hemorrhagic in nature and may lead to physical, cognitive and behavioral changes^1,2, physical capacity being one of the most affected domains³. In Brazil, stroke represents one of the main causes of morbidity and mortality⁴.

Post-stroke assessment instruments have become increasingly important^5-7 and are helpful in the rehabilitation process by making it easier to determine objectives, procedures and the treatment's efficacy^2,8. Several motor assessment scales have been developed and can be used by physical therapists^9,10. However, for clinical use, a scale must satisfy aspects such as time, easy management and scoring, cost and application⁹, in addition to being valid and reliable¹¹.

The aim of the Motor Assessment Scale (MAS) is to determine the motor function of post-stroke patients by means of eight motor tasks: supine to side-lying onto intact side, supine to sitting over side of bed, balanced sitting, sitting to standing, walking, upper arm function, hand movements, and advanced hand activities¹⁰. Initially, the scale included item 9 (General Tonus), which was deleted due to its poor reliability¹². This scale is easy to manage, provides objective measures^10,13 and has good indicators such as validity^14-16, reliability^10,14,15,17, sensitivity, simplicity, and clinical relevance^14,16-19. Furthermore, its emphasis on measuring function makes it useful for both research and clinical practice¹⁴.

Although two thirds of stroke cases occur in underdeveloped countries²⁰, most assessment instruments in rehabilitation have been developed in English, and the MAS¹² is no exception. We found no Portuguese version or studies on its reliability in Brazil. When planning to use an instrument in an investigation, ideally a previous study on reliability must be carried out, because this will determine whether the instrument has any flaws²¹. Thus, the objective of the present study was to evaluate the inter- and intra-rater reliability of the Portuguese version of the MAS and its comprehension and acceptability by Brazilian physical therapists without prior training on its use.

Methods

This study was approved by the Research Ethics Committee of the Disabled Children's Aid Association (Associação de Assistência à Criança Deficiente - AACD), under protocol number 027/2007. A Portuguese version of the MAS (Appendix 1) was prepared by the authors. Video footage was taken of fifteen ischemic or hemorrhagic stroke patients at various recovery stages, with different amounts of time since stroke, and with different degrees of impairment so that all possible scores would be represented in the scale. Patients with aphasy or other neurological diseases did not take part in the experiment. All of them signed an informed consent form. Twenty-three physical therapists with a background in post-stroke rehabilitation voluntary took part in the study. The mean age was 30.3±5.5 years, and the mean time since graduation was 7.5±4.6 years. Each physical therapist received a copy of the scoring criteria and the general rules of the scale one week in advance.

For the inter-rater reliability, six (out of the 15 patients) were randomly selected, five men and one woman, two with right hemiparesis, three with left hemiparesis and one with double hemiparesis with left-predominance. The mean age was 55.8±10.5 years, and the mean time since stroke was 37.5±23.3 months. The assessments of the six patients were scored by all 23 physical therapists. For the intra-rater reliability, 15 patients were assessed (nine men and six women), one of them with double hemiparesis with left-predominance, nine with left hemiparesis, and five with right hemiparesis. The mean age was 56.5±10 years, and the mean time since stroke was 32.7±22.7 months. The assessments of the 15 patients were scored by seven physical therapists on two occasions, with a three-week interval. The data referring to the patients' age, sex, time since stroke and type of stroke can be found in Table 1.

The number of patients and physical therapists chosen to be evaluated was based on requisites such as the availability of both patients and physical therapists, as well as factors inherent to the reliabilities being assessed. Thus, for inter-rater reliability, it was more important to have a significant number of therapists than patients, whereas for the intra-rater reliability, the criterion was the opposite. Only the distances and the times necessary to establish a score were provided. The physical therapists carried out the assessments individually and, at the end of the study, they answered a questionnaire for the analysis of the comprehension and acceptability of the version being used (Appendix 2).

The intraclass correlation coefficient (ICC) was used for reliability analysis, and the 95% confidence interval (95% CI) was also reported. ICC values below 0.40 represent a weak correlation; ICC<0.40 and ≤0.75 represents moderate correlation and ICC>0.75 represents high correlation²². The Kruskal-Wallis non-parametric test was used to compare the physical therapist assessments, and Wilcoxon's test was used to compare the two assessments of each physical therapist. The statistical analysis was carried out using the software Statistical Package for the Social Sciences (SPSS) version 10, with the significance level set at 0.05.

Results

The results for the comparison of the physical therapist scores are displayed in Table 2. The ICC and 95% CI values of the inter-rater reliability for each item of the Portuguese version of the MAS are also shown in Table 2. Reliability was high for all items, with values over 0.92.

As a complement to this analysis, the Kruskal-Wallis test did not show significant differences for any of the following items: item 1, p=0.514; item 2, p=0.972; item 3, p=1; item 4, p=0.987; item 5, p=1; item 6, p=1; item 7, p=0.985; item 8, p=0.982. Table 3 shows the ICC and 95% CI values for each one of the items in the Portuguese version of the MAS regarding the comparison between the first and second assessments of the seven participating physical therapists, i.e. the intra-rater reliability.

The Wilcoxon analysis of the data showed significantly different values for item 2 (p=0.004). Most of the physical therapists (78%) did not report any difficulties in the comprehension of the translated scale. However, 100% had questions on the scoring criteria. Item 8 generated the greatest percentage of questions, cited by 44% of the physical therapists, followed by items 3 (35%), 1 and 4 (30%), 7 (26%) and 2 (17%). As a result, 91% of the physical therapists pointed out the need to implement a training program before the scale is applied.

None of the physical therapists classified the Portuguese version of the MAS as excellent; 70% classified it as good; 26%, fair, and 4%, poor. Overall, 91% of the physical therapists reported that the scale makes it easier to visualize and specify the patients' therapeutic objectives, and 78% used it clinically. However, 13% claimed not to have assessed their patients with this instrument, and 9% were unsure.

Discussion

By reporting the characteristics, the value, and the effectiveness of the assessments in rehabilitation, it is possible to access the quality of the results and increase the efficacy of the research and of the assessment²³. To achieve that, it is necessary to study the reliability, because if it is poor, the error between different raters or between consecutive tests will increase²⁴. Assessment instruments must have suitable levels of reliability^2,6,17,25 to justify its clinical and research use¹⁰.

The MAS has been extensively used in research and clinical practice worldwide^12,15,26, and it includes both qualitative and quantitative aspects of functional tasks^12,14-16,26. It is one of the scales recommended for post-stroke patients¹⁸ and it has great potential in clinical assessment and in post-stroke rehabilitation research^13,14,26,27. In the present study, both intra- and inter-rater reliability of the Portuguese version of the MAS were evaluated, with the participation of 23 physical therapists and 15 post-stroke patients.

Inter-rater reliability

The results showed high inter-rater reliability for all items (ICC from 0.93 to 1.00), as seen in Table 2. No significant difference was found in the Kruskal-Wallis test. These figures corroborate the previous studies that found a value of 0.99 in Spearman's correlation coefficient¹⁴, inter-rater agreement of 95% (68% to 99%)¹⁰ and of 48% to 99%²⁸ for the English version and ICC between 0.72 and 1.00 for the Norwegian version¹².

It is important to note that item 1 also had the lowest values in previous studies, such as those by Carr et al.¹⁰ and by Poole and Whitney¹⁴. This item requires a more qualitative assessment of the required tasks^12,17. It is more difficult to obtain reliable results when the descriptions are based on the quality of the movement¹⁷ because the qualitative descriptions vary between physical therapists¹⁴.

Intra-rater reliability

High intra-rater reliability was found for all items of the Portuguese version of the MAS (ICC from 0.80 to 0.97), as observed in Table 3. Previous studies also found high reliability, with Pearson coefficient values of 0.98 (0.87 to 1.00)¹⁰. Contrary to expectations, the intra-rater reliability values were lower than the inter-rater ones. However, in both analyses, items 6, 5 and 2 had the highest ICC values, corroborating the results by Poole and Whitney¹⁴. Although a high rate of agreement was found in the intra-rater reliability test, the Wilcoxon test showed that item 2 had some significant difference between the two assessments by each of the seven physical therapists. It must be mentioned that the Wilcoxon test is not a specific test to evaluate the correspondence and agreement between different measures, but it is capable of identifying possible differences between the pairs of data. Thus, studies with a larger number of patients are needed to confirm the results found by this test.

Comprehension and acceptability

There was excellent comprehension of the scale, and only five physical therapists reported difficulty in understanding an item or instruction, which shows that the Portuguese version is clear. However, there were questions regarding the scoring criteria. The items that generated the most questions coincide with those with the lowest ICC values (items 1 and 3). Although item 8 had high reliability, it was the most mentioned in this aspect. It is likely that the scores were not organized in accordance with the increased difficulty in this item^14,26,27,29. A possible explanation for its high reliability would be that many patients are unable to carry out all the components of the upper extremity (UE) assessment, resulting in a reduction in error probability⁶. It was also verified that the MAS has a remarkable floor effect – the data reach the inferior level of the values due to the extreme difficulty of the tasks – in the UE items, indicating its non-functionality in most patients³⁰.

The percentage of questions found in this study calls attention to the need for the professionals to undergo a training program prior to the clinical use of the MAS, which was quoted by 90% of the participating physical therapists. According to Carr et al.¹⁰, the MAS can be used in a reliable way by physical therapists after instruction and a short period of practice (three weeks), and it must be tested in at least six patients for the individual to become familiar with the basic criteria and score in a reliable way. Because no certified training is available at present, discussions among physical therapists to clarify the standardization of the scoring criteria and the allowance of practice time to apply the scale are recommended. This is necessary because the assessment instruments are available in the scientific literature and must often be applied by the professionals without adequate instructions, leading to questions and contradictions and compromising the quality of the assessment.

It is worth noting that, although objective, the video-recording was carried out by amateurs, which may bring limitations such as difficulty in scoring, as only one camera was used, and each patient could only be seen in a single plane¹⁰. One may also question whether the use of the video for functional analysis is different to the real test in which the scoring criteria are based¹². It is also possible that the method used may have contributed to increase the reliability of the Portuguese version of the scale because the use of video images eliminates the variability of the physical therapists concerning how to apply the items of the scale and to give the command to the patient, therefore reducing the variability of the responses¹⁰.

In spite of these limitations and of the questions, the Portuguese version of the MAS was widely accepted by the participating physical therapists because most of them classified it as good and claimed that the instrument makes it easier to visualize and to specify the patients' therapeutic objectives. The results obtained show that the Portuguese version of the MAS has high reliability and good acceptability on the part of the physical therapists. Nevertheless, because of the great amount of questions concerning the scoring system, we suggest that these professionals undergo a training program before the clinical application of this scale.

Based on the favorable conclusions to its use, it is necessary to carry out a study on the validation of the Portuguese version of the MAS and on its internal consistency. This property shows the degree of correlation between the instrument's items and was not assessed in the present study. We also suggest that this process include a greater number of patients and raters (physical therapists), as well as a stricter methodology concerning the direct application of the scale without the use of video.

Acknowledgements

We would like to thank Dr. Milene S. Ferreira for the advice, and physical therapist Vera Leal for the technical assistance in editing the footage.

References

• 1. Ceccato RB. Aspectos clínicos: lesão encefálica adquirida. In: Moura EW, Silva PAC (editores). Fisioterapia: aspectos clínicos e práticos da reabilitação. São Paulo: Artes Médica; 2005. p. 257-69.
• 2. Maki T, Cacho EWA, Inoue M, Paz LP, Quagliato E, Nascimento NH, et al. Estudo da confiabilidade da aplicação da escala de Fugl-Meyer no Brasil. Rev Bras Fisioter. 2006;10(2):179-85.
• 3. Jaracz K, Kozubski W. Quality of life in stroke patients. Acta Neurol Scand. 2003;107(5):324-9.
• 4. Lotufo PA. Mortalidade pela doença cerebro vascular no Brasil. Rev Bras Hipertens. 2002;7(4):387-91.
• 5. Langhammer B, Stanghelle JK. Co-variation of tests commonly used in stroke rehabilitation Physiother Res Int. 2006;11(4):228-34.
• 6. Gladstone DJ, Danells CJ, Black SE. The fulg-meyer assessment of motor recovery after stroke: a critical review of its measurement properties. Neurorehabil Neural Repair. 2002;16(3):232-40.
• 7. de Oliveira R, Cacho EW, Borges G. Post-stroke motor and functional evaluations: a clinical correlation using fugl-meyer assessment scale, berg balance scale and barthel index. Arq Neuropsiquiatr. 2006;64(3B):731-5.
• 8. Jansa J, Pogacnik T, Gompertz P. An evaluation of the extended barthel index with acute ischemic stroke patients. Neurorehabil Neural Repair. 2004;18(1):37-41.
• 9. Horgan NF, Finn AM, O'Regan M, Cunningham CJ. A new stroke activity scale-results of a reliability study. Disabil Rehabil. 2003;25(6):277-85.
• 10. Carr JH, Shepherd RB, Nordholm L, Lynne D. Investigation of a new motor assessment scale for stroke patients. Phys Ther. 1985;65(2):175-80.
• 11. Brock KA, Goldie PA, Greenwood KM. Evaluating the effectiveness of stroke rehabilitation: choosing discriminative measure. Arc Phys Med Rehabil. 2002;83(1):92-9.
• 12. Kjendahl A, Jahsen R, Aamodt G. Motor assessment scale in Norway: translation and inter-rater reliability. Adv Physiother. 2005;7(1):7-12.
• 13. Williams BK, Galea MP, Winter AT. What is the functional outcome for the upper limb after stroke? Austr J Physiother. 2001;47(1):19-27.
• 14. Poole JL, Whitney SL. Motor assessment scale for stroke patients: concurrent validity and interrater reliability. Arch Phys Med Rehabil. 1988;69(3 Pt):195-7.
• 15. Lannin N. Reliability, validity and factor structure of the upper limb subscale of the motor assessment scale (UL-MAS) in adults following stroke. Disabil Rehabil. 2004;26(2):109-16.
• 16. Malouin F, Pichard L, Bonneau C, Durand A, Corriveau D. Evaluating motor recovery early after stroke: comparison of the Fugl-Meyer Assessment and the Motor Assessment Scale. Arch Phys Med Rehabil. 1994;75(11): 1206-12.
• 17. Loewen SC, Anderson BA. Reliability of the modified motor assessment scale and the barthel index. Phys Ther. 1988;68(7):1077-81.
• 18. Lennon O, Hastings M. Key physiotherapy indicators for quality of stroke care. Physiotherapy. 1996;82(12):655-64.
• 19. Carvalho BT, Relvas PCA, Rosa SF. Instrumentos de avaliação da função motora para indivíduos com lesão encefálica adquirida. Rev Neurocienc. 2008;16(2):137-43.
• 20. World Health Organization. The worl health report 2002. Geneva: World Health Organization; 2002.
• 21. Menezes PR. Validade e confiabilidade das escalas de avaliação em psiquiatria. Rev Psiquiatr Clin. 1998;25(5):214-6.
• 22. Fleiss JL. Statistical methods for rates and proportions. New York: John Wiley & Sons; 1981.
• 23. Gadotti IC, Vieira ER, Magee DJ. Importance and clarification of measurement properties in rehabilitation. Rev Bras Fisioter. 2006;10(2):137-46.
• 24. Lynden PD, Lau GT. A critical appraisal of stroke evaluation and rating scales. Stroke. 1991;22(11):1345-52.
• 25. van der Putten JJ, Hobart JC, Freeman JA, Thompson AJ. Measuring change in disability after inpatient rehabilitation: comparison of the responsiveness of the barthel index and the functional independence measure. J Neurol Neurosurg Psychiatry. 1999;66(4):480-4.
• 26. Aamodt G, Kjendahl A, Jahnsen R. Dimensionality and scalability of the motor assessment scale (MAS). Disabil Rehabil. 2006;28(16):1007-13.
• 27. Sabari JS, Lim AL, Velozo CA, Lehman L, Kieran O, Lai JS. Assessing arm and hand function after stroke: validity test of the hierarchical scoring system used in the motor assessment scale for stroke. Arch Phys Med Rehabil. 2005;86(8):1609-15.
• 28. Kjendahl A, Jahnsen R, Aamodt G. Reliability of the MAS instrument. Adv Physiother. 2005;8:12-7.
• 29. Dean C, Mackey F. Motor assessment scale scores as a measure of rehabilitation outcome following stroke. Austr J Physiother. 1992;38:31-5.
• 30. Hsueh IP, Hsieh CL. Responsiveness of two upper extremity function instruments for stroke inpatients receiving rehabilitation. Clin Rehabil. 2002;16(6):617-24.

Publication Dates

• Publication in this collection
  06 Nov 2009
• Date of issue
  Oct 2009

History

• Accepted
  24 Apr 2009
• Reviewed
  16 Mar 2009
• Received
  02 Sept 2008