Childhood autism: translation and validation of the Childhood Autism Rating Scale for use in Brazil

Abstracts

OBJECTIVE: To translate the Childhood Autism Rating Scale into Brazilian Portuguese and to determine the initial psychometric properties of the resulting version (CARS-BR). METHODS: The methodology used to produce an adequate version included translation, backtranslation and evaluation of semantic equivalence. In order to determine its psychometric properties (internal consistency, validity and reliability), the CARS-BR was administered to 60 consecutive patients with autism, aged between 3 and 17 years and seen at a university hospital. RESULTS: Internal consistency was high, with a Cronbach's alpha of 0.82. Convergent validity, in comparison with the Autistic Traits Assessment Scale, exhibited a Pearson's correlation coefficient of r = 0.89. When correlated with the Global Assessment of Functioning Scale in order to evaluate discriminant validity, the CARS-BR exhibited a Pearson's coefficient of r = -0.75. Test-retest reliability exhibited a kappa coefficient of 0.90. CONCLUSION: These results suggest that the CARS-BR is a valid and reliable instrument for evaluating autism severity in Brazil.

Autism; validation studies; questionnaires

OBJETIVO: Traduzir, adaptar e validar uma versão em português (do Brasil) da Childhood Autism Rating Scale (CARS). MÉTODOS: Após processo de tradução, a versão foi aplicada em 60 pacientes com diagnóstico de autismo infantil, de 3 a 17 anos de idade, selecionados consecutivamente de um ambulatório especializado a fim de analisar as propriedades psicométricas da nova versão (CARS-BR) (consistência interna, validade e confiabilidade). RESULTADOS: A consistência interna foi elevada, com valor de alfa de Cronbach de 0,82; a validade convergente (comparada com a Escala de Avaliação de Traços Autístícos) alcançou um coeficiente de correlação de Pearson de r = 0,89. Ao ser correlacionada à Escala de Avaliação Global de Funcionamento (para determinação da validade discriminante), a CARS-BR apresentou um coeficiente de correlação de Pearson de r = -0,75. A confiabilidade teste-reteste foi 0,90. CONCLUSÃO: A metodologia utilizada e os cuidados no processo de tradução permitem concluir que esse é um instrumento válido e confiável para avaliação da gravidade do autismo no Brasil.

Autismo infantil; estudos de validação; questionários

en_v84n6a04

ORIGINAL ARTICLE

Childhood autism: translation and validation of the Childhood Autism Rating Scale for use in Brazil

Alessandra Pereira^I; Rudimar S. Riesgo^II; Mario B. Wagner^III

^I

^IIDoutor. Neurologista pediátrico, Professor adjunto, Departamento de Pediatria, Faculdade de Medicina, UFRGS, Porto Alegre, RS, Brazil

^IIIPhD, University of London, London, UK. Professor adjunto, Departamento de Medicina Social, Programa de Pós-Graduação em Pediatria, Faculdade de Medicina, UFRGS, Porto Alegre, RS, Brazil

ABSTRACT

OBJECTIVE:

METHODS: The methodology used to produce an adequate version included translation, backtranslation and evaluation of semantic equivalence. In order to determine its psychometric properties (internal consistency, validity and reliability), the CARS-BR was administered to 60 consecutive patients with autism, aged between 3 and 17 years and seen at a university hospital.

RESULTS: Internal consistency was high, with a Cronbach's alpha of 0.82. Convergent validity, in comparison with the Autistic Traits Assessment Scale, exhibited a Pearson's correlation coefficient of r = 0.89. When correlated with the Global Assessment of Functioning Scale in order to evaluate discriminant validity, the CARS-BR exhibited a Pearson's coefficient of r = -0.75. Test-retest reliability exhibited a kappa coefficient of 0.90.

CONCLUSION: These results suggest that the CARS-BR is a valid and reliable instrument for evaluating autism severity in Brazil.

Keywords: Autism, validation studies, questionnaires.

Introduction

Although it was Bleuler, in 1911, who first described autism, it was Leo Kanner, in 1943, who defined the condition on the basis of observations of a group of children with peculiar behavior characterized by innate incapacity to establish affective and interpersonal contact.^1-3

Autism is a pervasive development disorder whose behavioral characteristics can be classified as four manifestations : qualitative deficits in social interaction, deficits in communication, repetitive and stereotypical behavior patterns and a restricted repertoire of interests and activities.⁴ In addition to these primary symptoms, autistic children often exhibit severe behavioral disorders, such as self-mutilation and aggression, in response to the demands of their environment, in addition to abnormal sensitivity to sensory stimuli.^3,5 In spite of decades of research and investigation, the etiology of autism remains undefined, since the disorder is complex and heterogenous with varying degrees of severity.^3,5 Several parts of the brain may be involved in the process of development of this pathology, including the cerebellum, hippocampus, amygdala, basal ganglia and corpus callosum, but the cellular and metabolic abnormalities that are the basis for this abnormal brain development are still unknown.^6,7 Progress in understanding the causes, nature and treatment for autism requires ever increasing integration between concepts, genetic findings, advances in cognitive neuroscience and clinical observations.^3,5,6

The prevalence of autism varies from 4 to 13/10,000, occupying third place among child development disorders in front of congenital malformations and Down Syndrome.^3,8 In the United States, at least one in every 1,000 children born will be diagnosed as having autism spectrum at some point during their development.^5,8 In the absence of any type of biological marker, diagnosis of autism is still clinical. The diagnostic criteria used are described in the American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders (DSM-IV).⁴ The DSM-IV criteria for autism diagnosis have a high level of specificity and sensitivity across a wide range of age groups and individuals with different levels of cognitive and language ability. Notwithstanding, other instruments are needed to evaluate symptoms in a qualitative manner and to refine differential diagnosis.⁹ The Childhood Autism Rating Scale (CARS) was developed over a 15-year period and is especially effective for distinguishing between mild, moderate and severe autism cases, in addition to discriminating between autistic children and those with mental retardation.^10-14 It offers several advantages over other instruments: it includes items representing a wide range of diagnostic criteria and which reflect the true dimensions of the syndrome, it is applicable to children of all ages, including preschool children, and also offers objective and quantifiable scores based on direct observations.¹⁰ Identification of autism is of fundamental importance and the use of a standardized and globally-accepted instrument offers accurate and reliable diagnosis in addition to making it possible to exchange information between different research centers.^14-16

Our objective in carrying out this study was to translate the CARS into Brazilian Portuguese, to adapt it and to validate it.

Methods

Population

The study was carried out between September of 2006 and April of 2007 at the Pervasive Development Disorder Clinic at the Hospital de Clínicas de Porto Alegre, RS, Brazil. Children and adolescents aged 3 to 17 years took part in the study, selected from among the clinic's patients and with diagnoses of primary autism according to the DSM-IV(inclusion criterion). The sample size was calculated on the basis of a Cronbach's alpha with a maximum margin of error of 0.1. Based on α = 0.05, it was estimated that the validation phase would require 60 patients with autism. The Brazilian Portuguese version of CARS (CARS-BR), the Autistic Traits Assessment Scale (ATA) and the Global Assessment of Functioning Scale (GAF) were all administered to a consecutive sample of 60 patients. Consecutive enrollment was based on order of attendance at the clinic for routine consultations. The CARS-BR was then administered a second time to 50 of the patients in the sample, chosen by lots.

The research objectives were explained to all of the patients' legal guardians and informed consent was obtained for each participant. This study was approved by the Research Ethics Committee at the Hospital de Clínicas de Porto Alegre and was analyzed and authorized by Western Psychological Services (WPS), who hold the copyright to the CARS.

Measures

The instruments employed in the present study are described below.

CARS

This is a 15-item scale which aids in the identification of children with autism and which distinguishes them from other children with compromised development but without autism. Its importance is based on its ability to differentiate mild-to-moderate from severe autism.^10-13 It is brief and is appropriate for use with any child over the age of 2 years. It was developed over a 15-year period on the basis of 1,500 autistic children. The scale incorporates diagnostic criteria based on the work of Kanner (1943), Creak (1961), Rutter (1978) and Ritvo & Freeman (1978) and from the 1980 Diagnostic and Statistical Manual of Mental Disorders (DSM-III).¹⁰ The scale evaluates behavior in 14 domains that are generally affected in autism, plus a single category for general impressions of autism.^9-12 These 15 items are as follows: relating to people, imitation, emotional response, body use, object use, adaptation to change, visual response, listening response, taste, smell, and touch response and use, fear or nervousness, verbal communication, nonverbal communication, activity level, level and consistency of intellectual response and, finally, general impressions. The scores assigned to each domain vary from 1 (within the limits of normality) to 4 (severe autistic symptoms). The total score varies from 15 to 60 and the cutoff point for autism is 30.¹⁰

The translation process described by Sperber¹⁵ was used for this study, since it is a useful and practical model, chosen by the majority of translation and transcultural adaptation studies.^16,17 The first step was to translate CARS from the original in English into the Portuguese spoken in Brazil. Two translators produced versions independently which were then compared by the researchers who compiled a composite version, which was itself then backtranslated into English by a bilingual psychiatrist who had not participated in the previous stages and was not in contact with the original text. The final version, named the CARS-BR, was administered to 60 patients in order to calculate its psychometric properties, and 50 patients were tested a second time after a minimum interval of 4 weeks in order to perform a test-retest analysis.

ATA

The ATA was developed by Ballabriga et al. and is made up of 23 easy-to-apply subscales which evaluate the behavioral profile of the child, based on a range of diagnostic features.¹⁸ The scale allows patient progress to be followed longitudinally and the psychometric characteristics of the Portuguese translation have been shown to be satisfactory.¹⁹ Additionally, the ATA is a questionnaire that can be used for screening aimed to differentiate autistic patients from those with mental disorders, but without autism. The cutoff point is 15. The objective of using it in this study was to make possible a convergent validity analysis of the CARS-BR.

GAF

GAF is a 100-point scale, the primary purpose of which is to provide a score that reflects a patient's global level of functioning. This scale can be used to plan treatment and measure its impact, to follow patient changes over time, to assess quality of life and to estimate prognosis. It can be used in any situation in which it is necessary to assess severity.²⁰ It was used in this study in order to obtain the discriminant validity of the CARS-BR questionnaire.

Statistical analysis

First, Cronbach's alpha was used to evaluate internal consistency. Pearson's correlation coefficient was then used to assess convergent validity and discriminant validity. Test-retest reliability was determined by calculating the 5% significance level. All data were analyzed using the statistical software SPSS 12.0.

Results

The sample studied contained a majority of male patients and mean age was 111.8 months (9.3 years). Approximately 55% of the patients were residents of the city of Porto Alegre, and 58.3% had access to a special school for children with pervasive development disorder.

The social and demographic characteristics of the group studied are shown in Table 1.

The Portuguese version of CARS is provided in Table 2.

Approximately 65% (39) of the patients assessed were classified as having severe autism, and 32% were put in the mild-to-moderate category. The remainder (3%) did not have autism according to the CARS-BR.

The association between childhood autism and epilepsy is well known and was present in 28.3% of the patients assessed here.

Psychometric properties

Internal consistency

The mean (± SD) total number of points was 39.4 (±5.07). Analysis of the internal consistency of the scale, using Cronbach's alpha coefficient, resulted in a figure of 0.82 (95%CI 0.71-0.88), indicating a high degree of internal consistency.

Convergent validity

The agreement observed between the CARS and the ATA was expressed as a Pearson's coefficient of r = 0.89 (95%CI 0.74-0.90); p < 0.001 (Figure 1).

Discriminant validity

As would be expected, there was a significant inverse correlation between the CARS and GAF scores: r = -0.75 (95%CI -0.84-0.61); p < 0.001 (Figure 2).

Test-retest reliability

After a minimum interval of 4 weeks, 50 of the patients were assessed once more. Analysis using Cohen's kappa coefficient demonstrated a concordance of 0.90. This result is an indication of the stability of scores over time, and it was not necessary to readminister the scale to the initial 60 patients.¹

Discussion

Since its description, more than 60 years ago, autism has represented a fascinating and enigmatic challenge for neurologists and psychiatrists.⁵ Nowadays, it is known that autism is not a single disease, but a complex developmental disorder associated with many different etiologies and varying degrees of severity, and which is characterized by abnormal behavior, language and cognition, with mental retardation in 70% of cases and epileptic crises in 30%.^6,7 There is no doubt about the importance of biological factors in the genesis of autism, however, since there is no marker, its diagnosis and knowledge of its limits continues to be a clinical decision^3,7 and, therefore, standardized tests to assess the disorder are of considerable interest to the scientific community.¹⁴

Our study undertook to translate the CARS into the Portuguese used in Brazil and to validate it in that country. This scale is used worldwide for the diagnosis and classification of autism and a measure of its importance is the fact that it has already been translated into Japanese, Swedish, French and other languages.^21-23 The resulting CARS-BR demonstrated good internal consistency, discriminant validity, convergent validity and test-retest reliability, when applied to a sample of children and adolescents with autism, treated at an outpatient clinic of a tertiary care hospital. These results are comparable with those of the original scale and of the other translated versions. Reliability is a measure of the reproducibility of a measurement and can be determined in a variety of ways: test-retest reliability, which is the degree of concordance between measurements taken at different times and which can be estimated using the kappa coefficient, interobserver reliability and internal consistency.^14,24 Interobserver reliability has earned the attention of several studies involving the CARS, but their results are difficult to evaluate and compare²² and it was not used in this study for this reason. Internal consistency represents the degree to which the scale, taken as a whole, measures an isolated phenomenon and is measured using Cronbach's alpha coefficient.^22,24 Measures of validity related to reliability findings and are the most important part of psychometric evaluation.^14,22 they can be defined as the capacity to truly measure that which the instrument claims to measure and include validity of criterion, content and construction (convergent, divergent and discriminant).²⁴

The psychometric characteristics of the Portuguese version of CARS are similar to the sample that gave rise to the scale.¹⁰ The literature, in general, supports the reliability of the CARS, with several different studies demonstrating internal consistency with acceptable values ≥ 0.90.^13,22,25 The internal consistency of the CARS-BR, measured using Cronbach's alpha coefficient is rated as good (0.83), in common with the original scale (0.94), and justifies combining the 15 individual items into a single score.^10,24 The Swedish version had an alpha of 0.91²² and Cronbach's alpha for the Japanese version was 0.87.^21,25

After each patient had been evaluated for each of the 15 items, their total score was calculated. Approximately 32% (19) of the patients studied here were classified as having mild-to-moderate autism, and 65% (39) of the patients met the criterion for severe autism. These results can be explained by the fact that the sample was selected from patients seen at a University Hospital offering specialized treatment.

The DSM-IV is a classification system developed by the American Psychiatric Association⁴ and which uses the three basic characteristics of autism, while the 15 items on the CARS allow a more objective diagnosis since they include the primary characteristics of autism described by Kanner, those observed by Creak & Rutter and additional scales (ICD-10 and DSM-IV).¹⁰ Agreement between the two methods can reach 98%,^9,11 and they therefore complement each other in diagnosis.

The majority of evaluation instruments developed for medical research originate from developed Western countries and are based on the concepts, formats, norms and expectations that are prevalent in these countries.^15,26 In the majority of cases, translation and transcultural adaptation of already-existing scales is chosen, since this is a more practical procedure than developing an original scale and also allows results from different countries to be compared.^15-17,27

The method used to translate instruments for different languages and cultures has been widely discussed, since, as many of these instruments are being used in socio-cultural settings very different from those in which they originated, the fundamental question is whether we can infer that the scores resulting from these evaluations have the same significance in ethnoculturally different populations.^26,27 Perneger et al.²⁸ investigated the characteristics of two different versions of quality-of-life instruments that had been translated using different methods and concluded that the version arrived at after an exhaustive process exhibited the same psychometric characteristics as that resulting from a more simple method, suggesting that a less refined technique does not compromise the quality of the final instrument. Translation and transcultural adaptation of any scale to be used in the area of healthcare requires linguistic care, and the importance of seeking equivalence between the original version and the version in Portuguese has been more and more recognized, particularly in a country with the dimensions of Brazil, in which regional differences and differences of educational level predominate.²⁷ Mattos, when presenting a Portuguese version of the MTA-SNAP-IV instrument for assessing the symptoms of attention deficit/hyperactivity disorder and symptoms of oppositional defiant disorder, emphasized the complexity involved in administering instruments in cultures different from those for which they were created. This author also indicates the use of methodology including translation, backtranslation, analysis of different versions and application to a target population.²⁷

The principal objective of this study was to determine the initial psychometric characteristics of the Brazilian Portuguese version of the CARS, but, in addition to these results, our data also demonstrated a strong association between autism and epilepsy (28.3%), which is consistent with previous specific studies that have reported epilepsy rates of 5 to 39% among children with autism.²⁹ Although all of these results are positive, additional studies are needed to supplement the data obtained here. Our findings, despite their agreement with published data, should be interpreted with caution, primarily as a result of the number of patients in the sample. This is an initial application of the instrument and regional cultural variables, in addition to sociocultural differences, must be studied in greater depth. The methodology employed, the precautions taken during the translation process and the psychometric assessment of the Portuguese version allow us to conclude that this is a valid and reliable instrument for assessing the severity of autism in Brazilian children. This study is the first step towards better diagnosis of autism in our country and, in the future, will make it possible to apply the CARS-BR in all five of Brazil's administrative regions and compare the results.

Acknowledgements

The authors would like to thank Dr. Andre Palmini for reading the article and providing constructive criticism.

References

• 1. Ajuriahuerra J. Manual de psiquiatria infantil. 4Ş ed. Barcelona: Toray-Masson; 1977.
• 2. Kanner L. Autistic disturbances of affective contact. Nerv Child.1943;2:217-50.
• 3. Gadia C, Tuchman R, Rotta NT. Autismo e doenças invasivas de desenvolvimento. J Pediatr (Rio J). 2004;80:S83-94.
• 4. American Psychiatric Association, editor. Diagnostic and statistical manual of mental disorders - DSM-IV. 4th ed. Washington, DC: American Psychiatric Association; 1994. p. 65-78.
• 5. Ozand PT, Al Odaib A, Merza H, Al Harbi S. Autism: a review. J Pediatr Neurol. 2003;1:55-67.
• 6. Dawson G, Webb S, Schellenberg GB, et al. Defining the broader phenotype of autism: genetic, brain, and behavioral perspectives. Dev Psychopathol. 2002;14:581-611.
• 7. Minshew NJ, Williams DL. The new neurobiology of autism: cortex, connectivity, and neuronal organization. Arch Neurol. 2007;64:945-50.
• 8. Rutter M. Incidence of autism spectrum disorders: changes over time and their meaning. Acta Paediatr. 2005;94:2-15.
• 9. Rellini E, Tortolani D, Trillo S, Carbone S, Montecchi F. Childhood Autism Rating Scale (CARS) and Autism Behavior Checklist (ABC) correspondence and conflicts with DSM-IV criteria in diagnosis of autism. J Autism Dev Disord. 2004;34:703-8.
• 10. Schopler E, Reichler R, Renner BR. The Childhood Autism Rating Scale (CARS). 10th ed. Los Angeles, CA: Western Psychological Services; 1988.
• 11. Eaves RC, Milner B. The criterion-related validity of the Childhood Autism Rating Scale and the Autism Behavior Checklist. J Abnorm Child Psychol. 1993;21:481-91.
• 12. Stella J, Mundy P, Tuchman R. Social and nonsocial factors in the Childhood Autism Rating Scale. J Autism Dev Disord. 1999;29:307-17.
• 13. Magyar CI, Pandolfi V. Factor structure evaluation of the childhood autism rating scale. J Autism Dev Disord. 2007;37:1787-94.
• 14. Matson JL, Nebel-Schwalm M, Matson ML. A review of methodological issues in the differential diagnosis of autism spectrum disorders in children. Res Autism Spectr Discord. 2006;1:38-54
• 15. Sperber AD. Translation and validation of study instruments for cross-cultural research. Gastroenterology. 2004;126:S124-8.
• 16. Alievi PT, Carvalho PR, Trotta EA, Mombelli Filho R. The impact of admission to a pediatric intensive care unit assessed by means of global and cognitive performance scales. J Pediatr (Rio J). 2007;83:505-11.
• 17. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol.1993;46:1417-32.
• 18. Ballabriga MC, Escudé RM, Llaberia ED. Escala d'avaluació dels trests autistes (ATA): validez y fiabilidad de una escala para el examen de las conductas autistas. Rev Psiquiatr Infanto-Juvenil. 1994;4:254-63.
• 19. Assumpção Jr FB, Kuczynski E, Gabriel MR, Rocca CC. Escala de avaliação de traços autísticos (ATA): validade e confiabilidade de uma escala para a detecção de condutas autísticas. Arq Neuro-psiquiatr.1999;57:23-9.
• 20. Spitzer RL, Gibbon M, Endicott J. Global Assessment Scale (GAS), Global Assessment of Functioning (GAF) Scale, Social and Occupational Functioning Assessment Scale (SOFAS). In: Rush AJ, editor. Handbook of psychiatric measures. Washington: American Psychiatric Association; 2000. p. 96-100.
• 21. Kurita H, Miyake Y, Katsuno K. Reliability and validity of the Childhood Autism Rating Scale-Tokyo Version (CARS-TV). J Autism Dev Disord.1989;19:389-96.
• 22. Nordin V, Gillberg C, Nyden A. The Swedish version of the Childhood Autism Rating Scale in a clinical setting. J Autism Dev Disord. 1998;28:69-75.
• 23. Pry R, Aussilloux C. Le Childhood Autism Rating Scale (CARS) chez l´enfant autiste jeune: analyse des items, étude des traits latents, validité concourante et généralisabilité. Psychologie et psychometrie. 2000;21:33-47.
• 24. Blacker D, Endicott J. Psychometric properties: concepts of reliability and validity. In: Rush AJ, editor. Handbook of psychiatric measures. Washington, DC: American Psychiatric Association; 2000. p. 7-14.
• 25. Tachimori H, Osada H, Kurita H. Childhood autism rating scale-Tokyo version for screening pervasive developmental disorders. Psychiatry Clin Neurosci. 2003;57:113-29.
• 26. Jorge MR. Adaptação transcultural de instrumentos de pesquisa em saúde mental. Rev Psiq Clin.1998;25:233-9.
• 27. Mattos P, Serra-Pinheiro MA, Rohde LA, Pinto D. A Brazilian version of the MTA-SNAP-IV for evaluation of symptoms of attention deficit/ hyperactivity disorder and oppositional-defiant disorder. Rev Psiquiatr Rio Gd Sul. 2006;28:290-7.
• 28. Perneger TV, Leplege A, Etter JF. Cross-cultural adaptation of a psychometric instrument: two methods compared. J Clin Epidemiol. 1999;52:1037-46.
• 29. Tuchman R, Rapin I. Epilepsy in autism. Lancet Neurol. 2002;1:352-8.

Correspondence

Publication Dates

• Publication in this collection
  13 Jan 2009
• Date of issue
  Dec 2008

History

• Received
  10 Mar 2008
• Accepted
  06 Aug 2008