INTRODUCTION

Pervasive developmental disorder (PDD) symptoms appear in the first years of life and are characterized by alterations in the development of social relationships, communication and language. Inappropriate behavior with limited and stereotypical interests are also common. According to the Diagnostic and Statistical Manual of Mental Disorders, 4 ^th Edition (DSM-IV), the PDD group has five different categories: autistic disorder, Rett’s disorder, childhood disintegrative disorder, Asperger syndrome and pervasive developmental disorder – not otherwise specified (PDD-NOS) ¹ . Associated symptoms such as hyperactivity ² , aggressiveness, epilepsy and insomnia are common and aggravate the clinical condition, which in some cases may require pharmacological management ³ .

Bearing in mind that the neurobiological origin of PDDs remains to be fully clarified, there are no specific diagnostic markers of this disorder. Therefore, specialist evaluation continues to be the cornerstone for diagnosing individuals in whom this disorder is suspected ^{4 , 5} . Diagnosis is essentially clinical and is based on criteria defined in manuals such as the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) ¹ . In children, diagnosis performed by a clinical specialist is considered the ideal standard ⁶ . Other instruments such as the Childhood Autism Rating Scale (CARS) and the Autism Diagnostic Observation Schedule (ADOS) are used to hasten diagnosis. The ADOS is widely used to confirm a PDD diagnosis in cases of patients whose symptomatology is not clearly specific ^{6 , 7} , while the CARS is used for screening and is particularly useful in community settings.

The overall score of the CARS ranges from 15 to 60, with scores <30 indicating that the individual is not autistic, scores of 30–36.5 indicating mild to moderate autism, and scores of 37–60 indicating more severe degrees of autism ^{8 , 9} . Some studies, however, have suggested lowering the CARS cut-off point in order to include diagnosis of the other PDDs. The Japanese version of the CARS (CARS-Tokyo Version) suggests cut-off points of 25.5 or 26 to distinguish individuals with PDD from mentally retarded individuals, with a sensitivity of 0.86, a specificity of 0.83, a positive predictive value of 0.97 and a negative predictive value of 0.5 ¹⁰ . A study conducted in Caucasian children suggested a cut-off point of 25.5 to distinguish four-year-old children with PDD from those without PDD, with a sensitivity of 0.82, a specificity of 0.95, a positive predictive value of 0.97 and a negative predictive value of 0.74 ⁹ . These studies raised the possibility that cultural factors may influence the validity of the CARS cut-off point. This may be important for multi-ethnic societies in which services have a large clientele of immigrant families from diverse cultural backgrounds. The objective of the present study is to describe how the CARS performs in relation to the ADOS and to clinical diagnosis reached in accordance with the criteria defined by the DSM-IV in a population of children of immigrants with a suspected diagnosis of some form of PDD. A secondary objective is to propose, if appropriate, a CARS cut-off point that would provide a diagnosis of PDD, including PDD-NOS and Asperger syndrome, for an immigrant population. The importance of this study is to increase the use of a simple, inexpensive screening tool such as the CARS, allowing it to be used to screen for milder clinical conditions within the autism spectrum. With the growth in migration to large urban centers, it is important to evaluate the appropriateness of these instruments for use in first-generation immigrant populations.

METHOD

INSTRUMENTS

Sociodemographic data and information regarding immigration were collected from the children’s parents. Each child was evaluated using three instruments: the CARS, the ADOS and DSM-IV criteria, using the same psychiatrist. The CARS is essentially an observational scale. It is short and can be applied to children of two years of age or older ⁸ . A trained observer evaluates the child’s behavior in the 15 domains that are affected in autism: relating to people; imitation; emotional response; body use; object use; adaptation to change; visual response; listening response; taste/smell/touch response; use, fear or nervousness; verbal communication; nonverbal communication; activity level; level and consistency of intellectual response; and general impressions ⁸ . The examiner then awards scores that range from 1 to 4 for each domain, 1 being indicative of age-appropriate behavior and 4 of a major deviation from normal. The CARS is applied within the context of a common clinical evaluation, with no pre-established activities to evaluate the child’s response.

The ADOS is an instrument designed to evaluate the child’s behavior with respect to certain structured activities that have been proposed ¹² . It consists of four modules, all of which have been found to show good reliability, sensitivity and specificity for both autism and ASD, both in English and Spanish ¹³ . The ADOS has also been used in other countries such as Greece ¹⁴ and Germany ¹⁵ , and its psychometric properties were maintained in the different settings. Each one of the four modules in the ADOS is aimed at individuals with specific language development levels. In each of the modules, there are protocols of structured activities and interactive dynamics that are conducted by a trained examiner over a period of approximately 45 minutes. The objective of the different tasks is to provide the observer with information on the social behavior, communication and general actions of the individual being evaluated. Based on observations on how the individual completes the suggested tasks, the professional answers a questionnaire, awarding scores for the different behaviors. The questions are organized on a 4-point scale, in which 0 indicates no abnormality and 3 indicates moderate to severe abnormalities ¹² . To be classified as having autism or an autism spectrum disorder according to the ADOS, the sum of the scores defined in the algorithms of the four modules has to reach the minimum cut-off points in the domains referring to communication and reciprocal social interaction ^{16 , 17} .

The children also underwent a clinical assessment after which they were given a multi-axial diagnosis using DSM-IV Text Revision (DSM-IV-TR) criteria ¹ . The severity of their impairment was assessed in accordance with the Global Assessment of Functioning (GAF) scale, a numerical scale ranging from 0 to 100 that is used by mental health clinicians and physicians in general to subjectively rate the social, occupational and psychological functioning of children and adolescents ¹⁸ . The highest scores are 91–100, reflecting “superior functioning in a wide range of activities”, while the lowest scores are 1–10, indicating “persistent danger of severely hurting self or others or persistent inability to maintain minimal personal hygiene or serious suicidal act with clear expectation of death”.

DATA ANALYSIS

The database was constructed using the SPSS statistical software package, version 14.0 for Windows. In the sample studied, measures of simple frequency and percentages were obtained for the qualitative variables, and measures of central tendency and dispersion for the quantitative variables.

RESULTS

The study sample consisted of 49 patients: 34 boys (69.4%) and 15 girls (30.6%). The mean age of the overall sample was 5.75±3.46 years (mean±standard deviation), range 2–15 years. With respect to the origin of the patients included in the study, 61.2% were Canadian; however, they were children of first-generation immigrants. A further 32.7% were born in their parents’ country of origin. The parents came from 33 different countries, which were identified and, for the purposes of analysis, grouped into: Asia, the Arab world, Latin America and the Caribbean, North America and Europe. The children’s mothers came from Asia (38.8%), the Arab world (20.4%), Latin America and the Caribbean (16.3%), North America (12.2%) and Europe (12.2%). Percentages for the fathers’ place of birth showed a similar geographical distribution, with a predominance of individuals from Asia (42.9%), followed by the Arab world (22.4%), Latin America and the Caribbean (14.3%), North America (8.2%) and Europe (12.2%).

The fact that the parents were immigrants affected the evaluation process in two ways. Firstly, the parents found it comforting that the psychiatrist who interviewed them was also a foreigner; they were able to identify with him or her. At various moments during the evaluation, they mentioned feeling at ease and stated that they identified with the interviewer’s strong accent. Secondly, the word “problem” or “difficulty”, when mentioned to families of Asian origin, particularly those from Bangladesh, India and Pakistan, was emphatically rejected. The parents responded that their child had no “problem” or “difficulty”. Substituting these words for more indirect questions such as “how is your child doing at school?” and “how does he/she get on with his/her siblings?” resulted in more comprehensive answers on the child’s actual limitations. In relation to the application of the ADOS, the only task that was found to be inappropriate for some of the children was imitating a birthday party, singing “Happy Birthday” and blowing out the candles on the cake, as this kind of celebration did not exist in the parents’ country of origin (the Asian countries mentioned above).

Comparison of the diagnoses obtained by the ADOS with those obtained using the DSM-IV-RT criteria showed almost total agreement between these two diagnostic methods. Table 1 shows the agreement between the CARS and the other diagnostic instruments. Using the DSM-IV-RT, children diagnosed as having an autism spectrum disorder, PDD-NOS or Asperger syndrome were considered to have a positive diagnosis of PDD; patients diagnosed with a language disorder or attention deficit hyperactivity disorder (ADHD) and those who were not classified as having any of the disorders described in the DSM-IV-RT were considered negative for PDD. Taking the DSM-IV-RT as the ideal standard, the sensitivity of the CARS with a cut-off point of 30 was 41%, with a negative predictive value of 16%, a specificity of 100% and a positive predictive value of 100%.

The children diagnosed with autism or an autism spectrum disorder in accordance with the ADOS were considered positive for PDD, whereas those diagnosed as non-autistic by the ADOS were considered negative for PDD. Taking the ADOS as the ideal standard, the sensitivity of the CARS was 39%, with a negative predictive value of 16%, a specificity of 100% and a positive predictive value of 100%.

The results obtained using the CARS were compared with those of the GAF. According to the scores obtained by applying the GAF, symptoms were dichotomized into “mild” or “severe” autism or ASD. The children with a GAF score ≤60 were rated as “mild”, whereas those with scores >60 were rated as “severe”. Therefore, by considering greater functional impairment (severe) as a positive diagnosis and less impairment (mild) as a negative diagnosis, the sensitivity of the CARS was 73%, with a specificity of 96%, a negative predictive value of 94% and a positive predictive value of 80%.

Table 2 shows that 95% of the 20 children diagnosed as PDD-NOS by the DSM-IV-RT were classified as non-autistic by the CARS, while 5% were classified as having mild/moderate autism. The child diagnosed with Asperger syndrome in accordance with the DSM-IV was classified as non-autistic by the CARS. All the children with language disorders, ADHD and no psychiatric diagnosis according to the DSM-IV had a negative diagnosis with the CARS. Of the 22 children diagnosed as having an autism spectrum disorder according to the DSM-IV, 27.3% were not considered autistic according to the CARS, while 45.5% had mild/moderate autism and 27.3% had severe autism. Table 3 shows that when symptoms were more severe (severe GAF), only 11% of the children considered to have an autism spectrum disorder according to the DSM-IV were considered non-autistic by the CARS, whereas for those with less severe symptoms (mild GAF), 80% of the children considered autistic by the DSM-IV were considered non-autistic by the CARS.

All the children who were considered to have an autism spectrum disorder or who tested negative for PDD in the ADOS were classified as non-autistic by the CARS ( Table 2 ). Of the 30 children considered autistic according to the ADOS, 46.7% were not considered autistic by the CARS, all of them having lower scores (between 14 and 21) in the ADOS. Of those who were considered autistic by the CARS, i.e. the remaining 53.3%, all had higher scores (between 19 and 31) in the ADOS.

Table 4 shows the sensitivity, specificity, and positive and negative predictive values for the different cut-off points in the CARS scale for distinguishing between children with and without a diagnosis of PDD. The cut-off point of 20/21 offered the best relationship between specificity (0.93), sensitivity (0.83), positive predictive value (0.97) and negative predictive value (0.63).

DISCUSSION

Despite the fact that the majority of the children and adolescents in the study were born in Canada, their parents were all first-generation immigrants who had come to Canada from various parts of the world. At the medical interview, any difficulties in obtaining the patient’s medical history for the purpose of reaching a diagnosis in accordance with the DSM-IV and applying the ADOS were not so marked as to prevent the data required for diagnosis from being collected. The task related to simulating a birthday party was not performed as part of the ADOS series of tests and did not affect how the scores were calculated, as other tasks evaluate similar aspects.

Taking the DSM-IV as the ideal standard, sensitivity and specificity were found to be good with the ADOS for the diagnosis of PDD. These results are in agreement with the pertinent literature for children of non-immigrants in which studies have reported sensitivity of 90–97% and specificity of 87–94% for each one of the four different applicable modules in the ADOS ¹⁶ . Therefore, the results of the present study suggest that the standard diagnostic instruments for autism spectrum disorders are well adapted for use with the children of immigrant parents, even for those who immigrated less than five years earlier.

Comparing the CARS with the ideal standard DSM-IV and with the ADOS, and using the standard cut-off point of 30 for the CARS, sensitivity was poor with this method for a diagnosis of any PDD and the negative predictive value was low, while specificity was high and the positive predictive value was also high. If the severity of symptoms (measured by GAF) is taken as the ideal standard, the sensitivity of the CARS was higher, as was the negative predictive value, thus indicating that the CARS is a useful instrument for diagnosing cases of PDD in which the degree of functional impairment is greater, or in which symptomatology is more severe. This hypothesis is corroborated by the descriptive findings, suggesting that the majority of children diagnosed with less severe PDDs, such as Asperger syndrome or PDD-NOS according to the DSM-IV, and with an autism spectrum disorder according to the ADOS, are considered non-autistic by the CARS. Furthermore, the majority of children considered as having an autism spectrum disorder by the DSM-IV, and who had more severe symptoms according to their GAF score and the ADOS, are considered autistic by the CARS. On the other hand, the majority of children considered as having an autism spectrum disorder according to the DSM-IV, with milder symptoms according to their GAF score and the ADOS, are considered non-autistic when evaluated by the CARS.

The cut-off point of 20/21 for the use of the CARS as a screening instrument for PDD in the study population was lower than the cut-off point used with children in Japan, where the CARS-Tokyo Version suggested 25.5/26 as the cut-off point for differentiating between PDD and non-PDD ¹⁰ . In Caucasian children, the cut-off point was reported as 25.5 for children of 2–4 years in age ⁹ . The population in the present study included older children (age range 2–15 years), which may have contributed to lowering the score because symptoms may decrease in older children with PDD-NOS as they acquire skills over time. Another aspect to be taken into consideration in relation to the cut-off point is that, according to the present study, the severity of symptoms may interfere in the CARS classification. Consequently, children with a diagnosis of PDD-NOS and more severe symptoms (severe GAF) were more likely to be classified as autistic by the CARS than those with milder symptoms (mild GAF). Therefore, it is possible that the population in the present study may have included more patients with less severe symptoms compared with earlier studies, thus lowering the cut-off point.

The results of this study contribute to validating the use of instruments for the diagnosis of autism, specifically the CARS, the ADOS and the DSM-IV, in populations of children of recent immigrants who do not speak the language of the host country fluently and whose cultural background differs considerably from that of the countries in which the instruments were conceived. As the CARS is a simple screening tool used worldwide, it is advantageous that the present findings have contributed towards defining a CARS cut-off point that includes PDD-NOS, even when symptoms are mild.

LIMITATIONS

Definition of the CARS cut-off point was based on two instruments used as the ideal standard – the DSM-IV and the ADOS – as complete agreement was found between these two diagnostic methods. The new criteria contained in the DSM-V will eliminate diagnoses of Asperger syndrome and PDD-NO, including both under the category of “a mild autism spectrum disorder” ¹⁹ . It is expected therefore, that the data from the present study will remain valid following publication of the DSM-V.

The scales were applied by a single psychiatrist who had trained for more than 10 years in the evaluation, diagnosis and treatment of children with PDD. As the CARS can be used in various settings and by professionals with various levels of training, the results obtained with it in the present study may not be comparable with the findings of professionals with less experience in diagnosing autism or in settings in which such a detailed evaluation is impossible. Clinical judgment is an important component of the CARS score and is reflected in the results. In this study, the clinical diagnosis was not made separately from the CARS score, as the clinician who calculated the CARS score was the same individual who evaluated the patient using the DSM-IV criteria. This may have increased the association between the CARS scores and the clinical diagnosis. As the association between the CARS and the DSM-IV was poor with respect to the usual CARS scores, this bias does not appear to have affected the results.