Parent-teacher report reliability on the fourth edition of the Swanson, Nolan and Pelham scale in a Brazilian clinical sample of children and adolescents with attention-deficit/hyperactivity disorder

Background: Parents and teachers can be valuable sources of information for characterizing children’s ADHD-related impairments in different environments. However, evidence indicated that those categories of informants often provide conflicting responses in formal assessment scales, which may challenge diagnostic decisions regarding the condition. Objective: We aimed to investigate reliability rates between parents and teachers of children and adolescents with and without ADHD using SNAP IV. Methods: 199 children and adolescents aged 6 to 17 years were evaluated for ADHD symptoms using parent-rated and teacher-rated SNAP IV scales. Intraclass correlation coefficients were analyzed for ADHD domains (inattention and hyperactivity/impulsivity), as well as for defiant-oppositional behavior. Results: Reports from parents and teachers showed low reliability for all ADHD domains. Parents’ scores on the SNAP IV were higher than those of teachers. Parents and teachers provided highly discrepant responses concerning to the presence and severity of ADHD in children and adolescents, which might result from intrinsic aspects related to their daily functioning in different settings. Discussion: Clinicians should consider those trends in parental and teachers’ responses when interpreting results from informant-based instruments for detecting ADHD. Moraes PCB et al. / Arch Clin Psychiatry. 2020;47(2):35-9


Introduction
Attention-deficit/hyperactivity disorder (ADHD) is a life-long and potentially disabling neurodevelopmental disorder characterized by inattention and/or hyperactivity-impulsivity 1 . During the last decades, along with its establishment as a clinically relevant condition in the scientific and medical community, massive public interest for the disorder propelled large-scale pursuit for screening and parental guidance by millions of concerned individuals and their families 2 . Overall prevalence rates for ADHD have been estimated to be as high as 7% of children and adolescents and at least half of those individuals may present persistent symptoms into adulthood 3,4 . However, as awareness of ADHD increased, emerging allegations of overdiagnosis and overmedication, notably relative to school children, have been shifting public opinion towards skepticism for the importance of this clinical entity 5 . Controversies centered around ADHD assessment are abundant, especially regarding the validity of the diagnostic approaches 6,7 .
Recommended practices for detecting ADHD, according to panel of experts, include a systematic investigation of the subjects' history of symptoms, including age of onset, psychiatric comorbidities and evidence of impairments in at least two areas (school, work, home and interpersonal contacts) 8 . Gathering information from multiple respondents, such as family members, teachers or coworkers has been recommended to allow identification of functional impairments in different environments 1,9,10 . In special, considering that self-reports of children about their own behavior can be inaccurate and that adolescents tend to underestimate their symptoms 10,11 , collecting collateral impressions of the presence of ADHD clinical features in a variety of scenarios may be crucial for the assessment of this population. Among validated informant-based instruments for this purpose, the Swanson, Nolan and Pelham rating scale -4 th version (SNAP IV) is possibly the most widely used one in research settings 1,12,13 . It comprises twenty-six questions, corresponding to the list of ADHD symptoms, as depicted in the DSM-5 1 . Those include nine items aiming at detecting inattention, six assessing hyperactivity, and three measuring impulsive behavior.
Nonetheless, gaps in the knowledge regarding the best methods to assess ADHD are far from being filled. Several studies have suggested that reports from parents and teachers are often conflicting or weakly correlated, mainly regarding the presence and magnitude of clinical features. This shortcoming might challenge the interpretation of the results and raise serious doubts about the instruments' clinical utility, including for distinguishing the ADHD presentations (predominately inattentive type, predominately hyperactivity/impulsive presentation, and combined type) 14,15 . Detailed analyses of the responses suggested that teachers tended to be more benevolent in scoring symptom severity and that they might consistently report lower frequency of hyperactivity than parents 14,15 . Possible sources of divergences across informants may result from different cultural and environmental aspects, such as discrepancies between less structured home settings compared to typical schools and to varying amount of time spent with the children 15 . Moreover, it is known that raising a child with ADHD might demand more time and energy on caring that with non-ADHD children, which can lead to parental burnout 16 .
Several strategies have been proposed in the literature to overcome those inconsistencies and to clarify whether informantbased tools are valid approaches to assess the disorder. Among them, mathematically-derived indices obtained from comparing responses from different sources for the same instrument have been largely used 17 . Inter-rater agreement, for example, measures the matching rates of the absolute scores across evaluators for one specific test 17 . However, as previously mentioned, results from parents and teachers in standardized scales are generally highly discordant, which corresponds to low inter-rater agreement. Alternatively, inter-rater reliability could be employed to investigate consistency of classification patterns across different raters for one particular instrument, regardless of the absolute scores 17 . Noteworthy, interrater agreement and reliability are often uncorrelated, which means that high scores for one measure do not necessarily predict changes in the other index 17 .
In the present study, we aimed to investigate the reliability rate of responses of parents and teachers of children and adolescents aged 6 to 17 years in the SNAP IV. We hypothesized that despite low agreement, reliability might allow further information on the validity of the instrument as an assessment tool.

Participants
Participants were recruited at the Centro de Neuropsicologia Aplicada, D'Or Institute of Research and Education (IDOR) in Rio de Janeiro, Brazil. Most subjects were volunteers referred to the service by health professionals (physicians, psychologists and speech therapists), whereas others sought our institute due to recommendation from school personnel (9.5%) or upon spontaneous demand (20.3%).
Participants were included if the following criteria were fulfilled: (i) age between 6 to 18 years, corresponding to school-aged children and adolescents; (ii) regularly attending school, as informed by parents and teachers; (iii) living with one parent or both and (iv) all procedures were adequately completed. Exclusionary criteria were: (i) epilepsy, (ii) any psychotic disorder, (iii) current illicit drug use as reported by parents or self reported.

Procedures
Patients and parents underwent an initial medical interview, followed by neuropsychological tests including WISC-IV 18 and all subtests (intelligence, attention, working memory, vocabulary, reasoning, abstraction, visual perception, constructional abilities, dexterity) and CPT -Continuous Performance Test (attention). The Swanson, Nolan, and Pelham, version IV (SNAP-IV) scale was completed by one of the parents and one school teacher 12 . This instrument contains 26 items which evaluate ADHD domains and comorbid oppositional-defiant behavior (ODD), distributed as follows: Inattention (9 items), Hyperactivity/impulsivity (9 items) and Opposition (8 items). Existence of more than one respondent for the scale (for example, when the two parents or more than one teacher completed it separately) was resolved by randomly withdrawing one of the repeated versions. All individuals from the sample had collateral reports from both parents and teachers.

Diagnoses
Diagnoses of ADHD and other comorbid or concurrent mental disorders were based on results from the whole assessment protocol and were made according to the DSM-5 criteria by trained psychiatrists 1 . Upon this gold standard classification, sample was divided into groups with and without ADHD.

Ethics
All parents provided a written informed consent prior to enrollment in this study. The research has been approved by the Ethics Committee of Instituto D'Or de Pesquisa e Ensino (IDOR), Rio de Janeiro, RJ, Brazil.

Data analyses
For the present study, reliability was defined as a measure of consistency among evaluators in the ordering or relative position of performance evaluations irrespective of the absolute value of each evaluator's rating 19 . In other words, it reflects the relative pattern similarity between two or more sets of ratings 19 . To evaluate the reliability between parents' and teachers' reports on the symptoms of inattention, hyperactivity/impulsivity and challenging/oppositional behavior, intraclass correlation coefficients (ICC) were calculated through oneway randomized analysis and selected measures mean of intraclass correlation. Values lower than 0.5, between 0.5 and 0.75, between 0.75 and 0.9 and greater than 0.90 were considered as representing poor, moderate, good and excellent reliabilities, respectively 20 . The Cronbach's α was used to investigate the internal consistency of the ADHD domains in the scale across responders 21 . To examine which of the respondents observed higher and lower levels of symptoms, mean scores given by categories of observers were compared, using t-test analyzes of repeated samples. Bootstrapping (1,000 re-sampling, with 99% confidence interval) was implemented, aiming at correcting for possible deviations from the normal distribution. Statistical analysis was performed using SPSS for Windows version 19.

Differences between parents and teachers' responses
Parents' reports indicated higher mean rates for symptoms of inattention and hyperactivity than teachers in both ADHD (p < 0.001) and non-ADHD samples (p < 0.001). On the other hand, no group difference for the severity of ODD symptoms was detected across informants. Table 1 depicts those results.

Assessment of internal consistency and inter-rater reliability
For most SNAP IV items, a good internal consistency was observed for both ADHD and non-ADHD groups. On the other hand, a Cronbach's α of 0.71 for parents' reports of Inattention in ADHD group indicated a lower, but acceptable, internal consistency. Moreover, for all SNAP IV subscales, inter-rater reliability was fair, as indicated by low ICC values. Those results are summarized on Table 2.

Discussion
In line with previous reports 22,23 , the current study showed that parent-rated and teacher-rated SNAP IV adequately measured ADHD domains, as well as ODD symptoms, as indicated by a satisfactory internal consistency for all the subscales. In contrast, inter-rater reliability was low for all the instrument's subscales, which demonstrated that perception of symptoms was largely divergent across parents and teachers. Precisely, parents rated symptoms of inattention and hyperactivity as significantly more frequent and severe compared to teachers in the instrument, whereas such discrepancy was not verified for the ODD items.
Evidence of lower perception of children's cognitive, emotional and behavioral problems among teachers in comparison to parents had been previously described in the literature 24,25 . Those heterogeneities could be associated with intrinsic differences in their roles with respect to the children. It would be plausible to admit that parents and teachers participate in the children's lives and routines in a complementary, but far from coincident way. Students spend predetermined limited time at schools and teachers must follow structured educational programs, which contrasts with a less formally regulated home environment 15 . Hence, it could be predicted that divergent expectations regarding the children's behavior and particular strategies to deal with potential problems in each setting would be identified 26 . Convergingly, studies have suggested that rating agreement was greater when informants had equivalent relationships with the child. For instance, scores were more correlated among parents than across parents and teachers, suggesting that the nature of the liaison and the setting may influence the awareness of the symptoms 27 .
Other factors that might contribute for the disparity between responses ought to be discussed. Pervasive child difficulties and behavioral problems related to ADHD may result in considerable parental distress. Reports of feelings of irritation, isolation and exhaustion are common among those individuals, as well as experiencing marital conflicts, career setbacks and neglecting other children due to recurrent demands from the affected child 28 . In contrast, chronic occupational stress caused by students' misbehavior or low motivation tend to elicit a cynical attitude towards particular students or students in general on teachers 29 . Those results imply that dealing with ADHD children might evoke paradoxical effects on parents and teachers: whereas the former may respond with frustration and a trend for excessive complaining, the other group may react with less distress. Finally, parents often acknowledge that children are more attentive to activities related to leisure than to school tasks, which usually endorses their impressions of students' irresponsible attitude towards school and raises doubts about the ADHD diagnosis. Diversely, teachers' observations are restricted to a more structured classroom situation, and responses to ADHD scales were probably made simply through comparisons across peers of the same age and gender 30 .  The present study has its strengths, as it analyzed the reliability of SNAP IV across parents and teachers of ADHD subjects, for which the literature only provided agreement studies. However, some limitations should also be highlighted. Firstly, parental information may not be homogeneous between mothers and fathers. Mothers consistently report their children as having more problems of inattention and hyperactivity than fathers, therefore, the choice of the parental informant may impact considerably on the estimates of ADHD symptoms 31 . In addition, our sample has been drawn from one clinic, so data should be interpreted carefully when extrapolating to other situations. Finally, the presence of mood and stressrelated disorders among collateral responders were not analyzed. Considering that informant-based scales could be highly influenced by the participant's mental health, validity of our findings could be undercut by this limitation.

Conclusions
Diagnostic guidelines for ADHD in children, such as the DSM-5, strongly recommended that mental health practitioners collected reports from different informants, for the investigation of potential functional impairments at home and at school. For this purpose, formal validated instruments have been widely employed in both clinical and research practices. Our findings indicated that, for the SNAP IV, not only agreement, as previously demonstrated in the literature, but also reliability may be low regarding parents and teachers' impressions about the presence and severity of ADHD among children. A trend for overly perceiving the children's symptoms among parents and a much less emphatic report from teachers might be expected. With those results, we suggest that evaluators consider those traits of parental and teachers' responses when interpreting results from informant-based instruments for detecting ADHD.

Limitations
Our findings should be interpreted in light of some limitations. Our sample was collected from a private outpatient clinic where clientele belongs to middle, upper-middle and upper classes. For this reason, findings cannot be generalized to other settings. On the other side, our sample mirrors what is seen in most private settings in the country. In addition, referred clinical samples may present different clinical profiles than non-clinical epidemiological ones.

Financial support
None.