Cross-cultural adaptation and preliminary psychometric properties of the Affective Reactivity Index in Brazilian Youth: implications for DSM-5 measured irritability

Adaptação transcultural e propriedades psicométricas preliminares do Affective Reactivity Index em jovens brasileiros: implicações para a irritabilidade medida pelo DSM-5

ORIGINAL ARTICLE

Cross-cultural adaptation and preliminary psychometric properties of the Affective Reactivity Index in Brazilian Youth: implications for DSM-5 measured irritability

Adaptação transcultural e propriedades psicométricas preliminares do Affective Reactivity Index em jovens brasileiros: implicações para a irritabilidade medida pelo DSM-5

Diogo Araújo DeSousa^I; Argyris Stringaris^II; Ellen Leibenluft^III; Silvia Helena Koller^IV; Gisele Gus Manfro^V; Giovanni Abrahão Salum^V

^IPhD candidate, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil

^IIMD, PhD, Institute of Psychiatry, King's College London, London, United Kingdom

^IIIMD, Section on Bipolar Spectrum Disorders, Emotion and Development Branch, National Institute of Mental Health, Bethesda, MD, USA

^IVPhD. UFRGS, Porto Alegre, RS, Brazil

^VMD, PhD. UFRGS, Porto Alegre, RS, Brazil

^{Correspondence}

ABSTRACT

Objective: To describe the cross-cultural adaptation of the Affective Reactivity Index (ARI) to Brazilian Portuguese and to investigate preliminary psychometric properties of the adapted version.

Methods: Cross-cultural adaptation was based on the investigation of the theoretical and operational equivalences of the original ARI in the Brazilian context, followed by a process of translation, back-translation, and review by a committee of experts. Data analysis was carried out in a community sample of 133 schoolchildren aged 8 to 17 years to investigate the following characteristics of the ARI: 1) factor structure; 2) internal consistency; 3) construct validity comparing differential relationships between irritability and anxiety dimensions and impairment; and 4) item response theory (IRT) parameters.

Results: A final Brazilian Portuguese version of the instrument was defined and is presented. Internal consistency was good, and our analysis supported the original single-factor structure of the ARI. Correlations of the ARI with distress-related anxiety dimensions were higher than with phobic-related anxiety dimensions, supporting its construct validity. In addition, higher ARI scores were associated with higher irritability-related impairment. IRT analysis underscored frequency of loss of temper as essential to inform about pathological states of irritability.

Conclusion: The Brazilian Portuguese version of the ARI seems to be very similar to the original instrument in terms of conceptual, item, semantic, and operational equivalence. Our preliminary analysis replicates and extends previous evidence confirming promising psychometric properties for the ARI.

Keywords: Affective Reactivity Index, irritability, cross-cultural adaptation, psychometrics, item response theory.

RESUMO

Objetivo: Descrever a adaptação transcultural do Affective Reactivity Index (ARI) para o português do Brasil e investigar propriedades psicométricas preliminares da versão adaptada.

Método: A adaptação transcultural foi baseada na investigação das equivalências teórica e operacional da versão original do ARI no contexto brasileiro, seguida do processo de tradução, retrotradução e revisão por comitê de especialistas. A análise dos dados foi realizada em uma amostra comunitária de 133 escolares com idade entre 8 e 17 anos para investigar as seguintes características do ARI: 1) estrutura fatorial; 2) consistência interna; 3) validade do construto, comparando as relações diferenciais entre irritabilidade e as dimensões de ansiedade e prejuízo; e 4) parâmetros de teoria da resposta ao item (TRI).

Resultados: Uma versão final em português do Brasil do instrumento foi definida e é apresentada. A consistência interna foi boa, e nossa análise confirmou a estrutura unifatorial original do ARI. As correlações do ARI com as dimensões de ansiedade relacionadas a sofrimento foram maiores do que com as dimensões de ansiedade relacionadas a fobias, reforçando a validade do construto. Além disso, escores mais altos no ARI foram associados a maior prejuízo relacionado à irritabilidade. A análise do TRI enfatizou a frequência de perda de controle como essencial para determinar estados patológicos de irritabilidade.

Conclusão: A versão em português do Brasil do ARI parece ser muito semelhante ao instrumento original em termos de equivalência conceitual, de itens, semântica e operacional. Nossa análise preliminar reproduz e estende evidências anteriores que confirmam propriedades psicométricas promissoras para o ARI.

Descritores: Affective Reactivity Index, irritabilidade, adaptação transcultural, psicometria, teoria da resposta ao item.

Introduction

The adequate investigation of irritability is important to the field of mental health because irritability is a characteristic of multiple psychiatric diagnoses¹ and the core feature of a new diagnostic category in the Diagnostic and Statistical Manual of Mental Disorders, 5th edition (DSM-5), namely, disruptive mood dysregulation disorder (DMDD).² In addition, one important difference between the DSM-IV and the DSM-5 is that the latter recognizes the importance of dimensional assessments of psychopathology to complement traditional binary diagnoses.² A dimensional assessment of psychopathological symptoms can focus on several dimensional features, e.g., number, duration, and intensity of symptoms, and impairment related to them. It can therefore provide information about the disorder severity and about changes in symptoms over time through repeated measurements. Hence, valid and reliable instruments to dimensionally assess the irritability latent trait are needed.

The Affective Reactivity Index (ARI) is a concise instrument developed to measure irritability in childhood and adolescence.³ The ARI investigates three aspects of irritability: a) threshold for an angry reaction; b) frequency of angry feelings/behaviors; c) duration of such feelings/behaviors. The conceptualization of the ARI defines irritability as a mood of easy annoyance involving anger and temper outbursts.⁴ Importantly, the ARI was developed to measure specifically irritable mood, rather than related constructs such as aggressive behavior, because irritable mood can be present in the absence of aggressive behavior. In Brazil, a few recognized instruments measure aggressive behaviors in childhood and adolescents.⁵ However, to our knowledge, no instruments with documented validity and reliability specifically assess irritability.

The ARI has been shown to have adequate psychometric properties in American and British samples,³ encouraging further validation in different cultures. This study describes the process of cross-cultural adaptation of the ARI to the Brazil setting and reports preliminary psychometric properties of the Brazilian Portuguese version of the instrument, including factor structure, internal consistency, and construct validity. We also extend previous research3 by investigating item response theory (IRT) parameters.

Method

Cross-cultural adaptation process

The adaptation process can be divided into three major steps. First, the instrument was analyzed in terms of conceptual, item, and operational equivalence between the original and target contexts. Equivalence was assessed by two Brazilian experts in the field. The objectives were to investigate if: 1) the relationship between the ARI and its underlying concept (i.e., irritability) in the original setting would be the same in Brazil (conceptual equivalence); 2) the items comprising the original ARI would remain relevant in the Brazilian context (item equivalence); and 3) the instructions, method of administration, questionnaire format, and measurement methods used in the original ARI would be suitable to the Brazilian context (operational equivalence).

Second, a process of translation and back-translation was performed. The ARI was translated from English into Brazilian Portuguese by two independent translators, and a third one synthesized both translations into a single version in Brazilian Portuguese. Then, this synthesized translation was back-translated independently by two other translators, and a third one synthesized both back-translations into a single version in English.⁶ All translators involved in this process were fluent in both languages.

Third, results from the equivalence investigation and from the translation and back-translation process were reviewed by a committee of four experts, including one of the authors of the original instrument. The committee assessed if the translation was adequate and if the translated items were both semantically equivalent to the original ARI and relevant to the Brazilian context.⁶ Adjustments of the items in the Brazilian Portuguese version were performed after a consensus was reached among the members of the committee.

Participants and procedures

Participants were 133 schoolchildren aged 8 to 17 years old (mean ± standard deviation [SD] = 11.01±1.61), 54.1% girls. Students came from one private (n = 103) and one public (n = 30) Brazilian school. According to school records, youths in the private school came from middle socioeconomic-level families, and youths in the public school came from low socioeconomic-level families. School approval and child and adolescent assent were obtained before participation, as well as parental written informed consent. All youths were asked to complete the self-report instruments during their classroom period. Research assistants explained the research objectives and instructions before each data collection session. This study is part of a larger project whose protocol was approved by the Ethics Committee of the Institute of Psychology of Universidade Federal do Rio Grande do Sul (protocol no. 22264).

Instruments

The Affective Reactivity Index (ARI)³ is a self-report measure of irritability symptoms in childhood and adolescence. The instrument comprises six items assessing feelings and behaviors related to irritability and one follow-up item assessing impairment due to irritability (Table 1). Respondents rate each item using a 3-point scale (0 = not true; 1 = somewhat true; 2 = certainly true). Total ARI scores refer to the first six items and range from 0 to 12, with higher scores reflecting higher levels of irritability.

The Spence Children's Anxiety Scale (SCAS) is a self-report measure of anxiety symptoms in childhood and adolescence.⁷ The instrument comprises 38 items assessing anxiety symptoms related to six different dimensions/subscales: generalized anxiety (GAD), social anxiety (SoAD), separation anxiety (SeAD), panic/agoraphobia (PD), obsessive-compulsive problems (OCD), and fears of physical injury (FEARS). Respondents rate each item using a 4-point scale (never = 0; sometimes = 1; often = 2; always = 3). Total SCAS scores range from 0 to 114, with higher scores reflecting higher levels of anxiety. The SCAS has been cross-culturally adapted to Brazil⁸ and was used to identify anxiety symptoms in the sample and assess construct validity.

Data analysis

Confirmatory factor analysis (CFA) was used to evaluate whether the single-factor structure of the original ARI fit to the Brazilian context. To take into account the categorical nature of the items in the scale, the weighted least square mean variance (WLSMV) estimation method was employed in the Mplus software. For fit indices, the following indices were calculated: chi-square, comparative fit index (CFI), Tucker-Lewis index (TLI), root mean square error of approximation with 90% confidence interval (RMSEA-90%CI), and weighted root mean square residual (WRMR). A non-significant chi-square result (p > 0.05) represented a good fit. Similarly, CFI and TLI values above 0.90 or close to 0.95 represented a good fit. RMSEA values close to or below 0.05 represented a good fit, and those below 0.08 represented an acceptable fit. WRMR values below 0.60 represented a good fit.⁹

Descriptive statistics of frequencies, means, and SDs were calculated for the ARI items and total score. Age group (children and adolescents), gender (boys and girls), and school type (private and public) differences were examined using analysis of variance (ANOVA). We also used ANOVA to investigate differences in ARI scores among three categories of the ARI impairment item. To investigate item-by-item associations with the categories of impairment, logistic regression analyses were conducted using each ARI item as the independent variable and the seventh item (impairment) as the outcome variable, with two different thresholds (at least somewhat and certainly impaired). After that, we also conducted stepwise logistic regression analyses followed by all-possible-subsets (APS) logistic regression analyses to explore, considering the ARI items altogether, the set of items that best predicted impairment. APS analyses help in selecting the best subset from a larger set of highly intercorrelated predictors. In such situations, different subsets may present almost equivalent associations with the outcome variable, and a conventional stepwise regression analysis may select a suboptimal subset due to minor differences in bivariate associations. To overcome this problem, APS analyses generate results for a large number of different models with a fixed number of predictors, determined by the previous logistic regression analyses. We analyzed the five best models generated by the APS to investigate which ARI items best predicted the impairment outcome variable.

Pearson correlations were calculated between ARI scores and SCAS subscale scores to investigate construct validity. It was hypothesized that the correlation between the ARI score and distress-anxiety (measured by SCAS-GAD subscale) would be stronger than the correlations between the ARI score and phobic-anxiety (measured by the SCAS-SoAD, -SeAD, and -FEARS subscales), PD or OCD symptoms, given previous evidence linking childhood irritability to a later development of distress disorders such as GAD, dysthymia, and depression.¹⁰ Furthermore, Cronbach's alpha coefficient was calculated to evaluate the internal consistency of the ARI score.

An item response theory (IRT) analysis was conducted using the graded response model with different item discrimination.¹¹ The latent trait of irritability was represented by θ, centered on 0 with a SD of 1. The maximum marginal likelihood estimation implemented in the ltm package of the R software was used to estimate item parameters of discrimination (the slope, α) and severity (difficulty; category thresholds, β1 and β2).¹² The discrimination parameter (α) represents the ability of each item to discriminate people at different levels of θ (in this case, different severity levels of irritability). A higher α therefore indicates that the item performs better discriminating subjects at different severity levels. The severity parameters (β1 and β2) represent the θ level at which there is 50% probability of endorsing a given category or higher (in this case, β1 = endorsing somewhat true OR certainly true, and β2 = endorsing certainly true). For instance, an item with a β2 of +1 indicates that a subject at 1 SD above the mean θ has 50% of probability of answering certainly true to this item. The mean of both thresholds was computed to provide an estimate of the item difficulty, i.e., the item location in the severity continuum represented by θ.

Results

Cross-cultural adaptation

Both experts agreed that the theoretical rationale that served as the basis for the original ARI was relevant and adequate to the Brazilian context (conceptual equivalence). The items representing the irritability construct were also deemed relevant and important to the Brazilian context (item equivalence). Finally, the self-report format, instructions, and method of assessment and measurement were also considered appropriate to the objectives of the instrument (operational equivalence).

Concerning the review by the committee of experts, the synthesized translation was approved with only one major concern: the translation of the word "annoyed" (item 1, "Easily annoyed by others") into "irritado" by translator 1 and "perturbado" by both translator 2 and the third, synthesizer translator. The concern was based on the fact that "perturbado" was back-translated as "disturbed" rather than "annoyed" by all back-translators. The committee then agreed to replace the translation of the word "annoyed" to "incomodado." The committee agreed that, in its last version, the translated Brazilian Portuguese items of the ARI reflected the same concepts related to the irritability construct as the original instrument, with semantic equivalence (see Table 1 for the complete set of translations and back-translations).

Preliminary psychometric properties

Table 2 shows the preliminary psychometric properties of the Brazilian Portuguese version of the ARI. CFA results showed that the single-factor model fit the Brazilian sample very well, with all indices supporting good fit of the model. Furthermore, all items presented loadings above 0.45 (mean ± SD = 0.600±0.099).

Considering item response option frequencies, lose temper easily was most frequently endorsed as certainly true, whereas being angry most of the time was the least likely to be endorsed as somewhat/certainly true (Table 2). Mean item scores ranged from 0.43±0.71 (item 4 mean score) to 0.78±0.81 (item 6 mean score). Figure 1 shows a descriptive comparison between item mean scores obtained in our sample and in the original study of the ARI.³

ANOVA results showed that the ARI score did not differ between boys and girls (F(1, 129) = 2.47, p = 0.118) or between children and adolescents (F(1, 129) = 1.29, p = 0.258). Conversely, participants from the public school presented higher scores than those from the private school (F(1,129) = 5.22, p = 0.024). None of the interactions between these variables differentiated groups (all p-values > 0.10). Table 2 shows the means and SDs of the ARI scores obtained for the total sample and for each demographic group. The ARI score differed significantly among the categories of the ARI impairment item (F(2, 126) = 25.84; p < 0.001). Post hoc analyses revealed that increased impairment was significantly associated with increased irritability symptoms among all combinations of the ARI impairment categories (Figure 2).

Table 2 also shows the results of item-by-item logistic regression analyses. The stepwise logistic regression model revealed that a set of two items captured all the variance associated with impairment in both thresholds: at least somewhat impaired (Nagelkerke R² = 0.314), and certainly impaired (Nagelkerke R² = 0.394). Therefore, APS logistic regression analyses investigated all sets of two-by-two item combinations. When analyzing the frequency of each item included in the five best sets of predictors according to R² in the APS analyses, item 3 (stay angry for a long time) was present in all of them in both thresholds, suggesting that this is the item most strongly associated with impairment (Table 2).

Pearson correlations showed that all SCAS subscale scores correlated with the ARI score (Table 2). Correlations presented small to moderate magnitudes, with Z tests showing that the correlation between the ARI score and the SCAS-GAD subscale score was stronger than correlations between the ARI score and the SCAS-SoAD (p = 0.049), SeAD (p = 0.011), OCD (p = 0.038), and FEARS (p = 0.012) subscale scores. The correlation between ARI score and SCAS-PD subscale score did not differ from any other correlations assessed. Figure 3 graphically represents the correlation magnitudes. The Cronbach's alpha coefficient of the ARI score was good (α = 0.843).

IRT analysis demonstrated that ARI items concentrate their potential to differentiate subjects between -2 and +2 in the latent trait (91.25% of the test information), provide some information for those between +2 and +4 (7.82% of the test information), and have very low ability to differentiate those between -4 and -2 (0.73% of the test information). With respect to the discrimination parameter, item 2 (often lose temper) presented the highest α (Table 2), indicating that it is best able to discriminate people with different severity levels of the irritability latent trait. An analysis of the severity parameters revealed that items 2 and 6 presented the lowest severity estimates, whereas item 4 (angry most of the time) presented the highest severity estimate (Table 2). This indicates that subjects who endorse higher categories of item 4 have relatively higher levels of the irritability latent trait than those who endorse higher categories of items 2 and 6. Figures 4 to 6 provide graphic information on IRT analysis.

Discussion

The present study described the cross-cultural adaptation of the ARI to Brazilian Portuguese and investigated preliminary psychometric properties of the adapted version. A final Brazilian Portuguese version of the instrument was defined and is presented. Our analysis supports the original single-factor structure and finds good internal consistency and adequate construct validity. The IRT analysis underscored the importance of frequency of loss of temper in identifying pathological states of irritability.

When a foreign instrument is to be used in a new cultural setting, a careful and methodologically adequate process of cross-cultural adaptation is mandatory.⁶ Regarding the psychometric properties investigated, the single-factor structure fit the Brazilian sample very well, supporting the original theoretical conceptualization of the ARI.³ All items presented good factor loadings, and the ARI score demonstrated good internal consistency, in line with previous research.³ Also in line with previous research,³ no significant differences were found between gender and age groups in mean ARI scores, while a significant association was found between higher ARI scores and greater impairment due to irritability symptoms. The pattern of correlations between the ARI score and SCAS subscale scores was in the hypothesized direction: the correlation between the ARI and the SCAS-GAD was the strongest, consistent with previous evidence linking irritability symptoms to distress disorders such as GAD, depression, and dysthymia.¹⁰

It is interesting to point out that the mean ARI score obtained in our sample was relatively high when compared to ARI scores described for other settings.³ In fact, our Brazilian community sample scored as high as the U.S. sample of children meeting criteria for bipolar disorder in the work of Stringaris et al.³ Even though these two samples cannot be compared directly due to methodological differences between studies, the fact that Brazilian children presented relatively high ARI scores is consistent with previous research indicating that, among youths from 42 societies, Brazilian youths had the highest mean total Child Behavior Checklist problem scores.¹³ However, it can also be hypothesized that the Brazilian cultural context allows Brazilian children to express themselves more easily, not having to focus on self-control when dealing with some circumstances in public and private situations, which may have led to a looser reactivity reflected on the high scores found. Anyhow, this finding indicates the importance of properly evaluating psychopathological symptoms in Brazilian youth. It also stresses the fact that cross-cultural research on psychopathology should be based on the development of assessment instruments that are validated and applied across and within cultures¹⁴ - as in the case of the ARI. Further cross-cultural research might focus on investigating culturally universal, shared symptomatology (etic orientation) and notions that are part of experiences within a specific culture (emic orientation).¹⁴

Findings from the IRT analysis build on the original study reporting the development and investigation of the psychometric properties of the ARI.³ By analyzing the discrimination parameter of the items, we were able to demonstrate that item 2 (often lose temper) was the best discriminative item. This is consistent with both the theoretical definition of irritability in the ARI and the cardinal features of DMDD (even though DMDD also requires anger between outbursts).² By analyzing the severity parameter of the items, we were able to demonstrate that item 4 (angry most of the time) was the most severe item. To endorse such description, it is expected that the subject will present a relatively high level of the irritability latent trait and therefore this item may be best for differentiating subjects with high levels of the irritability latent trait. Findings from the APS logistic regression analyses, however, revealed that item 3 (stay angry for a long time) was the one most strongly associated with a higher probability of impairment. Even though severity levels and impairment are not necessarily interchangeable, it could be expected that the most severe item would be the one also most strongly associated with impairment. Therefore, this finding calls for further investigations with larger and more representative samples using IRT and classical theory psychometric analyses to increase our understanding of the functioning of the ARI measure at the item level.

The present study has some limitations. For instance, future studies are needed to examine further psychometric properties of the Brazilian Portuguese version of the ARI, such as convergent and divergent validity, discriminant validity, test-retest reliability, and sensitivity to treatment responses. Also, all children and adolescents in this study were part of a school sample, and therefore future studies should try to replicate these findings in clinical samples and investigate the clinical utility of the Brazilian Portuguese version of the ARI.

In conclusion, the ARI is a new instrument available to assess irritability symptoms in Brazilian youth. Our results demonstrate that the Brazilian Portuguese version of the ARI measure is very similar to the original ARI in terms of cross-cultural equivalence. Also, preliminary psychometric analyses showed good evidence of validity and reliability. If future studies demonstrate other adequate psychometric properties of the instrument, the Brazilian Portuguese version of the ARI could be an important tool in research and clinical settings to assist in the investigation of irritability in youth.

Acknowledgement

Dr. Argyris Stringaris gratefully acknowledges the support of the Wellcome Trust.

Submitted Jul 17 2013, accepted for publication Aug 07 2013.

This study was carried out at the Anxiety Disorders Outpatient Program for Child and Adolescent Psychiatry (PROTAIA), Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, RS, Brazil, and Center for Psychological Studies on At-Risk Populations (CEP-Rua), Institute of Psychology, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil.

No conflicts of interest declared concerning the publication of this article.

Financial support: Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq).

• 1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV). Washington: APA; 2000.
• 2. American Psychiatric Association. DSM-5 development. Arlington: American Psychiatric Association; 2012. http://www.dsm5.org/ Accessed 2013 May 13.
• 3. Stringaris A, Goodman R, Ferdinando S, Razdan V, Muhrer E, Leibenluft E, et al. The Affective Reactivity Index: a concise irritability scale for clinical and research settings J Child Psychol Psychiatry. 2012;53:1109-17.
• 4. Stringaris A. Irritability in children and adolescents: a challenge for DSM-5. Eur Child Adolesc Psychiatry. 2011;20:61-6.
• 5. Borsa JC, Bandeira DR. The use of psychological measures of child aggressive behavior: analysis of Brazilian scientific production. Aval Psicol. 2011;10:193-203.
• 6. Gjersing L, Caplehorn JR, Clausen T. Cross-cultural adaptation of research instruments: language, setting, time and statistical considerations. BMC Med Res Methodol. 2010;10:10-13.
• 7. Spence SH. A measure of anxiety symptoms among children. Behav Res Ther. 1998;36:545-66.
• 8. DeSousa DA, Petersen CS, Behs R, Manfro GG, Koller SH. Brazilian Portuguese version of the Spence Children's Anxiety Scale (SCAS-Brasil). Trends Psychiatry Psychother. 2012;34:147-53.
• 9. Yu CY. Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes. Los Angeles: University of California, Los Angeles; 2002.
• 10. Stringaris A, Cohen P, Pine DS, Leibenluft E. Adult outcomes of youth irritability: a 20-year prospective community-based study. Am J Psychiatry. 2009;166:1048-54.
• 11. Samejima F. Estimation of latent ability using a response pattern of graded scores [monograph]. Richmond: Psychometric Society; 1969. http://www.psychometrika.org/journal/online/MN17.pdf Accessed 2013 May 13.
• 12. Rizopoulos D. ltm: An R package for latent variable modelling and item response theory analyses. J Stat Softw. 2006;17:1-25.
• 13. Rescorla L, Ivanova MY, Achenbach TM, Begovac I, Chahed M, Drugli MB, et al. International epidemiology of child and adolescent psychopathology II: integration and applications of dimensional findings from 44 societies. J Am Acad Child Adolesc Psychiatry. 2012;51:1273-83.
• 14. Draguns JG, Tanaka-Matsumi J. Assessment of psychopathology across and within cultures: issues and findings. Behav Res Ther. 2003;41:755-76.

Publication Dates

• Publication in this collection
  16 Sept 2013
• Date of issue
  2013

History

• Received
  17 July 2013
• Accepted
  07 Aug 2013