Development and verifi cation of internal structure of the Adult Attention-Defi cit / Hyperactivity Disorder Screening Inventory

The aim of this study was to develop the Adult Attention-Defi cit/Hyperactivity Disorder Screening Inventory, a self-report instrument for assessing symptoms as well as to verify the internal structure of the instrument. Based on the proposed Diagnostic and Statistical Manual of Mental Disorders V Attention-Defi cit/Hyperactivity Disorder diagnostic criteria, the Screening Inventory was developed and administered. The participants were 421 individuals divided into two groups, with and without Attention-Defi cit/Hyperactivity Disorder diagnosis. The instrument items were analyzed and validated for content by a panel of expert judges. It was found that the instrument structure is composed of two factors, namely, Inattention and Hyperactivity/Impulsivity (Cronbach’s alpha 0.97 and 0.96, respectively). In general, the observed data provided validity evidence based on content and internal structure of the instrument, which complies with the latest edition of the Diagnostic and Statistical Manual of Mental Disorders, proving to be a favorable instrument for the investigation of Attention-Defi cit/Hyperactivity Disorder symptoms in Adults.

The DSM is the main reference for the diagnostic criteria of ADHD.In 2013, the publication of the fifth edition of the manual clarified the occurrence of ADHD symptoms in adults.In the previous edition [DSM-IV-TR (APA, 2003)], the diagnostic criteria were developed and validated for children, which used general and subjective terms, making it difficult to clearly define the symptoms that should be evaluated.Not long ago, as pointed out by Caliman (2010) and Barkley, Murphy, and Fischer (2013), ADHD was considered a childhood disorder that would disappear with maturation.According to these authors, as of the 1970s, follow-up studies found that the behavioral patterns of children persisted when they reached adulthood, thus highlighting the need for research on the occurrence of the disorder during development.
Despite progress regarding the diagnostic criteria for ADHD, subsequent to the release of DSM-5, the need for studies that address the disorder in adulthood is still evident, particularly regarding the diagnosis of the disorder.Barkley (2010) conducted a review on the most effective procedures for assessing ADHD and the tools for evaluating symptoms, which showed that the evaluation of this disorder, regardless of the age group investigated, is complex and requires collecting a series of information.The author lists four evaluative modes that are complementary for the assessment of ADHD in adults: (a) structured interviews, (b) neuropsychological assessment, (c) historical records, and (d) screening scales.Considering the scope of this study, we will give greater emphasis to information related to the screening procedures.
In Barkley's survey (2010), we observed that among the screening scales listed, there was only one available in Brazil, the Adult ADHD Self-Report Scale .This self-report scale of ADHD symptoms for adults consists of 18 items, and it was developed by a working group of the World Health Organization to assess ADHD among adults (Kessler et al., 2007).The ASRS-18 corresponds to symptoms listed on the DSM-IV-TR adapted for adults.The translation and adaptation to the Brazilian reality was carried out by Mattos et al. (2006), who, during the adaptation study, concluded that the translation was equivalent to the original version proposed by the WHO.Favorable evidence for its use was found by the study conducted by Leite (2011), who investigated the psychometric properties of the instrument.
In addition to Barkley, Murphy, & Fischer (2013) survey, the Attention Deficit/Hyperactivity Disorder Scale (Escala de Transtorno do Déficit de Atenção e Hiperatividade -ETDAH-AD) (Benczick, 2013), consisted of five factors aiming to assist the diagnostic process of ADHD.Although it is not a specific instrument for adults, it evaluates individuals aged 12-87 years.The instrument's manual presents studies suggesting its psychometric adequacy.It is noteworthy that, similarly to the ASRS-18, the ETDAH was based on the fourth edition of the DSM.However, no publications about this instrument were found in scientific journals.
We found a limited number of tools that assess ADHD symptoms in adults in Brazil.Testing for ADHD in adults should reflect the specifics of the symptoms in adults (APA, 2013;Louzã, 2010;Mattos, 2010).The instruments available in Brazil, despite updates in the literature, were mainly based on the literature focused on adult ADHD symptoms (APA, 2003).The aim of this study was to develop a tool for assessing symptoms of ADHD in adults; furthermore, we verified the psychometric properties, validity evidence based on content and internal structure, and reliability indices for internal consistency of the instrument.

Method
The method is divided into two stages.The first stage describes the development of the instrument, "Adult ADHD Screening Inventory" (Inventário de Rastreamento do TDAH em Adultos), and the second stage addresses the analyses conducted to determine the internal structure of the instrument.

Stage I -Development of the Adult ADHD Screening Inventory (IR-ADHD)
The development stage of the IR-ADHD was divided and carried out in five steps.The first step was to define the diagnostic guidelines for the operationalization of the construct; therefore, we conducted a literature review of the last five years in national and international databases.We found that the main reference in the ADHD diagnostic criteria was the Diagnostic and Statistical Manual of Mental Disorders (DSM), which was adopted as the main framework for the development of items, particularly the fifth edition.
The second step consisted of the operationalization of the construct.The authors of the instrument individually proposed the items, considering the symptoms listed in the fourth and fifth editions of the DSM.The next step was the selection of the items developed by the authors, excluding recurring items or items that were considered unsuitable for adults.
After selecting the items, the fourth step was to submit the selected items to the analysis of expert judges.Three experts with experience in ADHD were invited to evaluate the items.One of the experts holds a PhD and the other one a master's degree in Psychology.The third expert has experience in the development of psychological tests and holds a PhD degree in Psychology.The experts were asked to evaluate the developed items to classify them as either appropriate or inappropriate to compose the scale.This procedure addressed the distribution of items into one of the three possible dimensions of ADHD defined by the Statistical Manual of Mental Disorders (inattention, hyperactivity and impulsivity).
In addition to classifying the item into one of the three dimensions of ADHD, the judges had to assess whether the item was favorable or not to compose the instrument; and, finally, in the fifth step, the judges' considerations were analyzed.To remain in the inventory, the item had to receive the favorable opinion of all the judges.As a final product, the final version of the IR-ADHD was obtained.

Participants
A total of 421 participants, aged between 18 and 80 years (M = 24.8years, SD = 8.11), who were predominantly female (76.0%), from municipalities in the countryside of the state of São Paulo, Brazil, were recruited by convenience sampling.The sample mostly consisted of university students since the administration of the instrument took place in university classrooms, facilitating access to participants.This fact explains the expressive number of participants with incomplete higher education (83.6%, N = 352) and most participants did not have a diagnosis of ADHD (94.6%, N = 398).The sample was also composed of a clinical group diagnosed with ADHD by physicians (3.8%, N = 16) and a group of individuals who did not report the origin of diagnosis (1.7%, N = 7).It is noteworthy that in the case of the clinical group, participants were by a specialized center for adult patients with ADHD diagnosis, which had been established by the physician through clinical interviews.

Instruments
The IR-ADHD, instrument developed during the first stage of this study, was administered to participants.It is a self-report inventory and the version administered consisted of 119 items representing symptoms of inattention, hyperactivity, and impulsivity.Items had to be answered on a 4-point Likert-type scale, where 1 means the item has "nothing" to do with the individual; 2, the item has "little" to do with the individual; 3, the item "moderately" represents the individual's everyday life; and 4, the item has "a lot" to do with the individual.The administration time was approximately 20 minutes.

Procedures
After approval by the Research Ethics Committee (C.A.A.E.13080213.5.0000.5514) of the University San Francisco (Universidade São Francisco), data collection began.Participants responded to the instrument after reading and signing the informed consent form.Participants without a diagnosis of ADHD were recruited from universities where the administration was conducted collectively.In the group diagnosed with the disorder, collection was carried out individually, as the participants were patients at a specialized service for adults with ADHD and responded to the instrument after routine medical care with the psychiatrist.
After the collection and tabulation of data, statistical analyses were performed.First, to search for validity evidence based on internal structure, the maximum number of factors to be checked in the exploratory factor analysis was calculated based on the parallel analysis for polychoric items (Hayton, Allen, & Scarpello, 2004;Watkins, 2006) using the R software (R Foundation for Statistical Computing, Vienna, Austria), version 2.15.3.A database was generated for the MPlus software (Los Angeles, CA, United States) version 6.12, to perform the exploratory factor analysis with polychoric variables, in addition to obtaining fit indices Exploratory Structural Equation Modeling (ESEM) that indicate the adequacy of the structure based on this sample.Next, we used procedures for exclusion/maintenance of items and, finally, we verified the reliability for internal consistency of the factors to obtain the final version of the instrument?

Results and Discussion
At first, a bank of items was structured in a spreadsheet format, with 294 initial items arranged in the three dimensions (inattention, hyperactivity and impulsivity).It must be pointed out that positive (which indicated the presence of the symptoms) and negative items were developed.Three main sources of information for the development of items were used: the DSM-IV-TR (APA, 2003), DSM-5 (APA, 2013) and the Adult Self-Report Scale for Adults -ASRS-18 (Mattos et al., 2006).Table 1 shows the number of items developed during step two, the selection of items in step three, and the items selected to compose the instrument's version for administration.
Table 1 shows that, from step two (development of items) to step three (selection of items by the authors), almost half the items were excluded.This occurred due to the way the items were developed, i.e., independently, which generated a substantial number of similar items; therefore, in addition to the selection of items covering the relevant symptoms of ADHD in adults, we also sought to exclude items with similar content.At this stage, 145 items remained, which were selected and analyzed by the expert judges (step four).
The items that were pre-selected and analyzed by the judges had to be allocated according to the evaluative dimension (inattention with 82 items, hyperactivity with 49 items, and impulsivity with 14 items).Only items that received a favorable opinion from all judges remained in the inventory.Thus, the inattention dimension was composed of 63 items after the judges' analysis (with items such as "I have difficulty organizing my time").The hyperactivity dimension was composed of 47 items (example of item, "It is not uncommon for me to change activities before completing them"), and the impulsivity scale was composed of nine items (with items such as, "I often act before thinking and this harms me").The disparity in the number of items of the dimensions proposed by the authors can be justified by how the symptoms of impulsivity are placed in the group of DSM-5 criteria (APA, 2013).The manual presents symptoms of hyperactivity and impulsivity allocated in the same group of symptoms criteria, with six hyperactivity symptoms and three impulsivity symptoms.Consequently, the number of items proposed by the authors was lower.
After the judges' analysis, 26 items were excluded from the total number of items of the scale, so the final version of the instrument was composed of 119 items.Items from all dimensions were excluded, but only two items were excluded from the hyperactivity dimension after the experts' analysis.The analysis  process carried out by expert judges aimed to ensure that the remaining items did in fact seek to evaluate the, typical symptoms of ADHD in adults (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014).This is considered, therefore, as the first validity evidence based on the content for the items of the IR-ADHD.
After the IR-ADHD development phase, we verified the psychometric properties of the instrument.First, we sought to verify the maximum number of factors for the total set of items.To do so, from the parallel analysis for polychoric variables up to five factors were obtained with significant eigenvalue not shown by chance.Based on this information, we proceeded to exploratory factor analysis with confirmatory indices ESEM, forcing up solutions from one to five factors, using the Geomin oblique rotation and the Maximum Likelihood Robust (MLR) extraction method, which is a robust method suitable for polychoric variables.We analyzed the adjustment indices generated for models and observed a slight improvement in the indexes as we increased the number of factors.However, we also found that the first factor in all models was composed of items on inattention and the second factor by items on hyperactivity and impulsivity, so that the remaining factors (for the models with three, four and five factors) obtained few items with significant loads (five items, at most).In this sense, the best solution for interpretability was two factors.Therefore, and considering a minimum adequacy of fit indices, we opted for the two-factor model.The fit indices obtained by the twofactor model were c 2 /df = 2.06 (good); RMSEA = 0.056 (marginally good); CFI = 0.670 (unsatisfactory); and, SMR = 0.057 (good), which was based on Hooper, Coughlan, and Mullen (2008).Five procedures were used to determine whether each of the items would be maintained in the two dimensions of the instrument.
First, only items with factor loadings greater than 0.30 and minimum difference of 0.05 in the factor loading of both factors were retained, thus maintaining the discrimination between them.In Factor 1, factor loading variation was between 0.76 (item 90) and -0.31 (item 28), while in Factor 2 the maximum value was 0.77 (item 78) and the minimum value was 0.30 (item 9).Factor 1 is composed of five reversed items (items 3, 5, 20, 21 and 28), whereas Factor 2 had no negative items.
After this first selection, 18 items were excluded, since they did not meet the established values for the definition of factors.The second procedure adopted was checking the reliability of each of the instrument factor.To do so, we investigated all the items of each factor to verify if any of them would hinder or would not contribute to the accuracy of the instrument.There were no items that did not contribute to the internal consistency or that hindered it.Furthermore, we obtained the values α = 0.97 for the Inattention factor (61 items) and α = 0.96 for the Hyperactivity factor with impulsivity (40 items), and 0.98 for the total set of items.
Next, the "t"-test was used for comparing the group of participants with a diagnosis and the group of participants without diagnosis of ADHD.The comparison was made at the level of the items to verify if any of the items did not present a significant difference between the means of the two groups.Based on this criterion, two Factor-1 items (19 and 103) and nine Factor-2 items (9, 12, 15, 42, 64, 97, 106, 109, and 112) were excluded.Therefore, after the third stage, Factor 1 was composed of 59 items with an accuracy of α = 0.97, and Factor 2 consisted of 31 items with an accuracy of α = 0.95.
In this group of items, we sought to determine whether the factors consisted of items with high level of endorsement.To do so, the data were analyzed, by factor, based on the rating scale model using the Winsteps software.In the case of Factor 1, all items had levels of difficulty (b) equal to or higher than the average of the sample, so all items in this factor were maintained.However, during qualitative analysis of the sentences belonging to Factor 1, certain similarities were found between the content of items (3, 13, 14, 20, 28, 37, 44, 47, 114 and 116).We observed the level of difficulty of the similar items and decided to exclude the repeated items with lower level of difficulty.Therefore, three items were excluded (13, 28 and 47), and the other four items were kept, as they presented different levels of difficulty.
Factor 2, in turn, presented four items (31, 38, 75 and 78) with levels of difficulty below the levels of the sample.However, only one of the four items were excluded (75), since the authors concluded, in the qualitative analysis, that the contents of the other items were significant for ADHD.After the items were excluded, Factor 1 was composed of 56 items and Factor 2 of 30 items.
As a final procedure, still via rating scale model, we observed that the fit indices (infit and outfit) and the item-theta correlation.No item was excluded from Factor 2 and only one item was excluded from Factor 1 as it increased the fit indices (> 1.4), (Linacre, 2009) and content was not specific (item 63 -"I've always had trouble noticing details").After all stages used in the investigation of the items of the instrument, a final version of the instrument was obtained that consisted of 85 items: 55 Factor-1 items (Inattention) and 30 Factor-2 items (Hyperactivity/Impulsivity).The coefficients found for the dimensions and total score of the instrument were adequate (Nunnally, 1978), considering that the values given by Cronbach's α were 0.96 (Inattention) and 0.97 (Hyperactivity/Impulsivity), and the total score was equal to 0.98.
The data found, regarding the final version of IR-ADHD, shows that it is composed of two factors.The first factor was named Inattention and it aims to assess the typical behavior of this symptomatic manifestation.The items seek to evaluate the patients' ability to stay focused on a stimulus, failure to pay attention to details, and difficulties to remain seated and complete a task.The dimension also aims to assess the subjects' ability to organize their time and complete the tasks on schedule, and the ability to inhibit competing thoughts during the task they are performing.The second dimension, Hyperactivity/Impulsivity, aims to assess the typical symptoms of excessive anxiety, which can manifest through motor behavior and the feeling of mental agitation.In addition to this pattern of hyperactivity, the dimension still verifies classic symptoms of impulsivity in which the individual is unable to inhibit impulsive responses, which often put them at risk, such as, not waiting for their turn to do something, having difficulty staying in same place, and behaviors that may jeopardize their physical integrity.
Another major feature in the structure of the instrument is the disparity in the number of items when the two dimensions are compared, as it was based on the diagnostic criteria of the latest edition of the DSM (APA, 2013).According to the sets of symptoms proposed by criterion A (DSM-5), it is possible to observe a higher number of symptoms in the inattention group, which consists of nine symptoms; while the second group of symptoms, though also composed of nine statements, comprises six hyperactivity symptoms and three impulsivity symptoms.This difference in the number of symptoms reflected the number of items developed for the operationalization of the construct in each one of the dimensions proposed by the DSM.Verification carried out to understand the organization of items in the inventory and the decision to propose the IR-ADHD with two factors (Factor 1 -Inattention and Factor 2 -Hyperactivity/Impulsivity) corroborates with the symptoms proposed by the DSM-5 (APA, 2013), in which the symptoms of hyperactivity and impulsivity are defined within the same group of symptoms.
Another explanation for the difference in the number of items between the two dimensions is that the symptoms of inattention in adults with ADHD tends to be more visible, while signs of hyperactivity and impulsivity manifest through covert behaviors and agitation/mental restlessness (Carvalho, Kolling, Silva, Cunha, & Kristensen, 2008).This tendency is also supported by the DSM-5 (APA, 2013).

Conclusion
The shortage of psychological tests for evaluating ADHD is a reality in the scientific community dedicated to the study of this disorder, and, as a result, professionals do not have tools that aid them in the diagnosis process.This lack of materials is even more evident when considering instruments for assessing ADHD in adults.
As the objective of the present study was to develop an instrument to assist the investigation process of ADHD symptoms in adults, we can say that it was satisfactorily achieved, since the IR-ADHD is a selfreport instrument with validity evidence based on content and internal structure.The final structure of the instrument, which comprises two factors, Inattention and Hyperactivity/Impulsivity, is a proposal that meets the criteria established by the DSM-5.However, the findings are primary and exploratory, and further studies should verify validity evidence based on external criteria by investigating, for instance, the discriminative ability of the instrument and its relation to other tests in the area.In addition, we highlight the importance of conducting further analysis to identify the diagnostic accuracy of IR-ADHD.
Moreover, despite the good results obtained, we should mention the need to refine the instrument, since it is composed of 85 items, which can be considered a large scale for clinical use.For patients with ADHD, having to carry out an activity for a long period of time may be somewhat demanding.Therefore, the possibility of reducing the number of items is valid.In addition, further studies should investigate the psychometric properties of the hetero-report version, which was not investigated in the present study, and use larger psychiatric samples.Contributors J.T. BACCIOTTI contributed to the conception, design of the study, discussion of the results, and review of the manuscript.L.F. CARVALHO contributed to the conception, design of the study, statistical analysis, and discussion of the results.

Table 1
Items developed and selected for operationalization of the construct and analyses of expert judges