Combining neuropsychological tests to improve the assessment of arithmetic difficulties in children with ADHD

Abstract Objective: To compare the ways of evaluating arithmetic skills in Brazilian children with ADHD by combining three validated neuropsychological tests and determining whether they are sensitive to the methylphenidate treatment. Methods: Forty-two children (9–12 years old) participated in the present study: 20 were children with ADHD (DSM-IV) and 22 were age-matched controls. A classification criterion was used for each test separately and one, for their combination to detect the presence of arithmetic difficulties at two time points: baseline (time 1); and when children with ADHD were taking 0.3–0.5 mg/kg of methylphenidate (time 2). The study also assessed children’s subtraction performance, combining parts of these tests. Results: Separately, the tests were only sensitive to differences between groups without medication. However, by combining the three neuropsychological tests, we observed a difference and detected a reduction in arithmetic difficulties associated with the methylphenidate treatment. The same effects were found in subtraction exercises, which require a borrowing procedure. Conclusions: The present study detected arithmetic difficulties in Brazilian children with ADHD and the effects of methylphenidate. Given this improvement in sensitivity, combining tests could be a promising alternative when working with limited samples.

Attention deficit hyperactivity disorder (ADHD) and arithmetic disorders, both individually and in combination, have a significant impact on the academic performance of individuals, as well as on their ability to professionally insert themselves when adults. Despite the high prevalence of arithmetic disorders in ADHD 1 , the literature on this condition is very limited 2 . Although individuals with ADHD generally have more arithmetic difficulties than the general population 3 , the underpinnings of this overlap are not well known.
Arithmetic operations are thought to involve various cognitive dimensions. For instance, subtraction operations have a different mechanism when it is required to subtract larger numbers from small numbers (requiring "borrowing"). Since 1993, Geary argues that forgetting to borrow or borrowing indirectly while attempting to solve a subtraction problem is related to a poor working memory and attentive skills 4 . Methylphenidate (MPH) is shown to improve numerical abilities related to working memory functions in children with ADHD 5 . Other studies found that children with ADHD (who were not taking any medication) were more likely to have arithmetic difficulties than their healthy peers, and that taking MPH led to an improvement in arithmetic performance 6,7,8 . At different doses, MPH administration can increase overall academic performance 8 .
To assess arithmetic difficulties, two important issues must be considered: detection and classification. Therefore, our initial challenge was to develop an appropriate method to assess arithmetic functions in Brazilian children. However, there is no consensus as to which test is optimal for this purpose; there is not even a consensus on the cutoff level used to determine levels of performance 9 . Instead of developing a new test, the strategy was to combine three tests, which are currently being used in Brazil, to evaluate whether they can collectively identify difficulties with arithmetic. These three tests were chosen because they tackle a wide range of arithmetic functions (number comprehension, number production, and calculation), forming the principles of arithmetic knowledge development according to McCloskey 10 . A comparison was made between the detection of arithmetic difficulties of each test separately and the combination of them. Each test separately is expected to be able to assess some arithmetic difficulties in ADHD children, but their combination is expected to provide a more sensitive detection of arithmetic difficulties and MPHs effects. In the consulted scientific literature, this is the first time these three tests have been used together for the same purpose. Furthermore, subtraction performance was evaluated because of its previous association with (and this study had a particular interest in that) working memory abilities. This study considered all types of difficulties as arithmetic difficulties, instead of trying to differentiate the less severe and more general difficulties in arithmetic from severe forms, such as dyscalculia. Hence, the purpose of this study was to compare ways of evaluating arithmetic skills in Brazilian children with ADHD by combining three validated neuropsychological tests and to determine whether they are sensitive to the methylphenidate treatment.

METHODS
The present study was approved by the Local Research Ethics Committee (CAPPesq/HC -FMUSP). A written informed consent for study participation was obtained from each child's legal guardian.
This interventional prospective study evaluated 42 children: 20 with a diagnosis of ADHD and 22 matched controls. All participants were 9-12 years old (mean age 9.9±0.9) with normal intelligence (IQ≥85) and no neuropsychiatric comorbidities, except for oppositional defiant disorder in three children from the ADHD group. The SNAP-IV for parents and teachers and "Benczik Scale for ADHD" for teachers 11,12 questionnaires were used to screen ADHD symptoms. Diagnoses were confirmed by two experienced pediatric neurologists following the DSM-IV criteria for ADHD, and ADHD children were referred by the neuropsychiatry team in a large metropolitan pediatric health neuroscience center. The control group had no clinical evidence of ADHD and was matched according to age, IQ, and socio-economic status.
In order to assess the intelligence level and arithmetic skills of all participants, three tests were used: a) Protocol for Calculus and Mathematical Reasoning Test (Bastos Test) 13 ; b) the Academic Performance Test in Arithmetic (TDE Arithmetic) 14 ; and c) the WISC-III -Arithmetic Sub-test 15 . Each test is described in further details in the following paragraphs: a) The Bastos Test 13 is a 50-minute test that addresses lexical and syntactic forms of written Arabic numerals, number production, transcoding of numbers, and calculations using the four basic arithmetic operations. The protocol consists of four items: • Assessment of lexical and syntactic ability. Participants are asked to: read a number word and convert it to its numerical symbol. Number words became progressively more difficult (units, tens, hundreds and thousands); convert a numerical symbol into a number word with numerical symbols becoming increasingly difficult (units, tens, hundreds and thousands). • Estimation of magnitude: assessment of the knowledge of magnitude with numbers of increasing difficulty (units, tens, hundreds and thousands). • Evaluation of the ability to add, subtract, multiply, and divide, which also became progressively more difficult. • Evaluation of mathematical reasoning ability.
b) The TDE Arithmetic Test 14 test is a written test with no time limit, that assesses fundamental arithmetic skills (addition, subtraction, multiplication, and division) with 35 exercises that become increasingly difficult with alternating types of test. c) Finally, the WISC-III -Arithmetic Subtest 15 is a timelimited verbal subtest of the Wechsler Intelligence Scale for Children (WISC-III). Children must solve each of the 17 increasingly difficult problems mentally, without using any pencil or paper. This test assesses addition, subtraction, multiplication, division, fractions, sense of time, and distance. Each item has a time limit and the test is stopped after the third consecutive error.
All participants were evaluated for the 3 tests (Bastos Test, TDE Arithmetic, and WISC-III -Arithmetic Sub-test) at two time points. The ADHD group was assessed at time 1 without medication and at time 2 under medication. Children in this group were kept medication-free for at least 1 week, at time 1, and under medication for at least two months, at time 2. The medication used had a short action MPH 0.3-0.5 mg/ kg, administered 90 minutes prior to assessment. The control group underwent the same evaluations as the ADHD group, with an interval of three months between the first (time 1) and second (time 2) assessments. Children in both groups were tested, and all of them responded to the Wechsler Intelligence Scale for Children-Third Edition (WISC-III) 15 (complete) only at time 1.
Assessment for: • Arithmetic Difficulties: children were classified as having arithmetic difficulties in each test, according to the following criteria: • Bastos Test: the criterion used was proposed previously 13 for the presence of arithmetic difficulties. A score lower than or equal to 58% in the Bastos Test was considered a "poor performance". • TDE Arithmetic Test: inferior score 14 .
• WISC-III -Arithmetic Subtest: score lower or equal to 7 16 . • Besides that, a combination of these three tests was used as follows: children were considered as having overall arithmetic difficulties when they displayed "poor performance" in at least two of the three tests described above. The same cutoff levels were used, but overall arithmetic difficulties were linked to performance in the three tests.

Subtraction performance
Basic numerical knowledge and working memory were also explored via subtraction exercises. There were seven subtraction operations in each test (Bastos Test and TDE Arithmetic Test), except for the WISC-III -Arithmetic Sub-test, which is an oral test. These subtraction operations were categorized as "simple" or "complex". Simple subtraction consisted of subtracting small numbers from large ones (e.g. 88-53), and complex subtraction involved "borrowing" or subtracting bigger from smaller numbers (e.g. 318-189). Thus, there were seven items for simple subtraction (4 items from the Bastos Test and 3 from the TDE Arithmetic Test) and seven for complex subtraction (3 on the Bastos Test and 4 on the TDE Arithmetic). The subtraction performance was calculated based on the total number of correct responses.

Statistical analysis
Fisher's exact test was used to compare differences in arithmetic difficulties between groups: time 1: ADHD (medication-free) versus control group, and time 2: ADHD (with medication) versus control group.
To determine the effect of MPH on their arithmetic function, the McNemar test was used to evaluate the concordance of results observed in the ADHD group: ADHD without medication versus ADHD with medication. The McNemar test was also used to assess the "learning effect" in the control group, measured by the control performance at time 1 versus time 2.
Fisher's exact test and the McNemar test were performed for the results of each of the three tests (Bastos Test, TDE Arithmetic Test and WISC-III -Arithmetic Subtest) separately and for the results from combining these tests. This processing could evaluate the benefits of using the proposed criterion for classifying arithmetic difficulties. The results allowed us to see if the same differences between groups and the same effect of MPH were observed using only one test for each classification criterion.
The Wilcoxon and Mann-Whitney tests were used to test the means subtraction scores for both simple and complex subtractions. The Wilcoxon test was used to assess subtraction performance within the same group at the two time points ( for example: performance of complex subtraction in ADHD group without medication versus ADHD group with medication). Following that, the Mann-Whitney test was used to compare subtraction performance between groups (control vs. ADHD). All analyses were conducted using the SPSS 13.0 software, and the significance level was set at p<0.05. Table 1 summarizes the number of children with arithmetic difficulties according to each test at time 1. At time 1 (without medication), when comparing ADHD and control children as to a "poor performance" on the Bastos Test, a worse performance was seen in the ADHD group (Fisher's exact test: p<0.001). This difference was also found when adopting only the TDE arithmetic test (Fisher's exact test: p=0.002). However, the WISC-III Arithmetic Subtest criterion showed that the groups' classifications were equivalent (Fisher's exact test: p=0.460). According to the combination of the three tests (Table 1), there were significantly more children in the ADHD group at time 1 with arithmetic difficulties relative to children in the control group (Fisher's exact test, p<0.001). When medicated with MPH at time 2 (Table 2), the children with ADHD had fewer arithmetic difficulties than at time 1 (McNemar's test, p=0.046). The number of children classified with arithmetic difficulties decreased from 13 (at time 1) to 6 (at time 2). As expected, control participants showed no significant difference in their overall arithmetic performance between the two time points (McNemar, p=1). Most importantly, at time 2 (Table 2), the groups did not differ from each other in terms of number of individuals with arithmetic difficulties (ADHD: 6/20 vs. Controls: 3/22; Fisher's exact test, p=0.269).

Complex subtraction
At time 1, children with ADHD (unmedicated) scored significantly worse than control children on the "complex subtraction" task (mean scores [

Simple subtraction
The same analyses of the afore mentioned were conducted for the "simple subtraction" task. No differences were found between patients and controls at either time point for this task (mean scores [

DISCUSSION
ADHD and arithmetic difficulties are frequently associated, and these variables were studied here in Brazilian children. The choice of which arithmetic test should be used was the first challenge. There was no guarantee that, by applying each of the three tests widely used in Brazil (Bastos Test, TDE Arithmetic, and WISC-III Arithmetic Subtest), the results aligned with current literature, which had already shown that ADHD children have more arithmetic difficulties than their healthy peers would be found 3 . This is due to two simple issues: the lack of a "perfect" arithmetic test (no consensus among researchers of the field) and this being the first time in Brazil that these tests were applied collectively to detect the presence of arithmetic difficulties in ADHD children.
The reason for using MPH was to check how aligned the tests were with their known effects, regarding working memory and attention to improve arithmetic skills 6,7,8 . The identification of arithmetic difficulties according to each test in the context of MPH treatment for ADHD children was analyzed. Separately, the Bastos Test and TDE Arithmetic Test could identify differences between children with ADHD (without medication) and the control group, which agrees with those of the literature 3 . However, the WISC-III Arithmetic Subtest could not detect this difference. The results were statistical similar between controls and ADHD children when the ADHD children were medicated. This improvement is  probably related to MPH, because none of these children suffered any other interventions, except for the use of MPH.
In the ADHD group, the tests did not show better results separately, before and after MPH. However, by combining these three tests the following results were found: there was a worse performance in unmedicated ADHD children than in the control group; an improved performance was associated to the MPH treatment, and no significant difference was seen between medicated ADHD children and the control group.
These results could be explained by the fact that none of the tests individually fulfill McCloskey's principles 10 to study arithmetic knowledge: number comprehension, number production and calculation processing. However, combining these three tests could detect the presence of arithmetic difficulties and expands the sensitivity of the process.
In terms of mathematical cognition domain, complex subtractions were explored based on different authors statements, who say that working memory and attention are part of subtraction skills that involve the borrowing procedure 4,5 . Thus, combining the complex subtractions of Bastos and TDE Arithmetic tests provided similar results to the combination of all three tests. Simple subtraction could not identify any differences between groups. Furthermore, as found by Rubinsten et al. 5 , MPH had no effects on simple subtraction performance, probably because this is a basic processing skill, that relies on the number sense, which is an inherited ability 17 . The improved complex subtraction performance could be attributed to the MPH effects on executive functions, such as working memory. It may be possible that this discrepancy regarding simple and complex subtractions in children with ADHD suggests that basic arithmetic difficulties need to be addressed with specific educational interventions; those based on working memory and attention might benefit from medication 5,18 .
The study design is controlled for possible experimental placebo-related effects (task learning and research team interaction effects) and for time-related effects (two timepoints in both groups). However, possible effects of MPH's medication were not controlled with a specific placebo-medication. If a medication-wise placebo arm was used in both groups, the effects could also be present in the control group. Moreover, the experiment was not double-blinded. However, it is expected that these characteristics would affect the tests separately and their combination similarly. The placebo effects would also probably increase the differences between time 1 and time 2 in the ADHD group. Nonetheless, given that MPH benefits are well established, and only the tests in combination detected these improvements in the ADHD group, we find that these results were not altered by placebo effects. Another possible aspect is that the study sample size is relatively small. Even a small sample could provide strong evidence aligned with literature as seen with our subtraction data. These results were quite similar to that of Rubinsten et al. 5 . We reiterate that we are not trying to validate a new test but rather suggesting that combining tests could increase sensitivity, which could be especially useful in studies with smaller samples.
In conclusion, instead of creating a new test, the use of existing tests was recycled to offer a more complete assessment of arithmetical difficulties when applied in combination. When using this combination of validated tests, arithmetic difficulties could be detected in Brazilian children with ADHD, besides demonstrating the benefits of methylphenidate. Currently, the authors are working on a neuroimaging version of this protocol to explore the neural underpinnings of ADHD and arithmetic disorders. Recycling existing tools might be a way to align the different world groups that research arithmetic disorders and establish a consensual criterion to detect the presence of arithmetic difficulties.