Combining neuropsychological tests to improve the assessment of arithmetic difficulties in children with ADHD

Combinando testes neuropsicológicos para melhorar a avaliação de dificuldades aritméticas em crianças com TDAH

Abstract

Objective:

To compare the ways of evaluating arithmetic skills in Brazilian children with ADHD by combining three validated neuropsychological tests and determining whether they are sensitive to the methylphenidate treatment.

Methods:

Forty-two children (9‒12 years old) participated in the present study: 20 were children with ADHD (DSM-IV) and 22 were age-matched controls. A classification criterion was used for each test separately and one, for their combination to detect the presence of arithmetic difficulties at two time points: baseline (time 1); and when children with ADHD were taking 0.3‒0.5 mg/kg of methylphenidate (time 2). The study also assessed children’s subtraction performance, combining parts of these tests.

Results:

Separately, the tests were only sensitive to differences between groups without medication. However, by combining the three neuropsychological tests, we observed a difference and detected a reduction in arithmetic difficulties associated with the methylphenidate treatment. The same effects were found in subtraction exercises, which require a borrowing procedure.

Conclusions:

The present study detected arithmetic difficulties in Brazilian children with ADHD and the effects of methylphenidate. Given this improvement in sensitivity, combining tests could be a promising alternative when working with limited samples.

Keywords:
math test; arithmetic difficulties; learning disabilities; ADHD; methylphenidate

Resumo

Objetivo:

Comparar as formas de avaliar as habilidades aritméticas em crianças brasileiras com TDAH, combinando três testes neuropsicológicos validados, e verificar se são sensíveis ao tratamento com metilfenidato.

Métodos:

Quarenta e duas crianças (9‒12 anos) participaram deste estudo: 20 eram crianças com TDAH (DSM-IV) e 22 eram controles pareados por idade. Usamos um critério de classificação para cada teste separadamente e outro para a combinação entre eles, visando detectar a presença de dificuldades aritméticas em dois momentos: início (tempo 1) e quando as crianças com TDAH estavam tomando 0,3‒0,5 mg/kg de metilfenidato (tempo 2). O estudo também avaliou o desempenho dessas crianças em operações de subtração, combinando partes desses testes.

Resultados:

Separadamente, os testes foram sensíveis apenas às diferenças entre os grupos sem medicação. Entretanto, ao combinar os três testes neuropsicológicos, foi possível observar uma diferença e detectar uma redução das dificuldades aritméticas associadas ao tratamento com metilfenidato. Os mesmos efeitos foram encontrados em exercícios de subtração que exigem o procedimento de empréstimo.

Conclusões:

O estudo foi capaz de detectar dificuldades aritméticas em crianças brasileiras com TDAH e os efeitos do metilfenidato. Dada essa melhora na sensibilidade, combinar testes poderia ser uma alternativa promissora ao trabalhar com amostras limitadas.

Palavras-chave:
teste de matemática; dificuldades aritméticas; dificuldades de aprendizagem; TDAH; metilfenidato

Attention deficit hyperactivity disorder (ADHD) and arithmetic disorders, both individually and in combination, have a significant impact on the academic performance of individuals, as well as on their ability to professionally insert themselves when adults. Despite the high prevalence of arithmetic disorders in ADHD¹, the literature on this condition is very limited². Although individuals with ADHD generally have more arithmetic difficulties than the general population³, the underpinnings of this overlap are not well known.

Arithmetic operations are thought to involve various cognitive dimensions. For instance, subtraction operations have a different mechanism when it is required to subtract larger numbers from small numbers (requiring “borrowing”). Since 1993, Geary argues that forgetting to borrow or borrowing indirectly while attempting to solve a subtraction problem is related to a poor working memory and attentive skills⁴. Methylphenidate (MPH) is shown to improve numerical abilities related to working memory functions in children with ADHD⁵. Other studies found that children with ADHD (who were not taking any medication) were more likely to have arithmetic difficulties than their healthy peers, and that taking MPH led to an improvement in arithmetic performance^6,7,8. At different doses, MPH administration can increase overall academic performance⁸.

To assess arithmetic difficulties, two important issues must be considered: detection and classification. Therefore, our initial challenge was to develop an appropriate method to assess arithmetic functions in Brazilian children. However, there is no consensus as to which test is optimal for this purpose; there is not even a consensus on the cutoff level used to determine levels of performance⁹. Instead of developing a new test, the strategy was to combine three tests, which are currently being used in Brazil, to evaluate whether they can collectively identify difficulties with arithmetic. These three tests were chosen because they tackle a wide range of arithmetic functions (number comprehension, number production, and calculation), forming the principles of arithmetic knowledge development according to McCloskey¹⁰. A comparison was made between the detection of arithmetic difficulties of each test separately and the combination of them. Each test separately is expected to be able to assess some arithmetic difficulties in ADHD children, but their combination is expected to provide a more sensitive detection of arithmetic difficulties and MPHs effects. In the consulted scientific literature, this is the first time these three tests have been used together for the same purpose. Furthermore, subtraction performance was evaluated because of its previous association with (and this study had a particular interest in that) working memory abilities. This study considered all types of difficulties as arithmetic difficulties, instead of trying to differentiate the less severe and more general difficulties in arithmetic from severe forms, such as dyscalculia. Hence, the purpose of this study was to compare ways of evaluating arithmetic skills in Brazilian children with ADHD by combining three validated neuropsychological tests and to determine whether they are sensitive to the methylphenidate treatment.

METHODS

The present study was approved by the Local Research Ethics Committee (CAPPesq/HC - FMUSP). A written informed consent for study participation was obtained from each child’s legal guardian.

This interventional prospective study evaluated 42 children: 20 with a diagnosis of ADHD and 22 matched controls. All participants were 9‒12 years old (mean age 9.9±0.9) with normal intelligence (IQ≥85) and no neuropsychiatric comorbidities, except for oppositional defiant disorder in three children from the ADHD group. The SNAP-IV for parents and teachers and “Benczik Scale for ADHD” for teachers^11,12 questionnaires were used to screen ADHD symptoms. Diagnoses were confirmed by two experienced pediatric neurologists following the DSM-IV criteria for ADHD, and ADHD children were referred by the neuropsychiatry team in a large metropolitan pediatric health neuroscience center. The control group had no clinical evidence of ADHD and was matched according to age, IQ, and socio-economic status.

In order to assess the intelligence level and arithmetic skills of all participants, three tests were used: a) Protocol for Calculus and Mathematical Reasoning Test (Bastos Test)¹³; b) the Academic Performance Test in Arithmetic (TDE Arithmetic)¹⁴; and c) the WISC-III - Arithmetic Sub-test¹⁵. Each test is described in further details in the following paragraphs:

1. The Bastos Test¹³ is a 50-minute test that addresses lexical and syntactic forms of written Arabic numerals, number production, transcoding of numbers, and calculations using the four basic arithmetic operations. The protocol consists of four items:

   • Assessment of lexical and syntactic ability. Participants are asked to: read a number word and convert it to its numerical symbol. Number words became progressively more difficult (units, tens, hundreds and thousands); convert a numerical symbol into a number word with numerical symbols becoming increasingly difficult (units, tens, hundreds and thousands).

   • Estimation of magnitude: assessment of the knowledge of magnitude with numbers of increasing difficulty (units, tens, hundreds and thousands).

   • Evaluation of the ability to add, subtract, multiply, and divide, which also became progressively more difficult.

   • Evaluation of mathematical reasoning ability.

2. The TDE Arithmetic Test¹⁴ test is a written test with no time limit, that assesses fundamental arithmetic skills (addition, subtraction, multiplication, and division) with 35 exercises that become increasingly difficult with alternating types of test.

3. Finally, the WISC-III - Arithmetic Subtest¹⁵ is a time-limited verbal subtest of the Wechsler Intelligence Scale for Children (WISC-III). Children must solve each of the 17 increasingly difficult problems mentally, without using any pencil or paper. This test assesses addition, subtraction, multiplication, division, fractions, sense of time, and distance. Each item has a time limit and the test is stopped after the third consecutive error.

All participants were evaluated for the 3 tests (Bastos Test, TDE Arithmetic, and WISC-III - Arithmetic Sub-test) at two time points. The ADHD group was assessed at time 1 without medication and at time 2 under medication. Children in this group were kept medication-free for at least 1 week, at time 1, and under medication for at least two months, at time 2. The medication used had a short action MPH 0.3‒0.5 mg/kg, administered 90 minutes prior to assessment. The control group underwent the same evaluations as the ADHD group, with an interval of three months between the first (time 1) and second (time 2) assessments. Children in both groups were tested, and all of them responded to the Wechsler Intelligence Scale for Children-Third Edition (WISC-III)¹⁵ (complete) only at time 1.

Assessment for:

• Arithmetic Difficulties: children were classified as having arithmetic difficulties in each test, according to the following criteria:

• Bastos Test: the criterion used was proposed previously¹³ for the presence of arithmetic difficulties. A score lower than or equal to 58% in the Bastos Test was considered a “poor performance”.

• TDE Arithmetic Test: inferior score¹⁴.

• WISC-III - Arithmetic Subtest: score lower or equal to 7¹⁶.

• Besides that, a combination of these three tests was used as follows: children were considered as having overall arithmetic difficulties when they displayed “poor performance” in at least two of the three tests described above. The same cutoff levels were used, but overall arithmetic difficulties were linked to performance in the three tests.

Subtraction performance

Basic numerical knowledge and working memory were also explored via subtraction exercises. There were seven subtraction operations in each test (Bastos Test and TDE Arithmetic Test), except for the WISC-III - Arithmetic Sub-test, which is an oral test. These subtraction operations were categorized as “simple” or “complex”. Simple subtraction consisted of subtracting small numbers from large ones (e.g. 88‒53), and complex subtraction involved “borrowing” or subtracting bigger from smaller numbers (e.g. 318‒189). Thus, there were seven items for simple subtraction (4 items from the Bastos Test and 3 from the TDE Arithmetic Test) and seven for complex subtraction (3 on the Bastos Test and 4 on the TDE Arithmetic). The subtraction performance was calculated based on the total number of correct responses.

Statistical analysis

Fisher's exact test was used to compare differences in arithmetic difficulties between groups: time 1: ADHD (medication-free) versus control group, and time 2: ADHD (with medication) versus control group.

To determine the effect of MPH on their arithmetic function, the McNemar test was used to evaluate the concordance of results observed in the ADHD group: ADHD without medication versus ADHD with medication. The McNemar test was also used to assess the “learning effect” in the control group, measured by the control performance at time 1 versus time 2.

Fisher's exact test and the McNemar test were performed for the results of each of the three tests (Bastos Test, TDE Arithmetic Test and WISC-III - Arithmetic Subtest) separately and for the results from combining these tests. This processing could evaluate the benefits of using the proposed criterion for classifying arithmetic difficulties. The results allowed us to see if the same differences between groups and the same effect of MPH were observed using only one test for each classification criterion.

The Wilcoxon and Mann-Whitney tests were used to test the means subtraction scores for both simple and complex subtractions. The Wilcoxon test was used to assess subtraction performance within the same group at the two time points (for example: performance of complex subtraction in ADHD group without medication versus ADHD group with medication). Following that, the Mann-Whitney test was used to compare subtraction performance between groups (control vs. ADHD). All analyses were conducted using the SPSS 13.0 software, and the significance level was set at p<0.05.

RESULTS

Arithmetic performance

Table 1 summarizes the number of children with arithmetic difficulties according to each test at time 1. At time 1 (without medication), when comparing ADHD and control children as to a “poor performance” on the Bastos Test, a worse performance was seen in the ADHD group (Fisher's exact test: p<0.001). This difference was also found when adopting only the TDE arithmetic test (Fisher's exact test: p=0.002). However, the WISC-III Arithmetic Subtest criterion showed that the groups’ classifications were equivalent (Fisher's exact test: p=0.460).

Table 1.
Number of children with arithmetic difficulties at time 1.

Group (n)	Arithmetic Difficulties at time 1
	Bastos Test	TDE Arithmetic Test	WISC-III Arithmetic Subtest	Combination of the three tests
ADHD (20)	13	15	3	13
Control (22)	2	5	6	2

• ADHD: Attention Deficit Hyperactivity disorder; TDE Arithmetic Test: Academic Performance Test in Arithmetic; WISC-III: Wechsler Intelligence Scale for Children - Third Edition - Arithmetic Subtest; Math: Combination of the three tests

Table 2 summarizes the number of children with arithmetic difficulties according to each test at time 2. At time 2, considering each test separately, no difference was found between groups (Fisher's exact test: Bastos Test p=0.152; TDE Arithmetic p=0.758; WISC-III Arithmetic Subtest p=0.414). Also, the analysis could not detect a significant effect of the MPH treatment on ADHD children (McNemar's test: Bastos Test p=0.077; TDE Arithmetic p=0.289; WISC-III Arithmetic Subtest p=1). There were no differences between time 1 and time 2 in the control group (McNemar's test: Bastos Test p=1; TDE Arithmetic p=0.182; WISC-III Arithmetic Subtest p=1).

According to the combination of the three tests (Table 1), there were significantly more children in the ADHD group at time 1 with arithmetic difficulties relative to children in the control group (Fisher’s exact test, p<0.001). When medicated with MPH at time 2 (Table 2), the children with ADHD had fewer arithmetic difficulties than at time 1 (McNemar’s test, p=0.046). The number of children classified with arithmetic difficulties decreased from 13 (at time 1) to 6 (at time 2). As expected, control participants showed no significant difference in their overall arithmetic performance between the two time points (McNemar, p=1). Most importantly, at time 2 (Table 2), the groups did not differ from each other in terms of number of individuals with arithmetic difficulties (ADHD: 6/20 vs. Controls: 3/22; Fisher’s exact test, p=0.269).

Table 2.
Number of children with arithmetic difficulties at time 2.

Group (n)	Arithmetic Difficulties at time 2
	Bastos Test	TDE Arithmetic Test	WISC-III Arithmetic Subtest	Combination of the three tests
ADHD (20)	7	11	2	6
Control (22)	3	10	5	3

• ADHD: Attention Deficit Hyperactivity disorder; TDE Arithmetic Test: Academic Performance Test in Arithmetic; WISC-III: Wechsler Intelligence Scale for Children - Third Edition - Arithmetic Subtest; Math: Combination of the three tests

Subtraction performance

Complex subtraction

At time 1, children with ADHD (unmedicated) scored significantly worse than control children on the “complex subtraction” task (mean scores [SD]: ADHD=2.55 [2.11]; Controls=4.32 [1.96]) Mann-Whitney, p=0.007).

At time 2, children with ADHD (medicated) were more accurate on the complex subtraction task relative to themselves at time 1 (mean scores [SD]: ADHD time 1=2.55 [2.11]; ADHD time 2=4.05 [2.24]) Wilcoxon, p<0.001). Furthermore, their performance was similar to that of control participants (mean scores [SD]: ADHD=4.05 [2.24]; Control=4.91 [1.66]) Mann-Whitney, p=0.263).

Simple subtraction

The same analyses of the afore mentioned were conducted for the “simple subtraction” task. No differences were found between patients and controls at either time point for this task (mean scores [SD]: ADHD time 1=5.30 [1.69]; Control time 1=5.95 [1.09]; ADHD time 2=5.95 [1.15]; Control time 2=6.00 [1.11]; Mann-Whitney time 1, p=0.237; Mann-Whitney time 2, p=0.915). Moreover, there were no differences in ADHD groups between time 1 and time 2 (Wilcoxon, p=0.068).

DISCUSSION

ADHD and arithmetic difficulties are frequently associated, and these variables were studied here in Brazilian children. The choice of which arithmetic test should be used was the first challenge. There was no guarantee that, by applying each of the three tests widely used in Brazil (Bastos Test, TDE Arithmetic, and WISC-III Arithmetic Subtest), the results aligned with current literature, which had already shown that ADHD children have more arithmetic difficulties than their healthy peers would be found³. This is due to two simple issues: the lack of a “perfect” arithmetic test (no consensus among researchers of the field) and this being the first time in Brazil that these tests were applied collectively to detect the presence of arithmetic difficulties in ADHD children.

The reason for using MPH was to check how aligned the tests were with their known effects, regarding working memory and attention to improve arithmetic skills^6,7,8. The identification of arithmetic difficulties according to each test in the context of MPH treatment for ADHD children was analyzed. Separately, the Bastos Test and TDE Arithmetic Test could identify differences between children with ADHD (without medication) and the control group, which agrees with those of the literature³. However, the WISC-III Arithmetic Subtest could not detect this difference. The results were statistical similar between controls and ADHD children when the ADHD children were medicated. This improvement is probably related to MPH, because none of these children suffered any other interventions, except for the use of MPH. In the ADHD group, the tests did not show better results separately, before and after MPH.

However, by combining these three tests the following results were found: there was a worse performance in unmedicated ADHD children than in the control group; an improved performance was associated to the MPH treatment, and no significant difference was seen between medicated ADHD children and the control group.

These results could be explained by the fact that none of the tests individually fulfill McCloskey’s principles¹⁰ to study arithmetic knowledge: number comprehension, number production and calculation processing. However, combining these three tests could detect the presence of arithmetic difficulties and expands the sensitivity of the process.

In terms of mathematical cognition domain, complex subtractions were explored based on different authors statements, who say that working memory and attention are part of subtraction skills that involve the borrowing procedure^4,5. Thus, combining the complex subtractions of Bastos and TDE Arithmetic tests provided similar results to the combination of all three tests. Simple subtraction could not identify any differences between groups. Furthermore, as found by Rubinsten et al.⁵, MPH had no effects on simple subtraction performance, probably because this is a basic processing skill, that relies on the number sense, which is an inherited ability¹⁷. The improved complex subtraction performance could be attributed to the MPH effects on executive functions, such as working memory. It may be possible that this discrepancy regarding simple and complex subtractions in children with ADHD suggests that basic arithmetic difficulties need to be addressed with specific educational interventions; those based on working memory and attention might benefit from medication^5,18.

The study design is controlled for possible experimental placebo-related effects (task learning and research team interaction effects) and for time-related effects (two time-points in both groups). However, possible effects of MPH’s medication were not controlled with a specific placebo-medication. If a medication-wise placebo arm was used in both groups, the effects could also be present in the control group. Moreover, the experiment was not double-blinded. However, it is expected that these characteristics would affect the tests separately and their combination similarly. The placebo effects would also probably increase the differences between time 1 and time 2 in the ADHD group. Nonetheless, given that MPH benefits are well established, and only the tests in combination detected these improvements in the ADHD group, we find that these results were not altered by placebo effects. Another possible aspect is that the study sample size is relatively small. Even a small sample could provide strong evidence aligned with literature as seen with our subtraction data. These results were quite similar to that of Rubinsten et al.⁵. We reiterate that we are not trying to validate a new test but rather suggesting that combining tests could increase sensitivity, which could be especially useful in studies with smaller samples.

In conclusion, instead of creating a new test, the use of existing tests was recycled to offer a more complete assessment of arithmetical difficulties when applied in combination. When using this combination of validated tests, arithmetic difficulties could be detected in Brazilian children with ADHD, besides demonstrating the benefits of methylphenidate. Currently, the authors are working on a neuroimaging version of this protocol to explore the neural underpinnings of ADHD and arithmetic disorders. Recycling existing tools might be a way to align the different world groups that research arithmetic disorders and establish a consensual criterion to detect the presence of arithmetic difficulties.

References

• ¹
  Barkley RA. Attention-deficit Hyperactivity Disorder. New York: The Guilford Press; 2006.
• ²
  Monuteaux MC, Faraone SV, Herzig K, Navsaria N, Biederman J. ADHD and dyscalculia: evidence for independent familial transmission. J Learn Disabil. 2005 Jan-Feb;38(1):86-93. https://doi.org/10.1177/00222194050380010701
  » https://doi.org/10.1177/00222194050380010701
• ³
  Shalev RS. Developmental dyscalculia. J Child Neurol. 2004 Oct;19(10):765-71. https://doi.org/10.1177/08830738040190100601
  » https://doi.org/10.1177/08830738040190100601
• ⁴
  Geary DC. Mathematical disabilities: cognitive, neuropsychological, and genetic components. Psychol Bull. 1993;114(2):345-62. https://doi.org/10.1037/0033-2909.114.2.345
  » https://doi.org/10.1037/0033-2909.114.2.345
• ⁵
  Rubinsten O, Bedard AC, Tannock R. Methylphenidate has differential effects on numerical abilities in ADHD children with and without co-morbid mathematical difficulties. Open Psychol J. 2008;1(1):11-7. https://doi.org/10.2174/1874350100801010011
  » https://doi.org/10.2174/1874350100801010011
• ⁶
  Benedetto-Nasho E, Tannock R. Math computation, error patterns and stimulant effects in children with Attention Deficit Hyperactivity Disorder. J Atten Dis. 1999 Oct;3(3):121-34. https://doi.org/10.1177/108705479900300301
  » https://doi.org/10.1177/108705479900300301
• ⁷
  Grizenko N, Cai E, Jolicoeur C, Ter-Stepanian M, Joober R. Effects of methylphenidate on acute math performance in children with attention-deficit hyperactivity disorder. Can J Psychiatry. 2013;58(11):632-9. https://doi.org/10.1177/070674371305801109
  » https://doi.org/10.1177/070674371305801109
• ⁸
  Pelham WE, Bender ME, Caddell J, Booth S, Moorer SH. Methylphenidate and children with attention deficit disorder. Dose effects on classroom academic and social behavior. Arch Genl Psychiatry. 1985;42(10):948-52. https://doi.org/10.1001/archpsyc.1985.01790330028003
  » https://doi.org/10.1001/archpsyc.1985.01790330028003
• ⁹
  Landerl K, Bevan A, Butterworth B. Developmental dyscalculia and basic numerical capacities: a study of 8-9-year-old students. Cognition. 2004 Sep;93(2):99-125. https://doi.org/10.1016/j.cognition.2003.11.004
  » https://doi.org/10.1016/j.cognition.2003.11.004
• ¹⁰
  McCloskey M, Caramazza A, Basili A. Cognitive mechanisms in number processing and calculation: Evidence from dyscalculia. Brain Cognition. 1985 Apr;4(2):171-96. https://doi.org/10.1016/0278-2626(85)90069-7
  » https://doi.org/10.1016/0278-2626(85)90069-7
• ¹¹
  Mattos P, Serra-Pinheiro MA, Rohde LA, Pinto D. Apresentação de uma versão em português para uso no Brasil do instrumento MTA-SNAP-IV de avaliação de sintomas de transtorno do déficit de atenção/hiperatividade e sintomas de transtorno desafiador e de oposição. Rev Psiquiatr Rio Gd Sul. 2006;28(3):290-7. http://dx.doi.org/10.1590/S0101-81082006000300008
  » http://dx.doi.org/10.1590/S0101-81082006000300008
• ¹²
  Benczik EBP. Manual da Escala de Transtorno de Déficit de Atenção/Hiperatividade. 1. ed. São Paulo: Casa do Psicólogo; 2000. p.64.
• ¹³
  Bastos JA. Evaluation of the children's mathematics abilities from final elementary cycles of public and private schools in São José do Rio Preto SP, Brazil. Arq Neuropsiquiatr. 2003;61(3A):699-9. http://dx.doi.org/10.1590/S0004-282X2003000400034
  » http://dx.doi.org/10.1590/S0004-282X2003000400034
• ¹⁴
  Stein LM. TDE: teste de desempenho escolar: manual para aplicação e interpretação. São Paulo: Casa do Psicólogo ; 1994. p.35.
• ¹⁵
  Wechsler D. Wechsler Intelligence Scale for Children - Third Edition - WISC- III. San Antonio, TX: The Psychological Corporation; 1991. p.294.
• ¹⁶
  Strauss E, Sherman EMS, Spreen O. A compendium of neuropsychological tests. New York: Oxford University Press; 2006.
• ¹⁷
  Dehaene S, Molko N, Cohen L, Wilson AJ. Arithmetic and the brain. Curr Opin Neurobiol. 2004;14(2):218-24. https://doi.org/10.1016/j.conb.2004.03.008
  » https://doi.org/10.1016/j.conb.2004.03.008
• ¹⁸
  Geary DC. Consequences, characteristics, and causes of mathematical learning disabilities and persistent low achievement in mathematics. J Dev Behav Pediatr. 2011;32(3):250-63. https://doi.org/10.1097/DBP.0b013e318209edef
  » https://doi.org/10.1097/DBP.0b013e318209edef

Publication Dates

• Publication in this collection
  02 Mar 2020
• Date of issue
  Apr 2020

History

• Reviewed
  30 Jan 2019
• Received
  27 Aug 2019
• Accepted
  22 Oct 2019