Computerized cognitive training in children and adolescents with attention deficit/hyperactivity disorder as add-on treatment to stimulants: feasibility study and protocol description

Treinamento cognitivo para crianças e adolescentes com transtorno de déficit de atenção/hiperatividade como tratamento complementar aos psicoestimulantes: estudo de viabilidade e descrição de protocolo

Abstract

Background

Cognitive training has received increasing attention as a non-pharmacological approach for the treatment of attention deficit/hyperactivity disorder (ADHD) in children and adolescents. Few studies have assessed cognitive training as add-on treatment to medication in randomized placebo controlled trials. The purpose of this preliminary study was to explore the feasibility of implementing a computerized cognitive training program for ADHD in our environment, describe its main characteristics and potential efficacy in a small pilot study.

Methods

Six ADHD patients aged 10-12-years old receiving stimulants and presenting residual symptoms were enrolled in a randomized clinical trial to either a standard cognitive training program or a controlled placebo condition for 12 weeks. The primary outcome was core ADHD symptoms measured using the Swanson, Nolan and Pelham Questionnaire (SNAP-IV scale).

Results

We faced higher resistance than expected to patient enrollment due to logistic issues to attend face-to-face sessions in the hospital and to fill the requirement of medication status and absence of some comorbidities. Both groups showed decrease in parent reported ADHD symptoms without statistical difference between them. In addition, improvements on neuropsychological tests were observed in both groups – mainly on trained tasks.

Conclusions

This protocol revealed the need for new strategies to better assess the effectiveness of cognitive training such as the need to implement the intervention in a school environment to have an assessment with more external validity. Given the small sample size of this pilot study, definitive conclusions on the effects of cognitive training as add-on treatment to stimulants would be premature.

Attention deficit/hyperactivity disorder (ADHD); treatment; cognitive training

Resumo

Introdução

O treinamento cognitivo tem recebido atenção especial como abordagem não medicamentosa para o tratamento do transtorno de déficit de atenção/hiperatividade (TDAH) em crianças e adolescentes. Poucos estudos avaliaram o treinamento cognitivo como abordagem complementar à medicação em ensaios clínicos randomizados controlados por placebo. O objetivo deste estudo foi explorar a viabilidade para a implementação de um programa de treinamento cognitivo computadorizado, descrever suas características principais e potencial eficácia em um pequeno estudo piloto.

Métodos

Seis pacientes com TDAH entre 10-12 anos de idade, em uso de psicoestimulantes e apresentando sintomas residuais, foram recrutados e randomizados para um dos dois grupos (treinamento cognitivo ou placebo) por 12 semanas. O desfecho principal foram os sintomas nucleares do TDAH avaliados através do Questionário de Swanson, Nolan e Pelham (SNAP-IV).

Resultados

Encontramos maior resistência do que a esperada no recrutamento dos pacientes em função de problemas logísticos para atender às sessões presenciais no hospital assim como para preencherem os critérios de status medicamentoso e ausência de algumas comorbidades. Ambos os grupos apresentaram diminuição nos escores dos sintomas de TDAH reportados pelos pais, mas sem diferença estatística entre eles. Além disso, foi observada melhora nos testes neuropsicológicos em ambos os grupos – principalmente nas tarefas treinadas pelo programa.

Conclusão

Este protocolo revelou a necessidade de novas estratégias para melhor avaliar a eficácia do treinamento cognitivo tal como a necessidade de implementar a intervenção no ambiente escolar a fim de obter uma avaliação com maior validade externa. Devido ao pequeno tamanho amostral deste estudo, conclusões definitivas sobre os efeitos do treinamento cognitivo como abordagem complementar aos psicoestimulantes seriam prematuras.

Transtorno de déficit de atenção/hiperatividade (TDAH); tratamento; treinamento cognitivo

Introduction

Attention deficit/hyperactivity disorder (ADHD) is one of the most prevalent externalizing disorders in children and adolescents,¹with an estimated prevalence of 5.29%.²The core symptoms are inattention, hyperactivity and impulsivity that are age inappropriate, persistent and pervasive.³The presence of clinical symptoms and neurocognitive deficits in ADHD is consistently associated with morbidity⁴and impairments like poor academic performance and consequent school dropout,⁵higher risk to nicotine and cocaine addiction, automobile accidents⁶and criminality.⁷Evidence suggests that these outcomes could be avoided with ADHD treatment.⁸

Previous literature suggests that most of the patients with ADHD show deficits on multiple executive domains. Executive functions allow individuals to regulate their behavior, thoughts, emotions and self-control.⁹Deficits in executive functions might be one of the core symptoms of ADHD and probably explain part of the daily life problems found in children with this disorder.¹⁰

The use of medication, especially stimulants, is one of the most effective treatments for ADHD.¹¹However, about 30% of the patients do not respond to stimulant medication or do not tolerate the side effects.¹²In addition, ADHD cognitive symptoms usually do not fully improve and the long-term benefits of medication are still unknown.¹³

Among several non-pharmacological approaches available to treat ADHD,⁵cognitive training has received increasing attention. Researches demonstrate that cognitive training promotes improvement of symptoms manifested at home and at school.¹⁴It has been suggested that these programs are effective as ADHD treatment or as a tool to improve cognitive ability and academic performance in all age groups.¹⁵Training involves repetition of specific or multiple cognitive processes over several weeks after which period performance enhancement is expected on the trained tasks.¹⁶Working memory is often the target of cognitive training because of its assumed capacity to influence a range of other cognitive processes. Working memory deficits have also been associated with failure in academic performance.¹⁷Therefore, the training of executive functions can be a potential strategy for ADHD treatment.¹⁸Assessment of the literature suggests that there are methodological limitations (i.e., lack of blinded assessments; lack of an adequate control group)¹⁹,²⁰that indicate that current results must be further investigated and replicated.⁵

Our objective was to assess, in a randomized clinical trial (RCT), the benefits of a cognitive training program as an add-on treatment to stimulants in children and adolescents with ADHD. In this pilot study conducted before the RCT, we mainly aimed to: a) describe our procedures to enhance reproducibility; b) test the feasibility of the protocol, by assessing: b.1) recruitment potential and adherence rates; b.2) eligibility criteria; b.3) the equipment (software) used during the training; and b.4) if our functional magnetic resonance imaging (fMRI) paradigms are ready to assess our data and carefully appraise if there are any important data forgotten; and c) describe some very preliminary findings.

Method

The study was approved by the Ethics and Research Committee of Hospital de Clínicas de Porto Alegre (HCPA; protocol no. CAAE 25048913.8.0000.5327). At study intake, parents and children were informed about randomization to one of the two computerized programs; parent consent and child assent were obtained before the initial assessment with the signature of a free and informed consent form approved by the committee. There was no monetary compensation for participating in study procedures.

Study design

This was a pilot study for a bigger randomized clinical trial that aims to compare a standard cognitive training program to placebo offered as add-on treatments to ADHD patients medicated with stimulants and who present with residual symptoms.

Recruitment and enrollment

Participants

Participants were recruited from the Attention Deficit/Hyperactivity Disorder Outpatient Program (ProDAH) from April to September 2014. ProDAH, at the teaching hospital of Universidade Federal do Rio Grande do Sul (HCPA), is an area for teaching, research and clinical work with patients suffering from ADHD disorder. Its pediatric branch is linked to the Child and Adolescent Psychiatric Division at HCPA. Since we have complete data on clinical aspects and response to treatment for patients in our unit, we assessed our data set searching subjects with our established inclusion criteria. They were invited to participate in the trial during their regular attendance at ProDAH. At this moment, the study protocol was explained to parents and participants to assess their interest and the feasibility of maintaining regular face-to-face training sessions. When the invitation was accepted, parents and children were forwarded to a psychiatrist for assessment and thereafter to a neuropsychologist.

Randomization

Simple randomization was performed using random numbers (representing the two study groups – active intervention or placebo). After randomization, there were four experimental cases and two controls.

Eligibility criteria

Our inclusion criteria were: a) age range 6-13 years-old; b) clinical diagnosis of ADHD according to the Diagnostic and Statistical Manual of Mental Disorders, 5th edition (DSM-5); c) patients medicated with stimulants (at least 3 months of medication with doses ≥ 0.3 mg/kg/day of methylphenidate or 30 mg/d of lisdexamfetamine; subjects were also asked about the possibility of not changing their treatments during the cognitive training trial); d) presence of residual symptoms of ADHD despite medication (we considered at least 50% residual symptomatology according to scores obtained on the Swanson, Nolan and Pelham Questionnaire [SNAP-IV scale]); and e) internet access at home. Exclusion criteria were: a) evidence of a clinically significant comorbid psychiatric disorder requiring any additional treatment; b) an estimated intelligence quotient (IQ) score < 80 (scores were determined using block design and vocabulary subtests from Wechsler Intelligence Scale – Third or Fourth edition, depending on the period that the child was assessed).²¹

Study settings

Face-to-face sessions were performed at HCPA. A dedicated area equipped with computers, earphones and access to high-speed internet was created. We carefully placed the participants in the room so as to avoid interactions between them. During each session, they received individual assistance from a learning tutor or a member of the staff. The active and placebo group sessions were scheduled at different times.

Intervention conditions

Active cognitive training – Computerized Cognitive Remediation Training (CCRT) (ACTIVATE™).This software was created by C8 Sciences based on research from Yale University. A Brazilian Portuguese version of the software was used for this clinical trial. CCRT comprises six different games that target neurocognitive functions, such as working memory, speed processing, sustained and divided attention, category formation and control inhibition. During the training, participants perform a wide range of cognitive tasks like memorizing sequences, completing patterns, task-switching and assigning objects into categories. Each session lasts 45 minutes and the proposed treatment length was four sessions per week over a 12-week period. The sessions were carried out after school. Adequate implementation of the cognitive training program was defined as successful completion of 85% of the sessions.

The program presents four innovative procedures:

1. Automatic individualization of treatment, with graduation and plateau criteria. The games move participants quickly through exercises in areas of their strength and keep them working longer in areas of their weaknesses. Also, they avoid keeping participants working in exercises for too long after their maximum gain has been reached. To address a wide range of cognitive deficits, the training has multiple exercises that focus on different aspects of cognition.

2. Online error diagnostics. Despite the records and evaluation of the subjects’ responses during the training sessions, the program recognizes different types of errors an individual makes, which could provide important information to teachers and clinicians.

3. Online corrective strategy messaging. Every time a child makes a specific type of error above a criterion frequency, the program automatically provides a corrective strategy message and an option for doing the problem correctly.

4. Attention alerts. The program has an attention alert function that helps to increase child performance during the exercises.

On the C8 Sciences website it is possible to find an example of the games and/or request a demo version: http://www.c8sciences.com/about/games/.

Placebo cognitive training. A package composed of educational videos and questions related to school content was developed by a learning tutor and psychologists from our staff; the training package considered the academic level of the participants and was hosted on an online platform (Moodle) at the hospital. Four strata were created according to the age of participants – 6-7, 8-9, 10-11 and 12-13 years old – and to expected performance for level of schooling. The questions were selected by a team of learning tutors and the videos were chosen by two psychologists. The placebo intervention was created to avoid any kind of cognitive training; it was offered in identical conditions as the CCRT. The content of this platform can be visualized on https://www.youtube.com/watch?v=dAv6Y83BDqc – subtitles in English can be triggered at the bottom of the video as indicated.

Procedure

Study procedures were explained to all participants and at least one parent. Participants were randomly assigned to the different treatment conditions (CCRT, n = 4; placebo, n = 2; see Figure 1for CONSORT diagram) using a macro in Microsoft® Excel. All participants and their families received a schedule for implementing the intervention at home and the dates of face-to-face sessions – two face-to-face sessions and two home sessions per week were expected as well as weekly coaching calls. A member of the staff made phone contact to detect any kind of difficulties with the platform and ensure compliance with the training.

Figure 1
CONSORT diagram showing patient recruitment and enrollment. fMRI = functional magnetic resonance imaging.

The training sessions were followed by a control register platform to identify potential challenges to treatment compliance. Participants in both conditions received equal support. Home sessions were completed under parental supervision and weekly coaching phone calls were held to remember the day sessions, check adherence to the protocol and medication, troubleshoot problems and provide motivational encouragement. Performance data were checked regularly via the online platform to verify progress and identify eventual difficulties and noncompliance. Data about medication status and adherence to the treatment were collected. Post-treatment assessments and rating scales were completed approximately 1-2 weeks after the final training day for each participant. The assessments were carried-out by a researcher who was blind to participant treatment group.

Outcome measures

Clinical outcome

Parent reports of ADHD symptoms were assessed using the SNAP-IV rating scale – a well-known instrument used in ADHD clinical trials.²²Questions were filled out by parents and the principal investigator, who was a trained child and adolescent psychiatrist blinded to treatment condition. The primary outcome measure was the difference in SNAP-IV scores collected at baseline vs. endpoint (inattention, hyperactivity/impulsivity and total score).

Neurocognitive outcome

All subjects were assessed using two neuropsychological batteries pre- and post-intervention, conducted in two different sessions which lasted approximately 90 minutes. The post-intervention assessment occurred immediately after treatment. The first battery included six neuropsychological tests: 1) Conner’s Continuous Performance Test II (CPT II)²³to assess response inhibition (through number of commissions) and sustained attention (through Hit Reaction Time Block Change – higher values indicate a slowing in reaction time as the test progresses); 2) Digit Span to assess verbal working memory (number of correct responses for backward condition)²¹; 3) Spatial Span to assess visual working memory (number of correct responses for backward condition) – the Spatial Span task was designed based on the Cambridge Neuropsychological Testing Automated Battery (CANTAB) Corsi Block Task²⁴; 4) Two Choice Reaction Time Task to assess speed of processing (mean reaction time in ms); 5) Trail Making Test to assess cognitive flexibility (time in seconds to complete part B)²⁵; 6) Picture Concepts to assess category formation and pattern recognition (number of correct responses).²¹

The second battery used was the NIH toolbox (www.nihtoolbox.org/), which includes: Flanker Test to assess control inhibition and attention; Go/NoGo task to assess control inhibition, cognitive flexibility and speed processing; and List Sorting Working Memory Test to assess working memory. These tests were selected for both their proven construct validity as well as their frequent use in ADHD clinical evaluation and research. We also chose tests that included the different cognitive functions involved in the cognitive training program, namely, working memory, processing speed, divided and sustained attention, category formation and control inhibition. Tasks were administered in different sessions by a trained neuropsychologist blind to treatment intervention; all subjects were instructed to take their stimulant medication 1 hour before the assessment. The outcome measure was the difference between baseline and endpoint scores.

Neuroimaging outcomes

Four participants were submitted to fMRI exams to assess the effects of cognitive training in brain areas (fronto-striatal and parietal areas). The tasks were developed based on literature experiments and with the assistance of the research team from the Brain Institute of Rio Grande do Sul. Before the fMRI, all tasks were explained to and practiced by participants on a laptop outside of the scanner. Subjects were instructed to take their medication 1 hour prior to the scan. During the fMRI exam, participants performed a neuropsychological battery of tests. The tasks were projected onto a screen and viewed by subjects through a prism mirror attached to the scanner’s headcoil cage. Functional images were acquired with a 3.0T Signa HDxt scanner (GE Healthcare, Milwaukee, WI, USA) with the following sequence parameters: time repetition (TR) 2 s, time echo (TE) 30 ms, field of view (FOV) 220 x 220 mm, matrix size 64x64, slice thickened 3.6 mm. The fMRI protocol comprised:

1. An isotropic T1 structural image.

2. Resting state fMRI scan.

3. Working memory task – N-Back,²⁶,²⁷64 trials (16 trials 0-Back letters; 16 trials 0-Back figures; 16 trials 1-Back letters; 16 trials 1-Back figures), a stimulus of 2 s and an interstimulus interval of 1 s. This working memory task consists of two conditions. During the 0-back condition, subjects have to respond any time the target (letter) presented in the beginning of the test appears on the screen. During the 1-back condition, subjects are presented with a series of letters and figures and have to respond whenever the stimulus presented is identical to the stimulus before it.

4. Conflict Control Task²⁸– 200 trials (150 congruent trials and 50 incongruent trials), a stimulus of 1.5 s and an interstimulus interval of 1.5 s. In congruent trials, green arrows appear on the screen pointing left or right and the subject is instructed to press the button in the same direction of the arrow (buttons in right and left hands); in incongruent trials, red arrows appear on the screen pointing left or right and the subject is instructed to press the button in the opposite direction of the arrow.

5. Go/NoGo²⁸– 200 trials (150 Go and 50 NoGo), a stimulus of 1.5 s and interstimulus interval of 1.5 s. Arrows are presented pointing either left or right (representing Go signals); at each arrow presentation, participants have to press a button. Arrows pointing up represented No-Go signals. A button response has to be selectively executed with the right thumb to Go stimuli or inhibited to No-Go signals.

The order of presentation of the three different tasks was randomly assigned for each participant. All images were analyzed on the Analysis of Functional NeuroImages (AFNI) software (http://afni.nimh.nih.gov/), using the automated preprocessing pipeline (afni_proc.py).

Feasibility

We assessed each step as feasible using specific criteria. Compliance with the intervention (adherence) was defined as completing ≥ 41 of the 48 training sessions (85%) within a 12-week period. Another important issue was to assess if our inclusion criteria were too restrictive – we evaluated it considering how many subjects were excluded due to comorbidities or not meeting the criteria for medication status. The training software was assessed during the sessions – we were concerned if the platform was running normally, if the internet was sufficient to support its use, and if there were any kind of bugs. To test our fMRI paradigms, we assessed the images observing activation maps and running a statistical analysis; in addition, we carefully evaluated movements during the exam.

Data analyses

Analyses were performed with the Statistical Package for the Social Sciences (SPSS) version 22.0. Results were presented as mean ± standard deviation (SD). We also performed a t-test to compare the means, considering a p value ≤ 0.05.

Results

Sample characteristics

Table 1presents the demographic and clinical characteristics of the study sample. A total of six subjects fulfilled the inclusion criteria and consented participation in the study. The study sample included children aged 10 to 12 years. Mean age was 10.83 years (SD 0.75); 66.7% of the participants were male.

Table 1
Sociodemographic characteristics

Subject	Age (y)	Gender	IQ	SES	ADHD diagnoses	Group	Comorbidity	% sessions completed	SNAP-IV baseline			SNAP-IV endpoint			Medication status (mg/kg/day)
									Inatt	H/I	T	Inatt	H/I	T
1	10	M	90	C	C-ADHD	Case	ODD	-	2.44	2.11	2.27	-	-	-	0.59
2	11	F	95	B	C-ADHD	Control	-	93.75	2.55	2	2.27	1.33	1.55	1.44	0.98
3	11	M	80	C	C-ADHD	Case	-	95.8	2.55	2.22	2.38	1.33	1.33	1.33	0.71
4	12	M	87.5	B	C-ADHD	Case	ODD, LD	66	1.77	0.33	1.05	1.44	0.66	1.05	0.7 (LDX)
5	10	F	102.5	C	C-ADHD	Control	-	77	1.88	1.33	1.6	1.22	1	1.11	0.48
6	11	M	87.5	-	I-ADHD	Case	-	91.66	2	0.66	1.33	1.33	1	1.17	0.32

• C-ADHD = combined ADHD subtype; I-ADHD = inattentive ADHD subtype; H/I = hyperactive/impulsive score; IQ = intelligence quotient; Inatt = inattention; LD = learning disability; LDX = lisdexamfetamine; ODD = oppositional-defiant disorder; SES = socioeconomic status (ABEP – http://www.abep.org/criterio-brasil); T = total score.

Clinical outcomes

We analyzed data for the completers. In general, participants in both groups showed a decrease in their scores for different domains of the parental SNAP-IV scale during the protocol (Table 1). The mean inattention and total SNAP-IV scores were, respectively, 2.19 (0.36) / 1.74 (0.68) for cases and 2.21 (0.47) / 1.93 (0.47) for controls at the beginning of the study and 1.36 (0.06) / 1.18 (0.14) for cases and 1.27 (0.07) / 1.27 (0.23) for controls after the intervention. As expected, due to the small sample size, there were no statistically significant differences in pre- and post-intervention clinical measures.

Neurocognitive functioning outcomes

Tables 2, 3and 4show results from our neuropsychological assessment and from the NIH toolbox, respectively. Pre- and post-intervention scores for each individual are presented. Again, the population sample was too small to establish statistical significance. In addition, due to the wide age range, the scores should have been corrected for age. Pre- and post-intervention results are presented individually for each subject. Higher values indicate better performance for Picture Concepts, Digit Span Backwards, Spatial Span Backwards, No-Go Accuracy and Working Memory Total Score.

Table 2
Neuropsychological assessments

Subject	Group	Picture Concepts*		Digit Span Backward*		Spatial Span Backward*		CPT II Commissions		CPT II RT Block Change (time in ms)		Trail Making Test B (time in seconds)		Two Choice RT (time in ms)
		Pre	Post	Pre	Post	Pre	Post	Pre	Post	Pre	Post	Pre	Post	Pre	Post
1†	Case	11	-	6	-	0	-	28	-	0.00	-	38	-	289.38	-
2	Control	13	16	8	6	3	3	20	22	0.03	0.00	39	32	300.05	301.56
3	Case	15	19	6	5	0	3	27	-	0.01	-	42	92	433.46	127.29
4	Case	19	21	9	7	5	1	21	31	-0.03	0.02	35	83	363.55	-
5	Control	17	18	7	8	3	5	32	25	0.03	-0.01	43	44	283.98	356.22
6	Case	14	17	7	5	2	4	18	15	-0.01	0.02	58	60	394.26	444.13

• All measures are presented in raw scores.
• CPT II = Continuous Performance Test II; RT: Reaction Time;
• * Higher values indicate better performance.
• ^†Subject 1 dropped out, therefore only pre-intervention results are available.

Table 3
Neuropsychological assessments: NIH toolbox, within subject analysis

Subject*	Group	Flanker Correct Incongruent RT	No-Go Accuracy	WM Total score
		Pre/Post	Pre/Post	Pre/Post
2	Control	501.85 / 571.46 ms	0.733 / 0.366	20 / 10
3	Case	869.9 / 677.82 ms	0.266 / 0.266	2 / 16
4	Case	660.35 / 583.43 ms	0.6 / 0.6	3 / 14
5	Control	684.82 / 498.66 ms	0.46 / 0.36	9 / 2
6	Case	554.11 / 608.82 ms	0.266 / 0.833	6 / 12

• RT = reaction time; WM = working memory.
• * Subject 1 dropped out.

Table 4
Neuropsychological assessments: NIH toolbox, active vs. placebo condition

	Flanker Incongruent Accuracy Pre	Flanker Incongruent Accuracy Post	Flanker Correct Incongruent RT Pre	Flanker Correct Incongruent RT Post	No-Go Accuracy Pre	No-Go Accuracy Post	WM Pre	WM Post
Placebo	0.91	0.88	593 ms	535 ms	0.47	0.37	14.5	6
Active	0.86	0.98	695 ms	623 ms	0.38	0.57	4	14

• RT = reaction time; WM = working memory.

Neuroimaging

After the pre-intervention scans, it was possible to identify some limitations involving our fMRI paradigms: one of the limitations was the need to improve the Go/No-Go task (with a jitterinclusion); the Conflict Control task did not generate a good activation map; among others. Due to the limitations in the results obtained with fMRI scans, post-intervention scans were not performed. It was decided that pre- and post-intervention scans would be carried out in a larger, subsequent study.

Compliance with training

Among the four participants assigned to CCRT and two assigned to placebo, 50% of each group met compliance criteria. Case 1, male, 10 years, member of the active group, dropped out during the first weeks of the protocol due to logistic issues – difficulty attending the face-to-face sessions at the hospital due to schedule problems.

Feasibility analyses

We faced some problems with the adherence of one participant to the face-to-face sessions at the hospital due to difficulties involving distance and logistic issues. In two other cases, the participants did not reach the minimum sessions for an adequate cognitive training trial due to internet problems at home. Based on these findings, in order to optimize the cognitive training program, we decided to modify our protocol for the RCT by running the training three times a week at the schools, as an after-school program, and one time at home. We learned that careful assessment of the quality of high-speed internet availability at home would be crucial for developing any session at home.

Regarding the enrollment of participants, there were difficulties to find patients that fit the inclusion criteria. In this pilot phase, it was requested that participants should be on stimulant regimen for a minimum of 3 months, and with a stable dose. Taking into account that most of the participants interrupted their treatment during school holidays and considering that stimulants present rapid onset of action, it was decided that the criterion would change to one month of stable dose to facilitate the allocation of participants. Similarly, it was decided that subjects with some comorbidities would be accepted, like oppositional defiant disorder, anxiety disorders, tic disorders, enuresis, and those using other psychotropic medications; however, the request that patients were clinically stable and without changes in drug regimen in the previous month before the protocol were maintained. This changes in prerequisites are in line with the issue of external validity, as it is known that comorbidities are found in the majority of patients with ADHD.¹,⁴,²⁹,³⁰The criterion regarding the clinical outcome was also adapted: instead of considering a threshold of at least 50% of residual symptoms, which made recruitment of patients a difficult task, an average score in the SNAP-IV inattention dimension by parents and teachers ≥ 1 was adopted. A teacher-rated SNAP-IV score was also included to assess outcome. Considering that the training focuses mainly on cognitive domains like working memory, attention and speed processing, it was decided that the inattention scores would be a better parameter to assess the main outcome

The present findings contributed to the understanding of which adjustment was necessary to improve the protocol in order to test cognitive training as an add-on treatment approach to ADHD. An ongoing randomized controlled trial will generate further evidence concerning this cognitive training program. The new protocol for the current study may be found at clinicaltrials.gov.

Discussion

The purpose of this preliminary study was to describe the procedures employed in a protocol for investigating the benefits of CCRT compared to a placebo intervention in an ADHD sample and to assess the feasibility of the strategy proposed. In addition, some very preliminary findings are discussed. As far as we know, there is a scarcity of published studies exploring the CCRT as an add-on intervention for ADHD stimulant treated subjects.

Regarding the design of the study, we believe that the add-on design of this protocol could be very interesting since different profiles of treatment – in this case stimulants plus cognitive training – can be used together to cover a greater magnitude of symptoms and therefore reaching the goal of improving the psychiatric illness. Reflecting the real-world situation, the use of concomitant psychotropic medications and adjuvant therapy approaches has substantially increased. Evidence-based guidelines to ADHD mostly recommend treatments that include pharmacological and psychological interventions.³¹Working memory elements can be modified by both therapeutic interventions for ADHD: working memory training and psychostimulants. Moreover, it has been suggested that cognitive training could lead to greater enhancements in working memory elements than medication alone.³²We believe, in this study, that the use of medication during training could enhance the benefits of the cognitive training program.

Across the literature, cognitive training appears to have effects on certain aspects of working memory domains, as already showed by randomized clinical trials.¹⁹,²⁰,³³Nevertheless, the effectiveness of cognitive training on some cognitive and clinical symptoms of ADHD has been questioned in several meta-analyses¹⁵,³⁴and reviews,³⁵,³⁶and those results should be carefully interpreted, as many studies present methodological limitations (i.e., lack of an adequate control group).³⁶Moreover, in terms of potential variable influence, it has been suggested that working memory training could be superiorly effective in visual working memory,³⁷in children with learning disabilities³⁶and in individuals with low-performing cognitive ability, as there is more room for improvement.³⁸However, a recent study that evaluated the effect of a computerized adaptive working memory intervention program on improving long-term academic outcomes in children with low working memory did not find any benefits except for visuospatial short-term memory.³⁹

Similarly to other feasibility studies, and according to unpublished observations, we faced difficulties while enrolling participants because of the frequency of the sessions and the long time of follow-up (12 weeks). Furthermore, the fact that the subjects must be on psychostimulant treatment for at least 3 months before entering the study reduced our possibility to reach an optimal sample size. Nevertheless, the difficulties in allocating subjects helped us to rethink our inclusion/exclusion criteria, and some logistic arrangements were made for the ongoing study (see below). Also, we had one drop-out and two participants that did not reach a minimum of training sessions for the protocol to be considered adequate, due to logistical barriers for the families. In this sense, a study that has assessed a protocol of computer-based attention training held at schools showed that it is feasible to implement this kind of treatment approach in a school setting as well as to support the inclusion of a large and more diverse sample.⁴⁰We believe that implementing our protocol at schools could be a good strategy to ensure adherence to the intervention.

Below we list some strategies proposed to deal with the problems found:

1. Intervention: a) we started a partnership with private and public schools in Porto Alegre in order to implement a substantial part of the cognitive program in this environment, enhancing the acceptability of the approach by parents and participants; b) modification to the protocol: each session of the cognitive training or placebo training was shortened (they are now 30 minutes long and occur three times at school/hospital and one time at home – in some particular cases, four times at school, for participants with internet or computer access difficulties). In addition, C8 Sciences has developed a more inviting and interactive layout platform to optimize adherence to games.

2. Outcomes: a) we added an internet scale⁴¹and questions about the time spent with videogames and internet to assess potential adverse events of the cognitive training, e.g., increasing internet/game addiction; b) we included the Clinical Global Impression (CGI)/Children´s Global Assessment Scale (CGAS) to increase our coverage of clinical improvement; c) we added a teacher-rated SNAP-IV score; d) we decided to change the inclusion criteria regarding the domain and intensity of residual symptoms (a mean parent + teacher inattention SNAP-IV score ≥ 1); previous literature suggests that the training could be mainly effective in inattention symptoms⁴²; similarly, we decided to include children on medication when type and dosage were unchanged for at least 4 weeks prior to the start and during the intervention period⁴³; e) regarding the neuropsychological battery, a divided attention task – an unpublished Brazilian Portuguese version of the Test of Everyday Attention for Children (TEA-Ch)⁴⁴– was added to the protocol to better assess the magnitude of cognitive training in this domain.

3. Randomization: in order to produce more comparable groups and reduce the source of bias in treatment assignments, we decided to include a minimization method to allocate the participants. To ensure this, we used the QMinim service (freeware minimization program available at http://qminim.sourceforge.net/demo/index.php). The aim of this method is to minimize the imbalance between the number of patients in each treatment group over a number of factors (we chose age, gender and socioeconomic status). The randomization process will be carried out by an external member of the research team, according to Cochrane guidelines.⁴⁵

4. Neuroimaging: to minimize problems involving subject movement inside the machine, we now carry out an initial rehearsal in a mock-scan. Similarly, our tasks suffered some modifications in order to improve the acquisition of activation maps, including a more complex and demanding working memory test (N-Back),⁴⁶a different Go-No/Go task⁴⁷and a sustained attention task.⁴⁸N-Back is a 6-min working memory task consisting of four conditions. During 1-back, 2-back and 3-back conditions, subjects are presented with a series of letters (1 s duration, inter-trial interval 2 s) and must respond with their right thumb using a button box whenever the letter presented is the same as one, two or three before it, respectively. This requires both storage and continuous updating of stimuli being held in the working memory. In the baseline vigilance 0-back condition, subjects must respond to each X that appears on the screen. The task consists of 12 randomized blocks. In Go-No/Go, frequent arrows (160 trials: 76%, 500 ms duration) pointing to either the left or right (Go signals) appear in the middle of the screen with a mean inter-trial interval of 1.8 s (jittered 1.6-2 s). Infrequently, arrows point up (24 trials, 12%, No-Go signals) or are slightly slanted (by 22.5%) arrows (24 trials, 12%, oddball signals). A button response has to be selectively executed with the right thumb to Go or oddball stimuli or inhibited to No-Go signals. Oddball trials control for the low frequency of No-Go trials and thus the oddball attentional capture effect. In the sustained attention task, for 12 minutes, subjects need to respond as quickly as possible to the appearance of a visual timer counting up in ms. The visual timer appears either after short predictable consecutive delays of 0.5 s (in series of 3-5 stimuli) or after unpredictable time delays of 2, 5 or 8 s pseudo-randomly interspersed into the blocks of 3-5 delays of 0.5 s. The long infrequent unpredictable delays place a higher load on sustained attention/vigilance, whereas the short, predictable 0.5 s delays are typically anticipated, placing a higher demand on sensorimotor synchronization.

Our very preliminary results suggesting that cognitive training did not improve parent-rated ADHD symptoms compared to the placebo group concur with previous meta-analyses.⁴²In our study, both the placebo and the active training group improved. This suggests that when a more rigorous control group is employed, no benefits regarding cognitive training approach emerge.

One of the strengths of this pilot intervention is the neuropsychological evaluations, which added objective data for the assessment process. Relative to the placebo condition, participants in the cognitive training group improved performance mainly on the working memory NIH toolbox, which has similarities between subtests and the tasks involved on cognitive training games. Unlike these trained tasks, cognitive training had no differential effect on non-trained outcome measures, as described previously, regardless of some tendency to improvement of visual working memory – this finding has already been shown in prior clinical trials.³⁷

Our main limitation was the small sample size. Thus, any findings regarding efficacy must be considered preliminary. However, it is important to highlight that our main goals describing this protocol were to offer opportunities for replicability of the strategies and procedures and to allow investigators to have an overview of challenges and on how to solve them while implementing a computerized cognitive training program for ADHD in a clinical trial.

Conclusions

Interventions that have the magnitude to improve ADHD symptoms and related executive functions like working memory are extremely important nowadays because of their potential relevant role in enhancing academic performance. Given the small sample size of this pilot study, conclusions on the effects of cognitive training as add-on treatment to stimulants would be premature.

This study sets the stage for our future steps in this research area to more consistently determine whether cognitive training could significantly improve ADHD clinical and neurocognitive symptoms, as well as to establish the impact of the intervention on brain interconnectivity through fMRI assessment.

Acknowledgements

Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul (FAPERGS) sponsored the study (0433-2551/14-0). ProDAH and Fundo de Incentivo à Pesquisa e Eventos - Hospital de Clínicas de Porto Alegre (FIPE-HCPA; CAAE 25048913.8.0000.5327) also partially supported the study. A PhD scholarship and post-doctoral research support were granted by Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq). UFGRS PROBIC-FAPERGS also partially financed the study through a scientific initiation scholarship.

We would like to thank all the families for their participation. We are also grateful to Yale University group (Professor Bruce E. Wexler, Professor James F. Leckman and colleagues) and to C8 Sciences™ for helping and providing us with the software licenses at no cost; to the Brain Institute of Rio Grande do Sul (Professor Alexandre Franco, Professor Augusto Buchweitz and Cristiano Aguzzoli for their support); to Giovanni Salum and Katya Rubia for helping us with neuroimaging issues.

References

• ¹
  Dias TG, Kieling C, Graeff-Martins AS, Moriyama TS, Rohde LA, Polanczyk GV. Developments and challenges in the diagnosis and treatment of ADHD. Rev Bras Psiquiatr. 2013;35:S40-50.
• ²
  Polanczyk G, Lima MS de, Horta BL, Biederman J, Rohde LA. The worldwide prevalence of ADHD: a systematic review and metaregression analysis. Am J Psychiatry. 2007;164:942-8.
• ³
  American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5). Arlington: American Psychiatric Publishing; 2013.
• ⁴
  Biederman J, Petty CR, Woodworth KY, Lomedico A, Hyder LL, Faraone SV. Adult outcome of attention-deficit/hyperactivity disorder: a controlled 16-year follow-up study. J Clin Psychiatry. 2012;73:941-50.
• ⁵
  Sonuga-Barke EJ, Brandeis D, Cortese S, Daley D, Ferrin M, Holtmann M, et al. Nonpharmacological interventions for ADHD: systematic review and meta-analyses of randomized controlled trials of dietary and psychological treatments. Am J Psychiatry. 2013;170:275-89.
• ⁶
  Shaw M, Hodgkins P, Caci H, Young S, Kahle J, Woods AG, et al. A systematic review and analysis of long-term outcomes in attention deficit hyperactivity disorder: effects of treatment and non-treatment. BMC Med. 2012;10:99.
• ⁷
  Danckaerts M, Sonuga-Barke EJ, Banaschewski T, Buitelaar J, Döpfner M, Hollis C, et al. The quality of life of children with attention deficit/hyperactivity disorder: a systematic review. Eur Child Adolesc Psychiatry. 2010;19:83-105.
• ⁸
  Lichtenstein P, Halldner L, Zetterqvist J, Sjölander A, Serlachius E, Fazel S, et al. Medication for attention deficit-hyperactivity disorder and criminality. N Engl J Med. 2012;367:2006-14.
• ⁹
  Durston S, Van Belle J, De Zeeuw P. Differentiating frontostriatal and fronto-cerebellar circuits in attention-deficit/hyperactivity disorder. Biol Psychiatry. 2011;69:1178-84.
• ¹⁰
  Barkley RA, Murphy KR. Impairment in occupational functioning and adult ADHD: the predictive utility of executive function (EF) ratings versus EF tests. Arch Clin Neuropsychol. 2010;25:157-73.
• ¹¹
  Faraone SV, Buitelaar J. Comparing the efficacy of stimulants for ADHD in children and adolescents using meta-analysis. Eur Child Adolesc Psychiatry. 2010;19:353-64.
• ¹²
  Carlson GA, Dunn D, Kelsey D, Ruff D, Ball S, Ahrbecker L, et al. A pilot study for augmenting atomoxetine with methylphenidate: safety of concomitant therapy in children with attention-deficit/hyperactivity disorder. Child Adolesc Psychiatry Ment Health. 2007;1:10.
• ¹³
  Rueda MR, Fan J, McCandliss BD, Halparin JD, Gruber DB, Lercari LP, et al. Development of attentional networks in childhood. Neuropsychologia. 2004;42:1029-40.
• ¹⁴
  Rutledge KJ, Bos W van den, McClure SM, Schweitzer JB. Training cognition in ADHD: current findings, borrowed concepts, and future directions. Neurotherapeutics. 2012;9:542-58.
• ¹⁵
  Melby-Lervåg M, Hulme C. Is working memory training effective? A meta-analytic review. Dev Psychol. 2013;49:270-91.
• ¹⁶
  Tajik-Parvinchi D, Wright L, Schachar R. Cognitive rehabilitation for attention deficit/hyperactivity disorder (ADHD): promises and problems. J Can Acad Child Adolesc Psychiatry. 2014;23:207-17.
• ¹⁷
  Chacko A, Bedard AC, Marks DJ, Feirsen N, Uderman JZ, Chimiklis A, et al. A randomized clinical trial of Cogmed Working Memory Training in school-age children with ADHD: a replication in a diverse sample using a control condition. J Child Psychol Psychiatry. 2014;55:247-55.
• ¹⁸
  Dovis S, Van Der Oord S, Wiers RW, Prins PJ. Improving executive functioning in children with ADHD: training multiple executive functions within the context of a computer game. A randomized double-blind placebo controlled trial. PLoS One. 2015;10:e0121651.
• ¹⁹
  Green CT, Long DL, Green D, Iosif AM, Dixon JF, Miller MR, et al. Will working memory training generalize to improve off-task behavior in children with attention-deficit/hyperactivity disorder? Neurotherapeutics. 2012;9:639-48.
• ²⁰
  Klingberg T, Fernell E, Olesen PJ, Johnson M, Gustafsson P, Dahlström K, et al. Computerized training of working memory in children with ADHD--a randomized, controlled trial. J Am Acad Child Adolesc Psychiatry. 2005;44:177-86.
• ²¹
  Wechsler D. The Wechsler intelligence scale for children—fourth edition. London: Pearson; 2004.
• ²²
  DuPaul GJ, Power TJ, Anastopoulos AD, Reid R. ADHD rating scale-IV: checklists, norms, and clinical interpretation. New York: The Guilford; 1998.
• ²³
  Conners CK. Conners’ continuous performance test (CPT II). North Tonawanda: Multi-Health Systems; 2000.
• ²⁴
  Luciana M. Practitioner review: computerized assessment of neuropsychological function in children: clinical and research applications of the Cambridge Neuropsychological Testing Automated Battery (CANTAB). J Child Psychol Psychiatry. 2003;44:649-63.
• ²⁵
  Strauss E, Sherman EMS, Spreen O. A compendium of neuropsychological tests: administration, norms, and commentary. 3th ed. New York: Oxford USA Trade; 2006.
• ²⁶
  Owen AM, McMillan KM, Laird AR, Bullmore E. N-back working memory paradigm: a meta-analysis of normative functional neuroimaging studies. Hum Brain Mapp. 2005;25:46-59.
• ²⁷
  Jaeggi SM, Buschkuehl M, Perrig WJ, Meier B. The concurrent validity of the N -back task as a working memory measure. Memory. 2010;18:394-412.
• ²⁸
  Hogan AM, Vargha-Khadem F, Kirkham FJ, Baldeweg T. Maturation of action monitoring from adolescence to adulthood: an ERP study. Dev Sci. 2005;8:525-34.
• ²⁹
  Bauermeister JJ, Shrout PE, Ramírez R, Bravo M, Alegría M, Martínez-Taboas A, et al. ADHD correlates, comorbidity, and impairment in community and treated samples of children and adolescents. J Abnorm Child Psychol. 2007;35:883-98.
• ³⁰
  Dalsgaard S, Østergaard SD, Leckman JF, Mortensen PB, Pedersen MG. Mortality in children, adolescents, and adults with attention deficit hyperactivity disorder: a nationwide cohort study. Lancet. 2015;385:2190-6.
• ³¹
  Seixas M, Weiss M, Müller U. Systematic review of national and international guidelines on attention-deficit hyperactivity disorder. J Psychopharmacol. 2012;26:753-65.
• ³²
  Holmes J, Gathercole SE, Place M, Dunning DL, Hilton KA, Elliott JG. Working memory deficits can be overcome: impacts of training and medication on working memory in children with ADHD. Appl Cogn Psychol. 2010;24:827-36.
• ³³
  Gray SA, Chaban P, Martinussen R, Goldberg R, Gotlieb H, Kronitz R, et al. Effects of a computerized working memory training program on working memory, attention, and academics in adolescents with severe LD and comorbid ADHD: a randomized controlled trial. J Child Psychol Psychiatry. 2012;53:1277-84.
• ³⁴
  Rapport MD, Orban SA, Kofler MJ, Friedman LM. Do programs designed to train working memory, other executive functions, and attention benefit children with ADHD? A meta-analytic review of cognitive, academic, and behavioral outcomes. Clin Psychol Rev. 2013;33:1237-52.
• ³⁵
  Shipstead Z, Redick TS, Engle RW. Is working memory training effective? Psychol Bull. 2012;138:628-54.
• ³⁶
  Chacko A, Feirsen N, Bedard AC, Marks D, Uderman JZ, Chimiklis A. Cogmed working memory training for youth with ADHD: a closer examination of efficacy utilizing evidence-based criteria. J Clin Child Adolesc Psychol. 2013;42:769-83.
• ³⁷
  Mohammadi MR, Soleimani AA, Farahmand Z, Keshavarzi S, Ahmadi N. A comparison of effectiveness of regulation of working memory function and methylphenidate on remediation of attention deficit hyperactivity disorder (ADHD). Iran J Psychiatry. 2014;9:25-30.
• ³⁸
  Donk ML van der, Hiemstra-Beernink AC, Tjeenk-Kalff AC, Leij A van der, Lindauer RJ. Predictors and moderators of treatment outcome in cognitive training for children with ADHD. J Atten Disord. 2016 Mar 7. pii: 1087054716632876. [Epub ahead of print]
• ³⁹
  Roberts G, Quach J, Spencer-Smith M, Anderson PJ, Gathercole S, Gold L, et al. Academic outcomes 2 years after working memory training for children with low working memory. JAMA Pediatr. 2016;170:e154568.
• ⁴⁰
  Steiner NJ, Sheldrick RC, Gotthelf D, Perrin EC. Computer-based attention training in the schools for children with attention deficit/hyperactivity disorder: a preliminary trial. Clin Pediatr (Phila). 2011;50:615-22.
• ⁴¹
  Conti MA, Jardim AP, Hearst N, Cordás TA, Tavares H, Abreu CN de. Avaliação da equivalência semântica e consistência interna de uma versão em português do Internet Addiction Test (IAT). Rev Psiquiatr Clin. 2012;39:106-10.
• ⁴²
  Cortese S, Ferrin M, Brandeis D, Buitelaar J, Daley D, Dittmann RW, et al. Cognitive training for attention-deficit/hyperactivity disorder: meta-analysis of clinical and neuropsychological outcomes from randomized controlled trials. J Am Acad Child Adolesc Psychiatry. 2015;54:164-74.
• ⁴³
  Donk M van der, Hiemstra-Beernink AC, Tjeenk-Kalff A, Leij A van der, Lindauer R. Cognitive training for children with ADHD: a randomized controlled trial of cogmed working memory training and “paying attention in class”. Front Psychol. 2015;6:1081.
• ⁴⁴
  Manly T, Robertson IH, Anderson V, Nimmo-Smith I. Test of Everyday Attention for Children (TEA-Ch). London: Pearson Education; 1998.
• ⁴⁵
  Turner L, Shamseer L, Altman DG, Weeks L, Peters J, Kober T, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. In: Turner L, editor. Cochrane database of systematic reviews. Chichester: John Wiley & Sons Ltd; 2012.
• ⁴⁶
  Cubillo A, Smith AB, Barrett N, Giampietro V, Brammer M, Simmons A, et al. Drug-specific laterality effects on frontal lobe activation of atomoxetine and methylphenidate in attention deficit hyperactivity disorder boys during working memory. Psychol Med. 2014;44:633-46.
• ⁴⁷
  Daly E, Ecker C, Hallahan B, Deeley Q, Craig M, Murphy C, et al. Response inhibition and serotonin in autism: a functional MRI study using acute tryptophan depletion. Brain. 2014;137:2600-10.
• ⁴⁸
  Christakou A, Murphy CM, Chantiluke K, Cubillo AI, Smith AB, Giampietro V, et al. Disorder-specific functional abnormalities during sustained attention in youth with Attention Deficit Hyperactivity Disorder (ADHD) and with autism. Mol Psychiatry. 2013;18:236-44.

• Erratum

  The authors of the article entitled “Computerized cognitive training in children and adolescents with attention deficit/hyperactivity disorder as add-on treatment to stimulants: feasibility study and protocol description” (doi: http://dx.doi.org/10.1590/2237-6089-2016-0039), published in Trends in Psychiatry and Psychotherapy, 2017, volume 39, issue 2, pages 65-76, have identified an error in the name of an author. “Carlos Roberto Moreira-Maia” should read “Carlos Renato Moreira Maia,” in both the author byline and in the Disclosure. Here we reproduce the final, correct version of the author byline:
  Virginia de Oliveira Rosa,^1,2 Marcelo Schmitz,^2,3 Carlos Renato Moreira-Maia,² Flavia Wagner,² Igor Londero,⁴ Caroline de Fraga Bassotto,² Guilherme Moritz,² Caroline dos Santos de Souza,² Luis Augusto Paim Rohde^2,3
  And here we reproduce the final, correct version of the author's disclosure:
  Carlos Renato Moreira-Maia has received financial research support from Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) and CNPq; received fees in February 2016 for the development of educational materials on excessive daytime sleepiness, for Libbs; has received travel and accommodation support for a speaker training from Shire; has received travel awards from the Health Technology Assessment Institute (IATS), UFRGS; and travel, accommodation and registration support to the fourth and fifth World Congress on ADHD from the World Federation of ADHD.

Publication Dates

• Publication in this collection
  Apr-Jun 2017

History

• Received
  30 May 2016
• Accepted
  30 Nov 2016