The Five Digits Test in the assessment of older adults with low formal education: construct validity and reliability in a Brazilian clinical sample

O Teste dos Cinco Dígitos na avaliação de idosos com baixa escolaridade: validade de construto e confiabilidade em uma amostra clínica do Brasil

Abstract

Introduction

In the assessment of older adults with very low formal education, typical tests of selective attention and inhibitory control are biased by reading abilities. In this sense, we aim to assess the psychometric characteristics of the Five Digits Test (FDT), a numerical Stroop paradigm, in older adults without cognitive disorders, with mild cognitive impairment, and with dementia.

Methods

We assessed 211 Brazilian older adults with low formal education using the FDT and other cognitive measures. Construct validity and reliability were assessed by correlations and internal consistency.

Results

The FDT test had weak correlations with crystalized intelligence tests and moderate-high correlations with fluid intelligence measures and tests of global cognitive status and executive functions. The split-half coefficient of reliability showed high internal consistency (>0.900).

Conclusion

Together, the results suggest that the FDT is a valid and reliable measure for the assessment of processing speed and executive functions in older adults with low formal education.

Executive function; processing speed; neuropsychological test; psychological test

Resumo

Introdução

Na avaliação de idosos com educação formal muito baixa, os testes típicos de atenção seletiva e controle inibitório são enviesados devido à dependência de habilidades básicas de leitura. Nesse sentido, pretende-se avaliar as características psicométricas do Teste dos Cinco Dígitos (Five Digits Test, FDT), teste de paradigma Stroop-numérico, em idosos sem transtornos cognitivos, em pacientes com comprometimento cognitivo e em pacientes com demência.

Métodos

Foram avaliados 211 idosos brasileiros com baixa escolarização formal por meio do FDT e outras medidas cognitivas. Analisamos a validade de construto e a confiabilidade por correlações e consistência interna.

Resultados

O FDT apresentou correlações fracas com testes de inteligência cristalizada e correlações de moderadas ou fortes com medidas de inteligência fluida, medidas cognitivas globais e testes de funções executivas. O teste apresentou consistência interna elevada (>0.900).

Conclusão

Em conjunto, os resultados sugerem que o FDT é uma medida válida e confiável para a avaliação da velocidade de processamento e funções executivas em idosos com baixa escolaridade.

Funções executivas; velocidade de processamento; testes neuropsicológicos; testes psicológicos

Introduction

Since the seminal study of John R. Stroop¹ on the behavior of human attention under conflicting situations, the use of interference effect paradigms has become one of the most commonly used neuropsychological measures in clinical and research settings. Stroop tests are associated with two aspects of the attentional system: “automatic” attentional processes (usually related to posterior brain regions and requiring little cognitive effort to be performed) and “controlled” attentional processes (usually related to anterior brain regions and requiring significant cognitive effort to be performed).² These two processes, under the Stroop paradigm, can be represented by the constructs of processing speed and selective attention (a component of the inhibitory control system of the executive functions).³ These cognitive processes are part of the fluid intelligence concept, which represents a series of cognitive functions that allow us to deal with new contextual demands and the resolution of everyday problems. According to Diamond’s model,³ high-ordered executive functions including reasoning and problem solving are the fluid aspects of human intelligence.

Although healthy aging involves a general loss of speed and efficiency in the selective attention process, following a decline in the fluid aspects of cognition,⁴ several clinical conditions that are more frequent in older age may accentuate these impairments. Deficits in executive functions, including inhibitory control, are common in different conditions associated with pathological aging. Patients with Alzheimer’s disease dementia (AD) usually show impairment in Stroop-like tests, probably due to progressive sensorial loss.⁵ Deficits in Stroop tests are related to rates of conversion to dementia.⁶

Classical Stroop tests (color-word interference) have important biases in the assessment of older adults with low formal education. Although performance on the color-word version of the Stroop test in high-educated Caucasian participants was not strongly associated with education,⁷ other studies with different populations found more expressive associations.⁸ A Brazilian study found moderate-to-high correlations between years of formal education and test scores.⁹ The test is also dependent on reading abilities, according to Cox et al.¹⁰ In older adults with very low formal education, reading routines may not be fully automated, biasing the analysis of test time, errors and interference scores. For these cases, alternatives include the All-Language and Mini-Verbal tests, designed to assess the same cognitive constructs as the classical neuropsychological tests, but adapted to be less affected by formal education, native language and reading routines.¹¹ An analogue to the Stroop Color-Word Test is the counting/numerical versions of the paradigms.¹¹ On these tasks, the conflicting condition is between reading numbers and counting quantities. One of the test versions adapted to clinical practice is the Five Digits Test (FDT).¹¹ The test involves the numbers 1, 2, 3, 4 and 5, as well the quantities from 1 to 5 elements (^{Supplementary Table 1}).

Research using the FDT in older adults involved AD, depression and mild cognitive impairment (MCI).^2,12 Although these previous studies assessed criterion-related and discriminant validity of the FDT, the association of the test with other cognitive measures and its internal consistency in older adults was not evaluated. The present study aims to analyze the construct validity and reliability of the FDT in the assessment of older adults with low formal education. We seek evidence of these properties in a mixed sample of healthy older adults, older adults with MCI, and older adults with AD. We expect, based on previous reports from other populations, to obtain high internal consistency and correlations with the global cognitive status of patients, with measures of intelligence (especially fluid intelligence) and with other executive functions.

Methods

Participants

This study is part of a broader research focusing on the relationship between depressive symptoms and dementia in older adults, approved by our university ethics board (protocol no. 334/06). The study was conducted in accordance with the Declaration of Helsinki and the applicable resolutions of the Brazilian National Council of Health. The participants or, when necessary, their caregivers, provided written consent for participation.

In the present study, we assessed 211 older adults from the city of Belo Horizonte, state of Minas Gerais, Brazil. Participants were enrolled in a public secondary/tertiary health unit specialized in the assessment of older adults. At this clinic, most patients have a very low socioeconomic profile, with monthly family incomes usually bellow 600 USD. These participants usually have very low formal education (mostly bellow 4 years of formal education). Participant characteristics are shown in Table 1.

Table 1
Participant characteristics

	Control (n=66)		MCI (n=89)		AD (n=60)
	n	%	n	%	n	%	χ 2
Sex
Male	17	26	40	45	25	42	6.35
Female	49	74	49	55	35	58
Age (years)
60-69	20	31	23	26	13	22	14.04*
70-79	38	57	33	37	24	40
80-89	9	13	33	37	23	38
Formal education
Illiterate	3	5	6	7	8	13	42.76*
≤ 8 years	28	42	70	79	46	77
9-11 years	19	29	7	8	5	8
> 11 years	16	24	6	7	1	2

• MCI = mild cognitive impairment, AD = Alzheimer’s disease dementia.
• * p<0.01.

We first assessed cognitively healthy older adults (n=62). These participants showed a Clinical Dementia Rating (CDR)¹³ score of 0, Mini-Mental State Examination (MMSE)¹⁴ scores above the cutoff based on formal education, scores below the cutoff for depression on the Geriatric Depression Scale (GDS)¹⁵ and no expressive impairment on instrumental activities of daily living.¹⁶ They were recruited at the memory clinic where this study was conducted or were invited trough announcements in our university or directly by researchers.

We investigated a second group formed by patients with MCI, diagnosed according to the criteria of Winblad et al.¹⁷ There were 42 patients with amnestic MCI, 36 with multiple-domain amnestic MCI, 8 with non-amnestic MCI and 3 with multiple-domain non-amnestic MCI patients, totalizing 89 participants. All the participants with MCI had a clinical history typical of Alzheimer’s disease, MMSE scores above the cutoff for formal education, CDR scores below 1 and no expressive impairment on instrumental activities of daily living.

A third group was formed by probable AD patients (AD, n=60) in mild stage (CDR=1), diagnosed according to the guidelines of McKhann et al.¹⁸ MCI and AD diagnoses were discussed in expert meetings that included at least one geriatrician and one clinical neuropsychologist.

Neuropsychological assessment

FDT. This is a numerical Stroop task divided into four components. The first component (reading) demands subjects to name numbers from 1 to 5 as fast as they can. On the second component (counting), they need to describe quantities from 1 to 5. The third component (choosing) involves a selective attention trial, where the subjects must not read the numbers, but rather tell how many numbers are present in each stimulus, in an incongruent condition. The last component (shifting) is similar to the choosing trial, but for each of the five stimuli, there is one previously where the subject must read the stimulus numbers. We recorded the partial (25 first stimuli) and total (50 stimuli) time and errors for each test component. We used the test total scores as the main measure in this study. Higher scores indicate worse performance. A detailed description of the FDT application, scoring and specific components is shown in ^{Supplementary Table 1}.

Global cognitive functioning. For this assessment we adopted the MMSE¹⁴ and the Mattis Dementia Rating Scale (MDRS)¹⁹ total scores. These two screening tests comprise several tasks of different cognitive aspects, involving language, attention, constructional praxis and executive functioning. Higher scores indicate better performance. MMSE scores may range from 0 to 30, and DRS scores, from 0 to 144.

Fluid and crystallized intelligence. We adopted two different tests for this assessment. For fluid intelligence, we used the Colored Raven Progressive Matrices test.²⁰ This is a well-validated measure of fluid intelligence, and it is less biased by cultural and educational factors. As a measure of crystallized intelligence, we selected the Vocabulary subtest of the Wechsler Adult Intelligence Scale, third edition.²¹ This subtest involves the oral definition of 33 different words stratified by their frequency and difficulty in Brazilian-Portuguese. We used the test’s scaled score, which ranges from 0 to 19 (higher scores indicate better performance).

Executive functions. For the assessment of this cognitive domain, we selected measures previously adapted and validated for the assessment of older adults with low formal education. First, the Frontal Assessment Battery (FAB),²² a brief scale for executive functions, comprises six subtests related to conceptualization, fluency, motor programming, selective attention, inhibitory control and environmental autonomy, with total scores ranging from 0 to 18. Second, the Tower of London Test (TOL)²³ is a planning measure which demands the organization of three colored balls on three pins of different sizes, to match a specific goal position. We used the scoring system proposed by Portella et al.,²⁴ ranging from 0 to 46. Finally, we used a composite score of three category fluency tests (animals, fruits and grocery items). These last measures demand executive functions and processing speed for its performance.²⁵ In all these three measures, higher scores indicate better performance.

The participants performed the neuropsychological tests during their clinical assessment no more than one month after the diagnosis, all by trained neuropsychologists or psychology students under direct supervision of two of the authors (JJdeP and LFMD). We used part of the neuropsychological tests described in the measures section for patient diagnosis.

Statistical analysis

Since most of the neuropsychological data collected violated normal distribution assumptions, we adopted non-parametric tests for the analysis. We presented sociodemographic data as percentages and frequencies, and computed group differences using chi-square tests. Medians and the 25th and 75th percentiles described the participants’ performance on neuropsychological tests.

We tested the association of FDT components with other neuropsychological tests using Spearman rank-order correlations. To avoid type-1 errors due to the exploratory nature of the study and the large number of comparisons, we adopted a more conservative significance level (0.01) and performed an analysis on all the participants simultaneously. We adopted the following guidelines to interpret effect sizes in correlations: strong (±0.5), moderate (±0.3) and weak (±0.1). For the assessment of FDT reliability, we used the Guttman split-half coefficient, since the FDT is a timed task with a fixed number of stimuli (50). This allowed to compare two halves of each FDT component (the first and the last 25 items) in terms of time and error measures, totalizing 16 test items (8 measures of time and 8 measures of errors).

We performed the analysis in the whole sample (controls, MCI and AD), since the three groups may be understood as a continuum of cognitive functioning and to achieve a higher statistical power for the procedures. The whole sample size has a 99% power to detect large and moderate effect sizes, those which are expected in terms of validity and reliability analysis.

Results

Table 1 and ^{Supplementary Table 2} show participant characteristics in terms of sociodemographic and neuropsychological measures, respectively. Table 2 shows the pattern of correlations between FDT and other neuropsychological measures.

Table 2
Correlations between FDT and measures of cognitive status, intelligence and executive functions

FDT component	MMSE	MDRS	Vocabulary	Raven	FAB	TOL	Fluency
FDT – Reading (time)	-0.683*	-0.700*	-0.314*	-0.538*	-0.672*	-0.368*	-0.681*
FDT – Counting (time)	-0.665*	-0.674*	-0.299*	-0.535*	-0.671*	-0.347*	-0.632*
FDT – Choosing (time)	-0.617*	-0.602*	-0.244*	-0.466*	-0.591*	-0.359*	-0.544*
FDT – Shifting (time)	-0.541*	-0.540*	-0.273*	-0.530*	-0.534*	-0.286*	-0.547*
FDT – Reading (errors)	-0.298*	-0.277*	-0.133	-0.124	-0.306*	-0.213*	-0.217*
FDT – Counting (errors)	-0.257*	-0.244*	-0.175	-0.241*	-0.277*	-0.126	-0.188*
FDT – Choosing (errors)	-0.482*	-0.475*	-0.160	-0.331*	-0.470*	-0.318*	-0.398*
FDT – Shifting (errors)	-0.580*	-0.626*	-0.286	-0.388*	-0.522*	-0.379*	-0.511*

• FDT = Five Digits Test; MMSE = Mini-Mental State Examination; MDRS = Mattis Dementia Rating Scale; FAB = Frontal Assessment Battery; TOL = Tower of London Test.
• * Significant rank-order correlation at p<0.01.

Moderate-to-high correlations were found between FDT times and measures of cognitive status (MMSE and MDRS), ranging from -0.540 to -0.700. While the data suggest a decreasing pattern of association between the FDT timed components and global cognitive status (the strongest associations were with the reading component and the weakest with the shifting component), the opposite pattern occurs with the measures of errors (more strongly associated with choosing and shifting and less strongly associated with reading and counting).

We found a different pattern of association between fluid/crystallized intelligence and FDT performance. Considering the timed measures, all the correlations were significant (p<0.01). However, associations with fluid intelligence were stronger (ranging from -0.540 to -0.538) than with crystallized intelligence (ranging from -0.244 to -0.299). The pattern was similar to test errors, where no correlations were found between FDT and the Vocabulary test, but significant ones were found for the Colored Raven Progressive Matrices (except for reading errors).

The correlations between FDT times and classical executive function tests revealed a more heterogeneous pattern of association. While correlations with the FAB (ranging from -0.534 to -0.672) and with the verbal fluency test (ranging from -0.547 to -0.681) were moderate to high, correlations with the TOL were less expressive (-0.213 to -0.368). The same pattern emerged from the test errors, but with weaker correlations. We also found a differential pattern of association of time and errors with the executive function tests, as seen in global cognitive measures. The “purer” processing speed measures (reading and counting times) were discreetly more strongly associated with the other executive function tests than the “executive” components of the test (choosing and shifting times). However, we found the opposite pattern for test errors.

Reliability analysis results are shown in Table 3. As suggested by the test manual,¹¹ we stratified the reliability analysis by group (control, MCI and AD), but also investigated the whole sample. The Spearman-Brown and Guttman coefficients for the two test parts were all above 0.9, suggesting high internal consistency and good reliability.

Table 3
Reliability of the FDT

Reliability	Whole sample (n=211)
Cronbach’s alpha (part 1)*	0.726
Cronbach’s alpha (part 2) †	0.704
Correlation between forms	0.918
Spearman-Brown coefficient	0.958
Guttmann coefficient	0.957

• Time and errors for the first* and last ^† 25 stimuli on each test component.

Discussion

Our results suggest a convergent pattern of correlations between FDT measures and tests of global cognitive functioning, intelligence and executive functions. These results are indicative of construct validity for the FDT in this heterogeneous clinical sample. The FDT also showed good internal reliability by the split-half method.

Correlations with measures of global cognitive functioning

We found a study that correlated FDT scores with MMSE, a measure of global cognitive functioning.²⁶ In that work, FDT reading time and shifting errors were correlated to MMSE scores (r= -0.31 and 0.30). These moderate correlations might be due to the relatively weak association between dementia severity and selective attention tests. These findings are different from the present study, since in our patients moderate-to-high correlations were found for times and weak-to-moderate ones for errors between FDT, MMSE and MDRS. Maybe the inclusion of patients with mild deficits (all with CDR scores ≤1) has strengthened the association between these measures.

An interesting finding is the decreasing strength of the correlations with global cognitive status measures along the four FDT components in terms of time, and the reverse pattern in terms of errors. This indicates that the cognitive status of these patients might be more related to a global deficit (represented by slow processing speed) than to executive dysfunction itself. In this sense, based on the present data, to characterize executive deficits in older adults, we recommend the use of choosing and shifting errors rather than timed measures.

Correlations with intelligence tests

We found no previous studies that correlated measures of fluid and crystallized intelligence with the FDT. The present results indicate a stronger association of FDT times with the fluid intelligence measure (correlations ranging from -0.46 to -0.53) than with the crystallized intelligence measure (-0.24 to -0.31), and a similar pattern for FDT errors (associated with fluid intelligence only, coefficients ranging from -0.24 to -0.38). These results are in accordance with previous studies which have associated measures of processing speed and executive functions to fluid intelligence.^3,4 In a more recent model of executive functions, Diamond⁴ suggests that fluid intelligence per se is a higher aspect of executive functioning. Since the attentional system would be directly connected to the inhibitory control module in her model, association of all FDT components with fluid intelligence is expected. Correlations with crystallized intelligence may reflect cultural and demographic peculiarities of our samples, aspects that are not strongly associated with test performance¹² but may account for a smaller portion of test variance.²⁷

Correlations with measures of executive functions

The association of the FDT with other executive function measures was explored in previous studies with different healthy and clinical samples. In the original FDT manual¹¹ a mixed clinical sample was assessed using the FDT and the Letter Alternation Test and Trail Making Test, and reported moderate-to-high correlations with the first measure (0.57 to 0.91) and weak correlations with the second (0.20 to 0.36), suggesting a stronger association with verbal-executive tests than with motor-executive tests. de Paula et al.²⁶ found correlations between reading time and verbal fluency (-0.47) and between shifting errors and the same measure (-0.30) in a sample of healthy older adults.

In synthesis, our results are in accordance with findings from previous studies that have also adopted the FDT and used correlations to assess its relationship with other measures of executive functions. Although significant, none of the correlations reported here showed more than 50% of shared variance, suggesting a relative independence of the FDT when compared to tests of other aspects of executive functions. Studies using the Stroop Color-Word Test, however, found stronger correlations.^12,28 These findings sustain the relevance of the FDT in neuropsychological assessment, as this test shows greater specificity to the inhibitory control process when compared to other executive function tests. An important limitation of our study is the lack of the Stroop Color-Word Test as a parameter of comparison in older adults with higher educational levels.

Reliability of the FDT

Test reliability was assessed using the same method adopted by Sedó¹¹ in a sample of 314 healthy adults in Spain. Working only with the timed measures of the FDT, the Spearman-Brown coefficient of the four test components ranged from 0.86 (choosing) to 0.94 (reading), all significant at p<0.001. A parallel reliability analysis conducted in a clinical sample of 92 American stroke patients using the same method and cited in the same manual¹¹ found coefficients ranging from 0.80 (shifting) to 0.92 (counting and choosing). Mixing these two samples, the authors reported coefficients from 0.92 (shifting) to 0.95 (reading). The previous results suggest that all FDT measures have very similar reliability. Our approach in the present study was different from that of those authors, since we combined all FDT measures in the first and second halves of the test, yielding coefficients ranging from 0.91 to 0.95, corroborating the previous findings. The results are also similar to studies that adopted the traditional Stroop Color-Word Test and other methods of reliability analysis. Golden²⁹ reported reliabilities from 0.73 to 0.86 using the test-retest method.

Our study has limitations that should be addressed. First, this was a heterogeneous sample, and the findings may not be generalized to specific groups of patients or to different settings. The study also had a non-balanced sample size (different number of participants in each group), and the subjects were not matched by demographic factors, which may have influenced the results. The lack of a criterion measure to the FDT (like the Stroop Color-Word Test) and of other methods to assess reliability (such as test-retest) is another limitation that should be addressed in future research.

Conclusion

Our study provides evidence of construct validity and reliability for the FDT. Studies aiming at the development and validation of neuropsychological tests developed for the assessment of older adults are a growing demand in both clinical and research settings, and the FDT seems an interesting measure for the assessment of processing speed and executive functions in this context.

Acknowledgements

This study received financial support via grants from Instituto Nacional de Ciência e Tecnologia em Medicina Molecular (INCT-MM; FAPEMIG: CBB-APQ-00075-09/CNPq 573646/2008-2) and Fundação de Amparo à Pesquisa do Estado de Minas Gerais (FAPEMIG; APQ-01972/12-10, APQ-02755-10).

The authors are thankful to the undergraduates and graduates who helped with data collection.

https://minio.scielo.br/documentstore/2238-0019/ZDf5VFJWCffBgctMKmdPkrv/0d704d246497c53da76c6023876f443711920e0b.pdf

References

• ¹
  Stroop JC. Studies of interference in serial verbal reactions. J Exp Psychol. 1935;18:643-62.
• ²
  de Paula JJ, Costa DS, Moraes EM, Nicolato R, Sedó M, Malloy-Diniz LF. Automatic and controlled attentional processes in amnestic mild cognitive impairment: the use of a mini-verbal test. Psychology. 2012;3:379-83.
• ³
  Diamond D. Executive functions. Annu Rev Psychol. 64;135-168
• ⁴
  Salthouse T. Consequences of age-related cognitive declines. Annu Rev Psychol. 2012;63:201-26.
• ⁵
  Ben-David BM, Tewari A, Shakuf V, Van Lieshout PH. Stroop effects in Alzheimer’s disease: selective attention speed of processing, or color-naming? A meta-analysis. J Alzheimers Dis. 2014;38:923-38.
• ⁶
  Balota DA, Tse CS, Hutchison KA, Spieler DH, Duchek JM, Morris JC. Predicting conversion to dementia of the Alzheimer’s type in a healthy control sample: the power of errors in Stroop color naming. Psychol Aging. 2010;25:208-18.
• ⁷
  Anstey KJ, Matters B, Brown AK, Lord SR. Normative data on neuropsychological tests for very old adults living in retirement villages and hostels. Clin Neuropsychol. - 2000;14:309-17.
• ⁸
  Moering RG, Schinka JA, Mortimer JA, Graves AB. Normative data for elderly African Americans for the Stroop Color and Word Test. Arch Clin Neuropsychol. 2004;19:61-71.
• ⁹
  Campanholo KR, Romão MA, Machado MAR, Serrao VT, Coutinho DGC, Benute GR, et al. Performance of an adult Brazilian sample on the Trail Making Test and Stroop Test. Dement Neuropsychol. 2014;8:26-31.
• ¹⁰
  Cox CS, Chee E, Chase GA, Baumgardner TL, Schuerholz LJ, Reader MJ, et al. Reading proficiency affects the construct validity of the Stroop Test Interference Score. Clin Neuropsychol. 1997;11:105-10.
• ¹¹
  Sedó MA. Five Digit Test. Madrid: TEA Ediciones; 2007.
• ¹²
  de Paula JJ, Ávila R, Costa DF, Moraes EN, Bicalho MA, Nicolato R, et al. Assessing processing speed and executive functions in low educated older adults: the use of the Five Digits Test in patients with Alzheimer’s disease, mild cognitive impairment and major depressive disorder. Clin Neuropsychiatr. 2011;8:339-46.
• ¹³
  Morris JC. The Clinical Dementia Rating (CDR): current version and scoring rules. Neurology. 1993;43:2412-4.
• ¹⁴
  Brucki SM, Nitrini R, Caramelli P, Bertolucci PH, Okamoto IH. Suggestions for utilization of the mini-mental state examination in Brazil. Arq Neuropsiquiatr. 2003;61:777-81.
• ¹⁵
  Almeida OP, Almeida SA. Short versions of the geriatric depression scale: a study of their validity for the diagnosis of a major depressive episode according to ICD-10 and DSM-IV. Int J Geriatr Psychiatry. 1999;14:858-65.
• ¹⁶
  Assis LO, de Paula JJ, Assis MG, Moraes EN, Malloy-Diniz LF. Psychometric properties of the Brazilian version of Pfeffer’s Functional Activities Questionnaire. Front Aging Neurosci. 2014;6:255.
• ¹⁷
  Winblad B, Palmer L, Kivilpeto M, Jelic V, Fratiglioni L, Wahlunf LO, et al. Mild cognitive impairment beyond controversies, towards a consensus: report of the International Working Group on Mild Cognitive Impairment. J Intern Med. 2004;256:240-6
• ¹⁸
  McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease Neurology. 1984;34:939-44.
• ¹⁹
  Porto CS, Fichman HC, Caramelli P, Bahia VS, Nitrini R. Brazilian version of the Mattis Dementia Rating Scale: diagnosis of mild dementia in Alzheimer’s disease. Arq Neuropsiquiatr. 2003;61:339-45.
• ²⁰
  Raven J, Raven JC, Court JH. Manual for Raven’s Progressive Matrices and Vocabulary Scales. San Antonio: Harcourt Assessment; 2004.
• ²¹
  Nascimento E. Adaptação, validação e normatização do WAIS III para uma amostra brasileira. In: Wechsler D. WAIS-III: manual para administração e avaliação. São Paulo: Casa do Psicólogo; 2004.
• ²²
  Dubois B, Slachevsky A, Litvan I, Pillon B. The FAB: a Frontal Assessment Battery at bedside. Neurology. 2000;55:1621-6.
• ²³
  Shallice T. Specific impairments of planning. Philos Trans R Soc Lond B Biol Sci. 1982;298:199-209.
• ²⁴
  Portella MJ, Marcos-Bars T, Rami-González L, Navarro-Odriozola V, Gastó-Ferrer C, Salamero M. Tower of London’: mental planning, validity and the ceiling effect. Rev Neurol. 2003;37:210-3.
• ²⁵
  de Paula JJ, Paiva GCC, Costa DS. Use of a modified version of the switching verbal fluency test for the assessment of cognitive flexibility. Dement Neuropsychol. 2015;9:258-64.
• ²⁶
  de Paula JJ, Costa DS, Bertola L, Miranda DM, Malloy-Diniz LF. Verbal fluency in older adults with low educational level: what is the role of executive functions and processing speed? Rev Bras Psiquiatr. 2013;35:440-2.
• ²⁷
  Hsieh SL, Tori CD. Normative data on cross-cultural neuropsychological tests obtained from Mandarin-speaking adults across the life span. Arch Clin Neuropsychol. 2007;22:283-96.
• ²⁸
  de Paula JJ, Abrantes S, Neves FS, Malloy-Diniz LF. The Five Digit Test on the assessment of psychiatric patients with heterogeneous educational backgrounds: evidences of validity on the assessment of bipolar disorder. Clin Neuropsychiatr. 2014;11:103-7.
• ²⁹
  Golden CJ. A group version of the Stroop Color and Word Test. J Pers Assess. 1975;39:386-8.

Publication Dates

• Publication in this collection
  31 Aug 2017
• Date of issue
  Jul-Sep 2017

History

• Received
  24 Aug 2016
• Accepted
  16 Apr 2017