SciELO - Scientific Electronic Library Online

 
vol.76 issue3Association between executive and food functions in the acute phase after strokeThe importance of central auditory evaluation in Friedreich's ataxia author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

  • text new page (beta)
  • English (pdf)
  • Article in xml format
  • How to cite this article
  • SciELO Analytics
  • Curriculum ScienTI
  • Automatic translation

Indicators

Related links

Share


Arquivos de Neuro-Psiquiatria

Print version ISSN 0004-282XOn-line version ISSN 1678-4227

Arq. Neuro-Psiquiatr. vol.76 no.3 São Paulo Mar. 2018

http://dx.doi.org/10.1590/0004-282x20180006 

Articles

Normative values of the Brief Repeatable Battery of Neuropsychological Tests in a Brazilian population sample: discrete and regression–based norms

Valores normativos da Brief Repeatable Battery of Neuropsychological Tests (BRB–N) em uma amostra da população Brasileira: dados discretos e contínuos

Alfredo Damasceno1 

Juliana Machado Santiago dos Santos Amaral2 

Amilton Antunes Barreira3 

Jefferson Becker4  5 

Dagoberto Callegaro6 

Kenia Repiso Campanholo6 

Luciana Azevedo Damasceno7 

Denise Sisterolli Diniz8 

Yara Dadalti Fragoso9 

Paula S Franco9 

Alessandro Finkelsztejn10 

Frederico M H Jorge6 

Marco Aurélio Lana–Peixoto2 

Andre Palma da Cunha Matta7 

Andréia Costa Rabelo Mendonça8 

Janaína Noal10 

Renata Alves Paes11 

Regina Maria Papais–Alvarenga11 

Adriana Gutterres Pereira4 

Carina Tellaroli Spedo3 

Benito Pereira Damasceno1 

1Universidade de Campinas, Departamento de Neurologia, Campinas SP, Brasil

2Universidade Federal de Minas Gerais, Centro de Investigação em Esclerose Múltipla, Belo Horizonte MG, Brasil

3Universidade de São Paulo, Faculdade de Medicina de Ribeirão Preto, Hospital das Clínicas de Ribeirão Preto, Departamento de Neurociências e Ciências do Comportamento, Ribeirão Preto SP, Brasil

4Hospital São Lucas, Serviço de Neurologia, Porto Alegre RS, Brasil

5Hospital São Lucas, Serviço de Neurologia, Porto Alegre RS, Brasil

6Universidade de São Paulo, Departamento de Neurologia, São Paulo SP, Brasil

7Universidade Federal Fluminense, Departamento de Neurologia, Niterói RJ, Brasil

8Universidade Federal de Goiás, Hospital das Clínicas, Departamento de Neurologia, Goiânia GO, Brasil

9Universidade Metropolitana de Santos, Departamento de Neurologia, Santos SP, Brasil

10Hospital das Clínicas de Porto Alegre, Departamento de Neurologia, Porto Alegre RS, Brasil

11Universidade Federal do Estado do Rio de Janeiro, Departamento de Neurologia, Rio de Janeiro RJ, Brasil

ABSTRACT

Objective

Cognitive dysfunction is common in multiple sclerosis. The Brief Repeatable Battery of Neuropsychological Tests (BRB–N) was developed to assess cognitive functions most–frequently impaired in multiple sclerosis. However, normative values are lacking in Brazil. Therefore, we aimed to provide continuous and discrete normative values for the BRB–N in a Brazilian population sample.

Methods

We recruited 285 healthy individuals from the community at 10 Brazilian sites and applied the BRB–N version A in 237 participants and version B in 48 participants. Continuous norms were calculated with multiple–regression analysis.

Results

Mean raw scores and the 5th percentile for each neuropsychological measure are provided, stratified by age and educational level. Healthy participants' raw scores were converted to scaled scores, which were regressed on age, sex and education, yielding equations that can be used to calculate predicted scores.

Conclusion

Our normative data allow a more widespread use of the BRB–N in clinical practice and research.

Keywords cognition; multiple sclerosis; neuropsychology

RESUMO

Objetivo

Disfunção cognitiva é comum em pacientes com esclerose múltipla. Por isto, a Brief Repeatable Battery of Neuropsychological Tests (BRB–N) foi desenvolvida para avaliar as funções cognitivas mais frequentemente alteradas na doença. Entretanto, estão faltando dados normativos desta bateria no Brasil. Assim, nosso objetivo foi fornecer valores normativos contínuos e discretos da BRB–N para a população brasileira.

Métodos

Foram recrutados 285 indivíduos sadios da comunidade em 10 centros do Brasil e aplicada a versão A em 237 e a versão B em 48 sujeitos. Normas contínuas foram calculadas com análise de regressão múltipla.

Resultados

Escores brutos médios e 5°percentil para cada subteste são fornecidos, estratificados por idade e nível educacional. Os escores brutos dos sujeitos sadios foram convertidos em escores de escalas e postos em regressão quanto a idade, sexo e educação, fornecendo equações que podem ser usadas para calcular escores previsíveis.

Conclusão

Nossos dados normativos permitem um uso mais amplo da BRB–N na prática clínica e na pesquisa, fornecendo normas para dados discretos e contínuos. Normas para dados discretos deveriam ser usadas com cuidado e escores demograficamente ajustados são geralmente preferidos quando interpretando dados neuropsicológicos.

Palavras-chave cognição; esclerose múltipla; neuropsicologia

Cognitive function abnormalities are increasingly recognized as a major complaint among multiple sclerosis (MS) patients. Neuropsychological studies have shown cognitive dysfunction in up to two thirds of MS patients1, and it has been described in the earliest stages of a clinically, or even radiologically, isolated syndrome2. Cognitive dysfunction generally affects information processing speed and episodic memory and thus has a substantial contribution to disability, impairing daily living and work capacity. Nevertheless, assessing such an important and complex domain requires a systematic approach. Given the low sensitivity of the widely–used Mini Mental State Examination (MMSE) for detecting cognitive dysfunction in MS3, Rao and co–workers of the American National MS Society developed the Brief Repeatable Battery of Neuropsychological Tests (BRB–N), a battery of neuropsychological measures covering functions most commonly impaired in MS, with 71% sensitivity and 94% specificity4. This battery includes tests for the assessment of verbal and visuospatial memory, sustained attention and information processing speed, working memory and verbal fluency. More recently, an international panel introduced a shorter cognitive battery for use when time constraints are strict, comprising tests for information processing speed, verbal and visuospatial memory (the Brief International Cognitive Assessment for Multiple Sclerosis)5. Although the BRB–N has been employed extensively in MS research worldwide and is also validated in the Portuguese language6, normative values for this test battery have been published only in few European countries and the USA7,8, limiting a more widespread application in clinical practice, especially in South America9. The Brief International Cognitive Assessment for Multiple Sclerosis battery has also recently been validated in Brazil, but normative data is not yet available10. An accurate classification of neuropsychological impairment depends on the normative comparison. If normative data are derived from individual samples that do not match the individual under assessment, misclassification may occur. Therefore, development of appropriate normative standards is critical for any neuropsychological measure11. Neuropsychological normative data have usually been presented in terms of discrete norms, where means and standard deviations for each age group are provided. Nevertheless, discrete norms have been subject to criticism in the last few years and the use of regression–based demographically–adjusted scores has increasingly been recommended11. In this setting, our aim was to provide both discrete and continuous normative values for the BRB–N in a sample of the Brazilian population, while assessing the effect of demographic factors on cognitive performance.

METHODS

Participants

We enrolled a total of 285 healthy individuals recruited from the community at 10 sites across Brazil (Belo Horizonte, Campinas, Rio de Janeiro, Niteroi, Goiania, Porto Alegre, Santos, São Paulo, and Ribeirão Preto).

The sample size was estimated for differences (or correlations) between two independent means by adopting an effect size of 0.5, α error probability of 0.05, and power (1–β error probability) of 0.80 for two groups (version A and B subgroups; two–tailed). In this way, the minimal sample size would be 128 individuals, divided into 64 for each subgroup. However, since most BRB–N normative studies in the USA and European countries employed at least 150 individuals for version A (the most used)7,8, we enrolled 237 participants for this version, and 48 participants for version B of the battery.

Inclusion criteria were ages from 18 through 65 years, educational level of at least primary school (four years), visual acuity of at least 0.5 (or 20/40) in both eyes (with or without lens correction), normal neurological examination, and having slept enough the night prior to the testing. Individuals were excluded if they had neurological disease or major psychiatric illness, history of alcohol or drug abuse, serious head trauma, learning disability and a recent major medical illness. The study was approved by the ethics committee of the faculty of medical sciences of the University of Campinas and all participants provided written informed consent.

Neuropsychological test procedures

The BRB–N version A was applied in 237 participants and version B was applied in 48 participants. Different versions are important, to minimize practice effects with longitudinal administration. These groups were not different regarding gender distribution, age and educational level (Table 1).

Table 1 Demographic data (expressed as mean values and standard deviation). 

Variable Total (n = 285) BRB–N Version A (n = 237) BRB–N Version B (n = 48) Comparisons*
Male/Female: % 42.6/57.4 44.5/55.5 33.3/66.7 p = 0.199
Age: years (range) 38.14 ± 13.37 (18–66) 37.72 ± 13.40 (18–66) 38.23 ± 11.79 (18–64) p = 0.599
Education: years (range) 12.34 ± 3.68 (4–22) 12.24 ± 3.59 (3–22) 11.98 ± 4.06 (4–21) p = 0.693
MMSE: total score (range) 28.51 ± 1.54 (23–30) 28.51 ± 1.57 (23–30) 28.52 ± 1.46 (25–30) p = 0.948
SRT–LTS 50.17 ± 11.98 (6–71) 49.41 ± 12.33 (6–71) 53.58 ± 9.64 (22–69) p = 0.037
SRT–CLTR 39.18 ± 14.28 (2–71) 38.18 ± 14.49 (2–71) 43.71 ± 12.46 (17–69) p = 0.011
SRT–DR 9.27 ± 2.02 (2–12) 9.18 ± 2.07 (2–12) 9.69 ± 1.73 (6–12) p = 0.167
SpRT 21.35 ± 5.34 (8–30) 21.09 ± 5.22 (8–30) 22.35 ± 5.76 (10–30) p = 0.104
SpRT–DR 7.54 ± 2.05 (1–10) 7.43 ± 2.00 (1–10) 8.02 ± 2.18 (3–10) p = 0.036
SDMT 55.89 ± 18.75 (12–110) 55.26 ± 19.54 (12–110) 58.69 ± 14.67 (16–90) p = 0.054
PASAT 39.28 ± 13.01 (8–60) 39.57 ± 12.88 (8–60) 38.00 ± 13.66 (13–58) p = 0.452
WLG 24.25 ± 7.04 (5–42) 23.88 ± 7.00 (5–42) 25.85 ± 7.07 (12–42) p = 0.093

BRB–N: brief repeatable battery of neuropsychological tests; CLTR: consistent long term retrieval; DR: delayed recall; LTS: long term storage; MMSE: mini mental state examination; PASAT: paced auditory serial addition test; SDMT: symbol digit modalities test; SpRT: spatial recall test; SRT: selective reminding test; WLG: word list generation.

*Group comparisons were performed with Mann–Whitney U tests, except for percentage distribution, where Fisher's exact test was used.

All neurologists and neuropsychologists who administered the battery had participated in an initial common training session in order to standardize the criteria of administration, data recording and scoring procedures. All individuals underwent testing during daytime and in a quiet room. The administration time of the BRB–N was around 30 minutes.

BRB–N subtests and scores

Verbal memory was tested with the Selective Reminding Test (SRT). The test consists of presenting orally to the individual a list of 12 unrelated words for up to six trials, at the rate of one word per two seconds. After the list has been presented, the participant is instructed to recall as many words as possible in any sequence. Words that are not recalled are repeated (reminded) by the examiner on the next trial, and the individual is requested to recall and say again the whole list of 12 words. The scoring system consists of a Long Term Storage (LTS) and a Consistent Long Term Retrieval (CLTR). If a word is recalled on two successive trials without a reminder, it is assumed to have entered LTS on the first of these two trials. With or without retrieval of this word on the subsequent trials, it is scored as LTS on all following trials. The CLTR score refers to consistent recall of this same word on all succeeding trials until the last one. After about 20 minutes, the subject tries to recall and replicate the list again (delayed recall)12,13. Versions A and B have a different list of 12 words. The scores employed are: LTS (total number of words in LTS in all six trials), CLTR (total number of words in CLTR in all six trials) and the number of correct words after delayed recall.

Visuospatial memory was tested with the 10/36 Spatial Recall Test (SpRT), which consists of a test where participants are shown, for a period of 10 seconds, a 6 x 6 checkerboard with 10 pieces placed in specific locations. Shortly afterwards, the participant is given 10 pieces and asked to replicate the pattern on a blank checkerboard. The test is repeated three times. After about 20 minutes, the participant tries to recall and replicate the pattern again (delayed recall)13. Versions A and B differ in the spatial configuration of the 10 pieces. There is an immediate recall score, equal to the total number of correct responses for the three trials, and a delayed recall score.

Sustained attention and information processing speed were tested with the Symbol Digit Modalities Test (SDMT). Only the SDMT oral version was employed in this study. The SDMT consists of a key with two rows, with nine stimulus symbols in the upper row and matched numbers (1–9) in the row below it. The task sequence consists of a series of symbols in random order, each with a blank space underneath, and the participants are asked to respond with the number that matches each symbol as rapidly as possible in 90 seconds, after completing a 10–item practice trial. The score of the test is the number of correct responses completed within the time limit, with the maximum score of 11013,14. Each alternate form (A and B) features a new arrangement of the symbol–digit pairings in the key (the same symbols matched in a different order).

Working memory and information processing speed were tested with the Paced Auditory Serial Addition Test. In this test, participants listen to an audio recording of 61 digits where single digits are presented every three seconds and the participant must add each new digit to the one immediately prior to it (and not to the result of the previous sum), providing each sum orally as quickly as possible. The test score is the number of correct sums given (out of 60 possible) in each trial. Versions A and B differ only in the sequence of digit presentation13,15.

Verbal fluency was tested with the Word List Generation. This is a semantic verbal fluency test evaluating the spontaneous production of names of a given category (i.e. vegetables and fruits in version A; animals in version B) within 90 seconds. The score is the number of correct names in either category13. Inflections of the same word (cow, cows) or its perseverations are counted as one response, and words not belonging to the category are not counted.

Translation of the BRB–N stimuli and instructions was performed by three translators, fluent in both Portuguese and English. At first, a native English speaker translated from English to Portuguese. Then stimuli and instructions were back–translated into English and then into Portuguese, independently by two neurologists, both experts in neuropsychology and fluent in English, and members of the Neuropsychology and Neurolinguistics Unit of the University of Campinas. The final version was piloted in 10 healthy participants and 10 MS patients for final adaptation of sociocultural and linguistic aspects.

Statistical analysis

Statistical analysis was performed with the Statistical Package for the Social Sciences (v21.0, SPSS Inc, Chicago). The level of statistical significance was p < 0.05. Normality tests for each variable were performed with the Shapiro–Wilk test and q–q plots. Group comparisons on clinical and cognitive tests were performed with Mann–Whitney U tests, except for percentage distribution, where Fisher's exact test was used. Continuous norms were calculated with multiple–regression analysis as previously described11,16,17. Briefly, healthy participants' raw scores on each neuropsychological measure were first converted to scaled scores based on cumulative distribution (M = 10, SD = 3). Scaled scores were then regressed on age, education and sex (male = 2; female = 1). We tested for multicollinearity among predictor variables. In addition, we inspected the distributions of residuals for normality by analyzing q–q plots. Plots of regression–standardized residual predicted values showed that variance of the residuals was constant (homoscedasticity). Given the small sample size of version B, norms (discrete and continuous) were calculated for version A only.

RESULTS

There were no significant differences of demographic variables between the participants in whom versions A or B were administered (Table 1). Mean scores were higher in version B for the SRT (LTS and CLTR) and SpRT (delayed recall only) (Table 1).

Mean raw scores and the 5th percentile for each neuropsychological measure in version A are shown in Table 2, stratified by age and educational level.

Table 2 BRB–N scores stratified by age and educational level (expressed as mean values and standard deviation). 

Age BRB-N subtests Educational level
6–11 years ≥ 12 years
Mean ± SD 5th percentile Mean ± SD 5th percentile
18–30 SRT- LTS 50.60 ± 8.97 34.2 54.74 ± 10.54 36.5
SRT–CLTR 36.65 ± 11.68 18.2 45.68 ± 14.42 23.9
SpRT 21.58 ± 5.55 10.0 23.62 ± 4.10 15.0
SDMT 59.88 ± 25.04 38.0 64.82 ± 14.45 44.65
PASAT 35.11 ± 15.01 10.0 41.63 ± 11.30 23.8
SRT–DR 9.52 ± 1.54 6.0 10.15 ± 1.54 6.95
SpRT–DR 7.76 ± 1.98 3.0 8.50 ± 1.47 5.95
WLG 17.41 ± 6.04 5.0 27.00 ± 6.58 16.75
31–45 SRT- LTS 48.04 ± 12.50 20.45 50.06 ± 11.01 31.5
SRT–CLTR 34.77 ± 12.80 13.9 41.15 ± 14.39 19.25
SpRT 20.00 ± 5.39 9.2 22.25 ± 3.92 15.2
SDMT 46.57 ± 18.27 26.1 60.51 ± 16.30 38.6
PASAT 33.38 ± 11.78 14.1 47.51 ± 8.61 29.0
SRT–DR 7.95 ± 2.76 2.3 9.46 ± 1.80 6.0
SpRT–DR 7.14 ± 2.61 1.2 7.53 ± 2.02 4.0
WLG 23.23 ± 5.61 9.8 27.53 ± 6.13 20.0
46–66 SRT- LTS 41.63 ± 15.02 15.6 46.83 ± 10.67 28.75
SRT–CLTR 29.53 ± 14.21 5.1 32.95 ± 10.64 18.25
SpRT 17.58 ± 3.87 9.4 18.78 ± 5.96 8.2
SDMT 36.63 ± 12.89 14.2 53.08 ± 14.60 34.6
PASAT 29.43 ± 11.70 11.1 42.60 ± 10.74 15.2
SRT–DR 8.19 ± 2.43 3.1 8.86 ± 1.68 6.2
SpRT–DR 6.12 ± 1.55 4.0 6.86 ± 2.05 2.4
WLG 19.43 ± 5.82 10.1 23.47 ± 6.14 16.0

BRB–N: brief repeatable battery of neuropsychological tests; CLTR: consistent long term retrieval; DR: delayed recall; LTS: long term storage; PASAT: paced auditory serial addition test; SDMT: symbol digit modalities test; SpRT: spatial recall test; SRT: selective reminding test; WLG: word list generation.

Healthy participants' raw scores in version A were converted to scaled scores (Table 3). These raw–to–scale–score conversions can be applied to MS patients. Scaled scores were regressed on age, sex and education yielding equations that can be used to calculate predicted scores (Table 4). Age was a significant predictor for all neuropsychological measures, except verbal fluency, while sex was the opposite.

Table 3 BRB–N raw scores to scaled score conversions. 

Scaled score Raw score
SRT SpRT SDMT PASAT WLG
LTS CLTR DR Total DR
1 < 6 1 1 < 12 < 8 < 5
2 06-09 2–3 2 < 8 1 12–13 8 5–6
3 10–20 4–6 3 8 2 14–16 9–11 7–9
4 21–22 7–13 4 9–10 3 17–22 12–14 10–11
5 23–26 14–18 5 11–12 23–27 15–18 12–13
6 27–33 19–21 13–14 4 28–35 19–23 14–15
7 34–39 22–25 6 15–16 5 36–39 24–26 16–17
8 40–43 26–29 7 17–18 40–45 27–31 18–19
9 44–48 30–33 8 19 6 46–50 32–37 20–21
10 49–52 34–38 9 20–22 7 51–55 38–43 22–23
11 53–56 39–44 23 8 56–62 44–46 24–26
12 57–60 45–50 10 24–25 9 63–68 47–51 27–30
13 61–62 51–57 11 26 69–75 52–55 31–32
14 63–64 58–60 27 10 76–88 56 33–34
15 65–66 61–64 12 28 89–101 57–58 35–36
16 67 65–67 102–103 59–60 37–38
17 68–69 68–69 29 104–110 39–40
18 70–72 70–72 30 41–42
19 > 42

BRB–N: brief repeatable battery of neuropsychological tests; CLTR: consistent long term retrieval; DR: delayed recall; LTS: long term storage; PASAT: paced auditory serial addition test; SDMT: symbol digit modalities test; SpRT: spatial recall test; SRT: selective reminding test; WLG: word list generation.

Table 4 Regression models for BRB–N measures and raw residuals standard deviation. 

Measure Predictors B t Significance CI95% R square SD residual
Lower bound Upper bound
SRT – LTS (Constant) 10.355 10.064 < 0.001 8.32 12.38 0.146 2.885
Sex 0.042 0.103 0.917 -0.76 -0.76
Age -0.064 -4.269 < 0.001 -0.09 -0.03
Education 0.179 3.129 0.002 0.06 0.29
SRT – CLTR (Constant) 9.953 10.138 < 0.001 8.01 11.88 0.194 2.753
Sex -0.410 -1.056 0.291 -1.17 0.35
Age -0.064 -4.470 < 0.001 -0.09 -0.03
Education 0.227 4.153 < 0.001 0.11 0.33
SpRT (Constant) 12.553 12.385 < 0.001 10.55 14.55 0.221 2.774
Sex 0.665 1.681 0.094 -0.11 1.44
Age -0.098 -6.670 < 0.001 -0.12 -0.06
Education 0.095 1.711 0.088 -0.01 0.20
SDMT (Constant) 9.669 11.292 < 0.001 7.98 11.35 0.372 2.343
Sex 0.092 0.275 0.783 -0.56 0.75
Age -0.090 -7.203 < 0.001 -0.11 -0.06
Education 0.297 6.294 < 0.001 0.20 0.39
PASAT (Constant) 7.047 7.187 < 0.001 5.11 8.98 0.197 2.684
Sex 0.667 1.742 0.083 -0.08 1.42
Age -0.031 -2.201 0.028 -0.05 -0.003
Education 0.314 5.811 < 0.001 0.20 0.42
SRT – DR (Constant) 10.642 11.285 < 0.001 8.78 12.50 0.171 2.581
Sex -0.725 -1.967 0.050 -1.45 0.001
Age -0.051 -3.718 < 0.001 -0.07 -0.02
Education 0.193 3.714 < 0.001 0.09 0.29
SpRT – DR (Constant) 11.300 12.948 < 0.001 9.57 13.02 0.213 2.388
Sex 0.128 0.376 0.706 -0.54 0.80
Age -0.073 -5.776 < 0.001 -0.09 -0.04
Education 0.149 3.108 0.002 0.05 0.24
WLG (Constant) 6.499 6.912 < 0.001 4.64 8.35 0.264 2.573
Sex -0.786 -2.141 0.033 -1.51 -0.06
Age -0.018 -2.141 0.178 -0.04 0.008
Education 0.389 7.506 < 0.001 0.28 0.49

BRB–N: brief repeatable battery of neuropsychological tests; CLTR: consistent long term retrieval; DR: delayed recall; LTS: long term storage; PASAT: paced auditory serial addition test; SDMT: symbol digit modalities test; SpRT: spatial recall test; SRT: selective reminding test; WLG: word list generation.

DISCUSSION

Although a remarkable increase in cognitive–related MS research has been observed in the last years, there is still a considerable gap in applying this knowledge into clinical practice. This may be due to several reasons including absence of normative data, time restriction for the use of extensive neuropsychological batteries and lack of established therapeutic options to overcome cognitive deficits. The widely–used MMSE has a low sensitivity to detect MS cognitive dysfunction. Therefore, Rao et al. developed the BRB–N, a neuropsychological battery with better sensitivity and specificity3,4. Accordingly, we aimed to include a fairly significant number of individuals in the present study, representative of areas in Brazil where MS is more prevalent and where most MS centers are located, and we were able to provide normative data for the version A of the battery.

The BRB–N has been widely used in MS research worldwide, including South America7,8,9. In Brazil, it was initially employed in the context of the CogniCIS, a multinational study evaluating cognitive performance in clinically isolated syndrome18. A further study by Brooks et al., has shown that the BRB–N is feasible for assessing cognition of MS patients in the daily clinic19. Moreover, this battery showed strong discriminating power between patients and controls in a Brazilian sample, insofar as large effect sizes were observed20. However, even though this battery had already been validated in the Portuguese language, Brazilian normative values were not yet available for these studies and analysis had to rely on adequate control groups6. Nevertheless, small control groups have potential limitations and one cannot adequately match all possible demographic variables.

We aimed to provide both discrete and continuous normative data. Discrete norms have some limitations and have been subject to criticism over the last years. For example, Morgan et al. have shown that the use of demographically–adjusted T–scores significantly improved sensitivity for discriminating impaired versus neurocognitively normal individuals in comparison to the raw cut score21. Similarly, Parmenter et al. demonstrated that using a continuous norm approach, higher rates of impairment were identified compared to standard norms for many of the minimal assessment of cognitive function in multiple sclerosis measures17, a neuropsychological battery developed with strong psychometrics properties for use in MS research, but with a considerably longer administration time compared to the BRB–N1,22. Recently, the shorter Brief International Cognitive Assessment for Multiple Sclerosis battery was validated in Brazil, but neither continuous nor discrete norms were available in our population10. Several authors have advocated the use of continuous norms based on multiple regression analysis accounting for demographic factors11,16,17. However, whether continuous norms are the most appropriate for answering all research and clinical questions is still a matter of discussion. For example, Silverberg and Millis showed that discrete norms may have worked better in determining whether the patients' cognitive abilities were sufficient for the demands of universal functional tasks, such as activities of daily living23.

Although, in our sample, raw scores obtained in all BRB–N measures were not normally distributed, we opted to also provide discrete norms stratified by age and education for ease of clinical use on a daily basis and to address selected questions. However, use of these norms should be made with the awareness of potential limitations, as discussed above and illustrated in the following case.

For example, consider a 32–year–old female MS patient with 11 years of education who scores 30 in the SDMT. According to discrete norms in Table 2, her score is within her age group mean–1 SD, and thus could be considered normal. However, her raw score corresponds to a scaled score of 6, according to Table 3 and her predicted scaled score on the SDMT is 10.15, based on the regression equation expressed in Table 4 [9.669 + 1(0.092) + 32(−0.090) + 11(0.297)]. We then divide the difference between her actual and predicted scaled score (6−10.15 =−4.15) by the standard deviation of the residual (2.343), obtaining a z score of–1.77. This value equals a T score of 32 and is considered impaired using an operational definition of “impairment” of either 35 or 40, as suggested by some authors11,17.

Regarding the BRB–N version B scores, we found them slightly higher only for the verbal (total learning) and visuospatial memory (delayed recall) tests when compared to version A and these findings were not attributable to gender, education or age differences. Although our sample size for the BRB–N version B test was relatively small, it is interesting that previous studies have found similar results for the SpRT (delayed recall) but not for the verbal memory test7,8. For example, Amato et al. have found higher scores in all BRB–N version B subtests except in the SRT total learning (CLTR and LTS)8. Nevertheless, in order to be used longitudinally with reliable results, both versions should be equivalent. Thus, even though the higher scores of the SRT version B may have been influenced by its smaller sample size, we need further studies comparing SRT versions A and B with equivalent words regarding length, frequency of use, and category membership. Subsequently, further research with more individuals performing version B will be able to address whether this version is suitable for longitudinal usage of the battery.

Also in line with previous results, we found that demographic characteristics, and especially age and education, were significantly related to all neuropsychological measures, except the Paced Auditory Serial Addition Test, where there was a trend for only sex and age7,8. Sex was only associated with the Word List Generation score. These findings reinforce the need for adjusting raw scores for demographic variables when interpreting neuropsychological test scores.

In conclusion, our normative data allow a more widespread use of the BRB–N in clinical practice and research, providing both discrete and continuous norms. However, discrete norms should be used with caution, and demographically–adjusted scores are generally preferable when interpreting neuropsychological test scores.

Support: This work was partially supported by Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP) (postdoctoral grant number 2016/04270–0).

References

1. Benedict RH, Zivadinov R. Risk factors for and management of cognitive dysfunction in multiple sclerosis. Nat Rev Neurol. 2011 May;7(6):332-42. https://doi.org/10.1038/nrneurol.2011.61Links ]

2. D'Anna L, Lorenzut S, Perelli A, Zanchettin B, Valente M, Gigli GL. The contribution of assessing cognitive impairment in radiologicallyisolated syndrome (RIS): a single case report follow-up study. Mult Scler. 2014 Dec;20(14):1912-5. https://doi.org/10.1177/1352458514523693Links ]

3. Beatty WW, Goodkin DE. Screening for cognitive impairment in multiple sclerosis. An evaluation of the Mini-Mental State Examination. Arch Neurol. 1990 Mar;47(3):297-301. https://doi.org/10.1001/archneur.1990.00530030069018Links ]

4. Rao SM, Leo GJ, Bernardin L, Unverzagt F. Cognitive dysfunction in multiple sclerosis. I. Frequency, patterns, and prediction. Neurology. 1991 May;41(5):685-91. https://doi.org/10.1212/WNL.41.5.685Links ]

5. Langdon DW, Amato MP, Boringa J, Brochet B, Foley F, Fredrikson S et al. Recommendations for a Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Mult Scler. 2012 Jun;18(6):891-8. https://doi.org/10.1177/1352458511431076Links ]

6. Neves MR, Passos AM, Ferreira A, Sousa C, Sá A, Sá MJ. BRBN-T validation: adaptation of the Selective Reminding Test and Word List Generation. Arq Neuropsiquiatr. 2015 0ct;73(10):867-72. https://doi.org/10.1590/0004-282X20150134Links ]

7. Boringa JB, Lazeron RH, Reuling IE, Adèr HJ, Pfennings L, Lindeboom J et al. The brief repeatable battery of neuropsychological tests: normative values allow application in multiple sclerosis clinical practice. Mult Scler. 2001 Aug;7(4):263-7. https://doi.org/10.1191/135245801680209385Links ]

8. Amato MP, Portaccio E, Goretti B, Zipoli V, Ricchiuti L, De Caro MF et al. The Rao's Brief Repeatable Battery and Stroop Test: normative values with age, education and gender corrections in an Italian population. Mult Scler. 2006 Dec;12(6):787-93. https://doi.org/10.1177/1352458506070933Links ]

9. Cáceres F, Vanotti S, Rao S, Workgroup RE. Epidemiological characteristics of cognitive impairment of multiple sclerosis patients in a Latin American country. J Clin Exp Neuropsychol. 2011 Dec;33(10):1094-8. https://doi.org/10.1080/13803395.2011.603690Links ]

10. Spedo CT, Frndak SE, Marques VD, Foss MP, Pereira DA, Carvalho LF et al. Cross-cultural Adaptation, Reliability, and Validity of the BICAMS in Brazil. Clin Neuropsychol. 2015;29(6):836-46. https://doi.org/10.1080/13854046.2015.1093173Links ]

11. Casaletto KB, Umlauf A, Beaumont J, Gershon R, Slotkin J, Akshoomoff N et al. Demographically Corrected Normative Standards for the English Version of the NIH Toolbox Cognition Battery. J Int Neuropsychol Soc. 2015 May;21(5):378-91. https://doi.org/10.1017/S1355617715000351Links ]

12. Buschke H, Fuld PA. Evaluating storage, retention, and retrieval in disordered memory and learning. Neurology. 1974 Nov;24(11):1019-25. https://doi.org/10.1212/WNL.24.11.1019Links ]

13. Rao SM. A manual for the brief repeatable battery of neuropsychological tests in multiple sclerosis. New York: National Multiple Sclerosis Society; 1990. [ Links ]

14. Smith A. Symbol Digit Modalities Test (SDMT): manual (revised). Los Angeles: Western Psychological Services; 1982. [ Links ]

15. Gronwall DM. Paced auditory serial-addition task: a measure of recovery from concussion. Percept Mot Skills. 1977 Apr;44(2):367-73. https://doi.org/10.2466/pms.1977.44.2.367PMID:866038Links ]

16. Testa SM, Winicki JM, Pearlson GD, Gordon B, Schretlen DJ. Accounting for estimated IQ in neuropsychological test performance with regression-based techniques. J Int Neuropsychol Soc. 2009 Nov;15(6):1012-22. https://doi.org/10.1017/S1355617709990713Links ]

17. Parmenter BA, Testa SM, Schretlen DJ, Weinstock-Guttman B, Benedict RH. The utility of regression-based norms in interpreting the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). J Int Neuropsychol Soc. 2010 Jan;16(1):6-16. https://doi.org/10.1017/S1355617709990750Links ]

18. Fredrikson S, Wicklein EM, Prayoonwiwat N, Beckmann K, Scherer P, Langdon D. Two-year data from CogniCIS, a multinational longitudinal study on cognition and patient-reported outcomes parameters in clinically isolated syndrome suggestive of multiple sclerosis: demographic and neurological characteristics of the study cohort. Mult Scler. 2010;16 suppl 10:S208. [ Links ]

19. Brooks JB, Borela MC, Fragoso YD. Assessment of cognition using the Rao's Brief Repeatable Battery of Neuropsychological Tests on a group of Brazilian patients with multiple sclerosis. Arq Neuropsiquiatr. 2011 Dec;69(6):887-91. https://doi.org/10.1590/S0004-282X2011000700007Links ]

20. Damasceno A, Damasceno BP, Cendes F. Subclinical MRI disease activity influences cognitive performance in MS patients. Mult Scler Relat Disord. 2015 Mar;4(2):137-43. https://doi.org/10.1016/j.msard.2015.01.006Links ]

21. Morgan EE, Woods SP, Scott JC, Childers M, Beck JM, Ellis RJ et al. Predictive validity of demographically adjusted normative standards for the HIV Dementia Scale. J Clin Exp Neuropsychol. 2008 Jan;30(1):83-90. https://doi.org/10.1080/13803390701233865Links ]

22. Strober L, Englert J, Munschauer F, Weinstock-Guttman B, Rao S, Benedict RH. Sensitivity of conventional memory tests in multiple sclerosis: comparing the Rao Brief Repeatable Neuropsychological Battery and the Minimal Assessment of Cognitive Function in MS. Mult Scler. 2009 Sep;15(9):1077-84. https://doi.org/10.1177/1352458509106615Links ]

23. Silverberg ND, Millis SR. Impairment versus deficiency in neuropsychological assessment: implications for ecological validity. J Int Neuropsychol Soc. 2009 Jan;15(1):94-102. https://doi.org/10.1017/S1355617708090139Links ]

Received: September 16, 2017; Revised: November 22, 2017; Accepted: December 12, 2017

Correspondence: Alfredo Damasceno; Departamento de Neurologia, FCM–UNICAMP; Endereço completo; Rua Tessália Vieira de Camargo, 126; 13083–970 Campinas SP, Brasil; alfredodamasceno@hotmail.com

Conflict of interest: There is no conflict of interest to declare.

Creative Commons License This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.