Is the Mini-Mental State Examination the best cognitive screening test for less educated people?

ABSTRACT Mini-Mental State Examination (MMSE) results are strongly influenced by educational level. The Brief Cognitive Screening Battery (BCSB) is an alternative assessment tool that provides more accurate results in individuals with less education. Objective: Our aim was to compare the MMSE and BCSB as screening tests. Methods: The MMSE and BCSB were assessed in 112 participants by two evaluators blind to the other test's result. Participants were classified according to their level of education. The influence of education level was analyzed using the Kruskal-Wallis and multiple comparison tests. Results: Scores of the MMSE (p < 0.0001) and the clock-drawing test (p < 0.0001) were influenced by education level but the delayed recall test score was not (p = 0.0804). The verbal fluency test (p = 0.00035) was influenced only by higher educational levels. It took three minutes less to apply the MMSE than to apply the BCSB (p < 0.0001). Conclusions: These findings suggest that the delayed recall test and the verbal fluency test of the BCSB are better than the MMSE and clock-drawing test as tools for evaluating cognition in people with limited education.

is the most commonly used test 2 . The MMSE has a sensitivity of 62.7% and a specificity of 63.3% when used to detect MCI in healthy individuals 3 . It is important, however, to take into account age, gender and education in the interpretation of MMSE scores, especially in populations with lower educational levels 4,5 .
Scores on the MMSE, the verbal fluency test and the clock-drawing test differ between literate and illiterate individuals without dementia. On the other hand, this difference is not observed on the BCSB delayed recall test scores 8 . The BCSB is a highly accurate method for the diagnosis of Alzheimer's disease in both illiterate and literate individuals 9 .
We hypothesized that education influences the performance of individuals more significantly on the MMSE than on the BCSB.
For the purpose of testing this hypothesis, the major objective of this study was to compare the influence of educational levels on the MMSE and BCSB scores.

METHODS
This was a cross-sectional observational study of 112 healthy individuals aged between 60 and 80 years, who attended outpatient clinics of the Hospital de Clínicas of the Federal University of Parana, located in Curitiba, Brazil. Participants were randomized by a simple randomization technique. Ten patients were chosen per day from a list of patients who attended the outpatient clinics that day. These selected patients were addressed, then were included or not in the study according to the inclusion and exclusion criteria and their acceptance of the consent terms. All the individuals performed both tests and each test was conducted by a different evaluator blind to the other test's results (the MMSE was always conducted by evaluator 1 and the BCSB was always conducted by evaluator 2). We chose this design to minimize measurement bias. Patients attending neurology and psychiatry outpatient clinics were excluded, as were those using barbiturates, benzodiazepines, anticholinesterases, antipsychotics and thyroid hormones, because these criteria could predict the presence of major cognitive declines or neurologic or psychiatric comorbidities that could impact cognitive functioning 10 . Therefore, these patients were excluded, to minimize the risk of including cognitively impaired individuals. However, we did not perform any diagnostic test for the exclusions, because we aimed to mimic the reality of a primary care context in which accurate diagnostic testing might not be available. The study was approved by the Human Research Ethics Committee of the Hospital de Clínicas of the Federal University of Parana.
Participants were categorized on the basis of their education (group 0: illiterate; group 1: 1-4 years of education; group 2: 5-8 years of education; group 3: more than eight years of education). The MMSE was administered to all participants according to the procedure described by Brucki et al. 11 and the BCSB was assessed as described by Vitiello et al. 6 .
Scores for the various components of the BCSB were calculated separately and only the scores for the delayed recall test, clock-drawing test and verbal fluency test were used in the analysis. These tests were chosen from the BCSB because they evaluate important cognitive domains impaired in MCI and dementia, such as memory in the delayed recall test, executive function and language in the verbal fluency test, and visuospatial skills in the clock-drawing test. They were chosen instead of the other tests because they had the highest sensitivity and specificity in the domain they evaluate. If we included all the tests in the analysis, the results would have been redundant and less reliable 12,13 . However, the time taken to apply the whole BCSB was used in the application time analysis.
For the analysis we used cut-off points proposed for the Brazilian population. The cut-off points for the MMSE were as follows: 20 for group 0; 25 for group 1; 26 for group 2; 28 for group 3 11 . The cut-off points for the BCSB tests analyzed in our study were: delayed recall test: 6; verbal fluency test: 13 7 . We did not analyze performance on the clock-drawing test based on cut-off points because participants who refused to perform that test were excluded from its analysis.
Statistical analysis was carried out using R Statistical Software, 2016. Test administration times were compared using the Mann-Whitney test. The effects of educational level were assessed using the Kruskal-Wallis test and pairwise multiple comparisons. Table 1 shows the educational profile of the sample. Eighty participants (71%) were women. Median assessment times for the MMSE and BCSB were respectively 5 and 8 minutes (p < 0.0001). Table 2 presents a group comparison of median scores on the MMSE, delayed test, clock-drawing test and verbal fluency test. Regarding the MMSE scores, all the pairwise comparisons between groups were significant (p < 0.0001), except for the comparison between groups 1 and 2 (p = 0.1189).
Pairwise comparison revealed that the performance of group 3 on the verbal fluency test was different from that of all the other groups: vs. group 0: p = 0.0032; vs. group 1: p = 0.0003; vs. group 2: p = 0.0001. All the other groups had similar verbal fluency test scores: group 0 vs. group 1: p = 0.5596; group 0 vs. group 2: p = 0.9618; group 1 vs. group 2: p = 0.435.
Delayed recall test scores were similar in all groups (p = 0.0804).

DISCUSSION
The median MMSE scores in our sample are comparable to those reported by Brucki et al. 11 : illiterates, MMSE = 20; 1-4 years of education, MMSE = 25; 5-8 years of education, MMSE = 26.5; over eight years of education, MMSE = 28. However, there is some debate about which cut-off points should be used for the MMSE. Brucki et al. 11 suggested that the cut-off points should be analyzed for each individual separately, whereas Bertolucci et al. 14 suggested values of 13 (illiterates), 18 (1-8 years of education) and 26 (more than eight years of education), and Almeida et al. 15 suggested 20 as a cut-off for illiterate individuals. In a study that compared illiterate and literate individuals, the median MMSE scores were 21 and 26.5 for illiterate and literate participants respectively 8 . In summary, the analysis of MMSE scores has substantially changed over time.
Education significantly influences MMSE scores and has been reported to be the most important determinant of variance in performance on the test 13, 16 . Therefore, most of the Brazilian studies on the MMSE performed between 1998 and 2013 used educational criteria to define cut-off points 17 . Our results also suggest that education is a decisive element in MMSE performance. Thus, the lack of standardization for the MMSE, shown by the significant variability of the scores among different educational levels, impairs its validity 17 .
As discussed previously, the weaknesses of the MMSE have gradually become more apparent. Therefore, the urge to provide an assessment of cognition better than the MMSE, which is valid for the entire Brazilian population, became even more evident. The BCSB has become the more appropriate option for that purpose. The tests that make up the BCSB have been studied since 1994. In fact, the accuracy of the selected tests for the BCSB is high, and comparable to that of the MMSE 7 .
Our study suggests that education more significantly influences the MMSE than the BCSB in our population. Our sample showed that the delayed recall test scores did not vary with levels of education. Nitrini et al. 7 reported that illiterate and literate participants, and participants with little education and standard education, obtained similar scores on the BCSB delayed recall test. They concluded that, unlike the MMSE, the delayed recall test is not educationally biased 8 .
The effect of educational level on the verbal fluency test that we observed was due to the performance of the group with more than eight years of education. This is consistent with another study, which reported that individuals with more than eight years of education spoke approximately six more words than individuals with less education 18 . Another study also found a significant difference between the verbal fluency scores of illiterate and literate participants 8 . In our sample, participants with more than eight years of schooling spoke between four and six more words than participants in the other groups. In our sample, educational level influenced performance on the clock-drawing test. Previous studies have also reported that individuals with more education perform better than those with less education 19,20 . It should be noted, however, that our analysis of the clock-drawing test was impaired by the high rate of refusals to perform the test, especially in the illiterate group. In most cases the refusal was due to lack of knowledge and embarrassment about having their drawings evaluated. This indicates a limitation for the clinical use of the clock-drawing test in individuals with less education.
We found that the MMSE took three minutes less to assess than the BCSB. The median assessment time for the  Besides the fact that our sample was selected from outpatient clinics from a tertiary health service, the proportion of each education group was compatible to the profile of users of the Brazilian public health system, as stated by Ribeiro et al. 21 : up to three years of education (groups 0 and 1): 41.9%; four to seven years (group 2): 24%; over eight years (group 3): 34.1%. The small number of illiterates in our sample made the observations in this group less reliable, but that number can be justified by the percentage of illiterate people in the Brazilian population, which was 8% in 2015 according to data from the Brazilian National Institute of Geography and Statistics 22 . Therefore, our sample possibly reflects the reality of the Brazilian population.
The main weaknesses of our study are the small number of illiterates and the sample selected from a tertiary hospital. As stated before, both weaknesses could impair external validity but the proportions of participants in each group were consistent with some data already published. Therefore, we suggest, although we cannot confirm, that our results could be representative of the Brazilian population. Also, we did not carry out any additional testing for illiterates who refused to perform the clock-drawing test, which significantly impaired that analysis.
Further studies are necessary to assess the need for new cut-off points for the clock-drawing and verbal fluency tests and to develop more appropriate tools to assess visuospatial functions in illiterate individuals.