Accuracy of the semantic fluency test to separate healthy old people from patients with Alzheimer’s disease in a low education population

ABSTRACT Objective Evaluate the accuracy of two semantic categories of the verbal fluency test (supermarket and animal categories) to separate healthy elderly individuals and lower educated Alzheimer’s disease patients. Methods We evaluated 69 older adults with less than 5 years of schooling, consisting of 31 healthy elderly, and 38 patients diagnosed with Alzheimer’s disease. Semantic verbal fluency was evaluated using the animal and supermarket categories. Mann-Whitney U and Independent t Tests were used to compare the two groups, and the diagnostic accuracy of the tests was analyzed by sensitivity, specificity, likelihood ratio’s, and the Area Under the Curve (AUC). Results We found a significant difference between the healthy older and Alzheimer’s disease groups, in both, animal (p = 0.014) and supermarket verbal fluency (p < 0.001). The supermarket category showed better overall diagnostic accuracy (AUC = 0.840, 95% CI = 0.746-0.933; p < 0.001) compared to the animal category (AUC = 0.671, 95% CI = 0.543-0.800; p = 0.014). Conclusion The supermarket category of semantic verbal fluency provides better accuracy than the animal category for the identification of dementia in a Brazilian elderly population with low educational level.


INTRODUCTION
According to International Alzheimer's Disease¹, the highest prevalence of Alzheimer's Disease (AD) is the highest number of cases occurring in Low and Middle-Income Countries (LMICs), reaching 58% of the elderly population, i.e., people over 60-79 years old 1 . Among the diagnostic evaluation methods tests and examinations to diagnose Alzheimer's disease methods the neuropsychological assessment is used to determine the cognitive characteristics of the individual². The neuropsychological assessment may provide differentiation of the types of dementia, together with neuroimaging and cerebrospinal biomarker exams³. However, educational level is an issue to take into consideration in most neuropsychological assessments 4 . Considering that illiteracy and low education levels are very common in LMICs 5 , the use of neuropsychological tests influenced by literacy may be an even greater problem in detecting cognitive decline in this population.
It has been reported that higher educated individuals tend to perform better in neuropsychological assessments because of more efficient and dense neural networks, which compensate for cognitive losses related to the aging process 6 . Therefore, researchers are increasingly searching for strategies to reduce the influence of socio-cultural and educational factors on neuropsychological testing in individuals who are illiterate or have low educational levels. Cut-off points, according to literacy, have been required to improve the accuracy of neuropsychological evaluations in these populations 3 .
As an example of some tests that apply cut-offs based on level of education, the Mini-Mental State Examination (MMSE) is the gold standard 7,8 , while other tests, such as the Cambridge Cognitive Examination (CAMCOG) 9 , the Alzheimer's Disease Assessment Scale -Cognitive Subscale -Cognitive Subscale (ADAS-Cog) 10 and the Montreal Cognitive Assessment (MoCA) 11 , are based on specific averages and scores for the various educational ranges. The MoCA is a brief screening tool designed for screening patients with Mild Cognitive Impairment (MCI), and an additional point is given to individuals with 12 or less years of schooling to correct the educational effects 12 . All these tests have been applied in the Brazilian population and confirm the influence, even if minimal, of schooling on the results.
Cognitive functions evaluated by Verbal Fluency (VF) tests include language, semantic declarative memory, storage capacity, retrieval and executive function 13 . VF tests are widely used in the evaluation of the elderly, as they are sensitive when used for the assessment of individuals with pathologies such as frontotemporal dementia, AD, schizophrenia, and depression 12,14 . The most common category used for semantic VF is the animal category. However, several other categories have also been adopted are in use. da Silva et al. 15 verified an interaction between level of illiteracy and semantic categories, showing that the animal category was influenced by education 14 . Brucki and Rocha 16 also verified a strong influence of education in VF, and the performance was not associated with age or gender. The authors pointed out the influence of environment in animal VF, showing that the most commonly mentioned animals were "dog" and "horse" 16 .
Nielsen and Waldemar 17 investigated the effect of literacy on semantic VF in an immigrant population, considering the ecological relevance of the semantic category. They compared animals and supermarket categories of VF and showed that the effect of literacy in semantic VF was reduced in the supermarket category, compared to that in the animal's category 17 . More recently, Nielsen et al. 18 verified that the acculturation level predicted supermarket item VF in Turkish immigrants, showing that factors like culture and language can affect the performance 17 . Indeed, compared to the variety of animals and supermarket items, there is a greater diversity of supermarket items in the daily lives of the elderly. Therefore, considering the simplicity of the test, VF related to supermarket items (supermarket VF) may have greater ecological validity and sensitivity than animal VF in elderly populations.
In the present study, we evaluated the accuracy of the two semantic categories for the VF test, namely an animal and a supermarket category to identify healthy elderly (HE) and Alzheimer's Disease (AD) patients with a lower educational level. Our hypothesis is that supermarket category of semantic verbal fluency provides better accuracy than the animal category for the identification of dementia in a Brazilian elderly population with low educational level.

Participants
Participants for the present study were drawn from a pool of participants recruited for a larger study in Brazil 19 . More details can be seen on the article previously published 19 . Older adults were recruited for this study according to the following inclusion criteria: individuals (male and female) aged over 60 years; residents of the city of Rio de Janeiro; with a clinical diagnosis of AD and HE without a diagnosis of mental illness. AD patients were recruited from a university center and a military hospital. Clinical diagnosis of AD was made by a psychiatrist or geriatrician, according to DSM-IV and National Institute of Neurological and Communicative Diseases and Stroke/Association of Alzheimer's Disease and Related Disorders (NINCDS-ADRDA), with mild or moderate severity stage according to Clinical Dementia Classification (CDR). Diagnostic investigation for patients with AD included J Bras Psiquiatr. 2020;69(2):82-7 complete blood cell count, platelet count, blood glucose, triglycerides, total cholesterol and fractions, phosphatase, glutamic oxaloacetic transaminase and glutamic pyruvic transaminase, bilirubins, urea, creatinine, total proteins, calcium, free levels of T4, TSH, VDRL and dosage of B12 and folates. Cranial computed tomography or magnetic resonance imaging with or without spectroscopy was also performed. Exclusion criteria were: patients with neurological or psychiatric disorder comorbidities, as well as physical limitations, visual or hearing impairments that prevent cognitive testing. Individuals with depressive symptoms were not excluded from the AD group.
The control group was recruited from a research center on aging, a private hospital, and a military hospital. The control group included individuals without neurological or psychiatric diseases, who were found to be cognitively healthy by clinical evaluation and MMSE 8 .
Only participants with ≤ 4 years of schooling, including illiteracy, were eligible for inclusion in the present study. This study was approved by the Ethics Committee of the Institute of Psychiatry of the Federal University of Rio de Janeiro and of the Naval Hospital Marcílio Dias, and all participants and caregivers provided written informed consent before any procedure took place.

Experimental procedure
All evaluations were carried out in a single visit with the older adults. The participants were informed and agreed about the research procedures before participating in an interview, various scales and a battery of neuropsychological tests. The older adults were assessed using the MMSE 8 and two semantic verbal fluency tests (animal and supermarket categories). The animal VF subjects are asked to name as many animals as possible within one minute. In supermarket VF, subjects are asked to name as many items that can be found or bought in a supermarket, also with a time-limit of one minute 20 . One point is allocated for each different supermarket item. Both main categories and subordinate items of the category are accepted as correct (e.g., fruit, citrus, lemon, each scoring 1 point). Incorrect responses and repetitions were put in brackets on the score sheet 19 . The number of different items named during the task period represented the score in both categories of the VF test 18

Statistical analysis
The normality and homoscedasticity of the sample data were analyzed using the Shapiro-Wilk and Levene tests, respectively. The chi-squared test was used to compare the categorical variables. Mann-Whitney U and independent t-tests were applied to compare variables between groups (HE vs. AD). The associations between semantic VF (supermarket and animal categories) and cognitive and Lawton performance were analyzed using Pearson's correlation or Spearman's correlation tests. A receiver-operating characteristic (ROC) curve was applied in order to examine the diagnostic accuracy of the animal and supermarket VF tests, using the clinical diagnosis of AD as reference standard. Sensitivity, specificity, positive and negative likelihood ratios (LR+ and LR-), and area under the curve (AUC) values were calculated. All statistical analysis was conducted using SPSS ® software version 20.0 (IBM Corporation, New York, USA). P ≤ 0.05 was considered to indicate statistical significance.

Results
In total, 136 elderly individuals were recruited; 67 subjects did not meet the inclusion criteria of having ≤ 4 years of schooling and were excluded. Thus, 61 elderly subjects with lower educational levels (HE = 24 and AD = 37) and 8 illiterates (HE = 7 and AD = 1) were included in the final sample. A flowchart of the inclusion of participants is presented in figure 1.  The demographic characteristics of the groups are shown in table 1. There were significant differences in age (p = 0.027), gender (p < 0.001), schooling (p = 0.008), Lawton (p < 0.001), MMSE score (p < 0.001), and CDR (p = 0.008) between the groups, showing that HE group was more functionally and cognitively preserved than the AD group. There were significant differences between the groups in both the animal (p = 0.014) and supermarket (p < 0.001) VF tests.  Diagnostic accuracy data is presented in table 2. Overall, the supermarket category showed better accuracy (AUC = 0.840, 95% CI = 0.746-0.933) than the animal category (AUC = 0.671, 95% CI = 0.543-0.800) in differentiating between HE and AD individuals with low levels of education. Supermarket VF test scores ≥15 (the optimal cut-point based on the ROC curve) detected AD with a sensitivity of 67.74% and a specificity of 86.84%. Animal VF test scores ≥12 (the optimal cut-point based on the ROC curve) detected AD with a sensitivity of 54.84% and a specificity of 73.68%.
Both animal and supermarket VF showed significant correlations with other cognitive tests. The animal VF test presented a significant correlation with MMSE (r = 0.438, p < 0.001), CDR (r s = -0.379, p = 0.001), and Lawton (r s = 0.306, p = 0.011). However, the correlations were stronger between supermarket category and MMSE (r = 0.632, p < 0.001), CDR (r s = -0.642, p < 0.001), and Lawton (r s = 0.619, p < 0.001), showing that the supermarket category was more associated with Lawton, cognition and severity of dementia. There were no significant correlations between GDS score and animal or supermarket VF tests.

DISCUSSION
The present study aimed to evaluate the discriminatory power of two semantic criteria of VF test (supermarket and animal categories) to separate identify HE from and AD patients with lower educational level. According to our results, the supermarket VF provided the best result. a higher degree of accuracy for the identification of AD individuals with lower educational levels. In the present study, the AUC was 0.671 for the animal category VF and 0.84 for the supermarket category VF, showing that supermarket verbal fluency provides greater accuracy in this population. In a Brazilian study, Brucki and Rocha 16 showed that educational performance was associated with higher fluency scores in the animal category. The cutoffs suggested (supermarket = 15.0 and animal = 12.0) in the present study corroborate previous studies in illiterate healthy subjects, both for animals 15,16 and supermarket 15 categories VF. To our knowledge, this is the first study that investigated animal and supermarket categories of semantic VF to identify people with AD and separate them form healthy older people with low educational level. It can be speculated that the better accuracy of the supermarket categories VF test is related to activities performed by individuals that are independent from relevant socio-cultural characteristics or level of education. This is a relevant ecological categoric test, which is easy to apply and less dependent on education. A recent meta-analysis investigated cognitive screening tools for dementia in LMICs and the authors verified that few instruments were validated in low-literacy settings 5 . The MMSE demonstrated the highest level of accuracy when used in low-literacy cohorts in Spain and Brazil, and the AUC for the MMSE reported in this meta-analysis was 0.853, very similar to the AUC observed in our result for supermarket category VF. However, it is worth mentioning that this test must be applied within a battery of tests, because although it has presented a reasonable result in specificity, its sensitivity was low, thus not allowing its use in isolation in a screening.
The poorer testing performance of the animal VF could be explained by the influence of education in this category of semantic verbal fluency task 24 . In another study, animal category VF was associated with education, but not with age or sex 16 . However, a study of various categories of VF tests, including supermarket items, showed that age had a more significant impact than education on overall VF testing performance 25 . In the present study, stronger correlations were observed between VF and Lawton, especially in the supermarket category. This is expected, considering the association between cognition and education with instrumental activities of daily living (IADL) 26,27 . Indeed, dementia is the leading cause of functional dependence in the elderly and IADL refers to the individual's ability to organize and plan in the environment, be it at home or at the wider community. These activities are related to the most complex actions, such as social participation, which includes using the telephone, driving, using public transportation, and the ability to shop 28 . This last ability in IADL is directly related to the supermarket VF, whereas animal VF is not related to any activity that affects the daily life of the elderly.
We also observed a correlation between MMSE and semantic VF. However, the correlation was stronger for the supermarket category than for the animal category. Although MMSE is influenced by education 23 , it is not merely the number of years of education that is important, but also the quality of the education 29 . Normally, studies determine the levels of education based on the number of years, rather that the quality of the education. This may be an important bias, especially in Brazil, as many elderly people in LMICs have had several years of schooling but are unable to perform more complex assessments or do not have good semantic fluency in terms of words that are not common in their daily lives. Finally, the present study has some limitations that needs to be considered, such as a small sample, controls not matched for age, sex, and education, in addition, the study was carried out in an urban area, being questionable whether a sample in the rural area would have a greater ease with the VF of animals than the VF of supermarket items.

CONCLUSION
In the semantic VF test, the supermarket category provides better diagnostic accuracy than the animal category animal category for the identification of dementia in an illiterate and LE elderly population.