Combining functional scales and cognitive tests in screening for mild cognitive impairment at a university-based memory clinic in Brazil

Combinação de escalas funcionais e testes cognitivos para rastreio de comprometimento cognitivo leve em ambulatório universitário de memória no Brasil

BRIEF REPORT

Combining functional scales and cognitive tests in screening for mild cognitive impairment at a university-based memory clinic in Brazil

Combinação de escalas funcionais e testes cognitivos para rastreio de comprometimento cognitivo leve em ambulatório universitário de memória no Brasil

Izabella Dutra de Abreu; Paula Villela Nunes; Breno Satler Diniz; Orestes Vicente Forlenza

Memory Clinic, Laboratory of Neuroscience (LIM 27), Department of Psychiatry, School of Medicine, Universidade de São Paulo (USP), São Paulo (SP), Brazil

^{Correspondence}

ABSTRACT

OBJECTIVE: To determine the diagnostic accuracy of the Mini-Mental State Examination combined to the Informant Questionnaire on Cognitive Decline in the Elderly for the identification of mild cognitive impairment.

METHOD: 191 elderly subjects were assessed with the Mini-Mental State Examination, and their informants were assessed with the Informant Questionnaire on Cognitive Decline in the Elderly. Subjects were divided into three groups according to their cognitive state (controls: n = 67, mild cognitive impairment: n = 65 and dementia: n = 59), which was ascertained by clinical and neuropsychological evaluation. The diagnostic accuracy of each test in the discrimination of diagnostic groups (mild cognitive impairment vs. controls, mild cognitive impairment vs. dementia and dementia vs. controls) was examined with the aid of ROC curves. We additionally verified if the combination of both tests would increase diagnostic accuracy for mild cognitive impairment and control identification.

RESULTS: The combination of the Mini-Mental State Examination and the Informant Questionnaire on Cognitive Decline in the Elderly scores did not increase the Mini-Mental State Examination diagnostic accuracy in the identification of patients with mild cognitive impairment.

CONCLUSIONS: The present data do not warrant the combination of the Mini-Mental State Examination and the Informant Questionnaire on Cognitive Decline in the Elderly as a sufficient diagnostic tool in the diagnostic screening for mild cognitive impairment.

Descriptors: Geriatric psychiatry; Dementia; Scales; Neuropsychological tests; Diagnosis

RESUMO

OBJETIVO: Avaliar a acurácia diagnóstica do Mini-Exame do Estado Mental combinado ao Questionário do Informante sobre Declínio Cognitivo na identificação de casos de comprometimento cognitivo leve.

MÉTODO: 191 indivíduos idosos foram avaliados com o Mini-Exame do Estado Mental e seus informantes com o Questionário do Informante sobre Declínio Cognitivo. Os indivíduos foram divididos em três grupos de acordo com o estado cognitivo (controles: n = 67; comprometimento cognitivo leve: n = 65; demência: n = 59), segundo avaliação clínica e neuropsicológica. Pela análise de curvas ROC, avaliamos a acurácia diagnóstica de cada teste para diferenciar os grupos diagnósticos (comprometimento cognitivo leve vs. controles; comprometimento cognitivo leve vs. demência; demência vs. controles). Adicionalmente, verificamos se a combinação dos dois testes fortalece a acurácia diagnóstica para a separação dos casos de comprometimento cognitivo leve dos controles normais.

RESULTADOS: A análise combinada do Questionário do Informante sobre Declínio Cognitivo e Mini-Exame do Estado Mental não aumentou a acurácia diagnóstica do Mini-Exame do Estado Mental em predizer a ocorrência de comprometimento cognitivo leve.

CONCLUSÕES: Os dados deste estudo não sustentam a utilidade clínica da combinação dos escores do Mini-Exame do Estado Mental e do Questionário do Informante sobre Declínio Cognitivo no rastreio diagnóstico do comprometimento cognitivo leve.

Descritores: Psicogeriatria; Demência; Escalas; Testes neuropsicólogicos; Diagnóstico

Introduction

The recognition of symptoms indicative of the risk of dementia in older adults is important, given the prospect of the development of drugs with disease-modifying properties, which would be more efficacious if started in the early stages of the disease.¹ At present, the best attempt to characterize such individuals is the definition of mild cognitive impairment (MCI), which refers to subjects with or without subjective cognitive complaints, objective impairment in cognitive assessment (provided scores are corrected for relation to age and educational level), minimal or no evidence of impairment in the performance of daily activities, and not demented.² In most studies, the neuropsychological evaluation remains as the gold standard for diagnosis of MCI. Nonetheless, a comprehensive neuropsychological evaluation is costly, time-consuming, and demands well-trained professionals, thus hindering its widespread use in primary care. Therefore, MCI patients may not be easily recognized in primary clinical settings, delaying the proper identification of subjects with an increased risk of dementia.

Cognitive screening tests can be divided into two broad categories: tests that objectively assess the cognitive performance (e.g. the Mini-Mental State Examination -MMSE, the Clock Drawing Test), and informant-based questionnaires (e.g. the Informant Questionnaire of Cognitive Decline in the Elderly - IQCODE). The MMSE is the most widely used cognitive screening test and has been validated in several languages and cultures for the diagnostic screening of dementia. The IQCODE is an informant-based questionnaire that compares the patient's present ability to perform several activities of the daily living, and tasks that demand intact recent episodic memory and intellectual abilities over a continuum of 10 years. It is composed of 26 items individually scored from 1-"much better" to 5 -"much worse". Intermediate scores of 2 and 4 indicate slightly better or worse, respectively; score of 3 indicates no significant changes in comparison to 10 years ago. The final score is given by the sum of the 26-item scores divided by 26, and it ranges from 1 to 5, with higher scores indicating worse functional status.³ The IQCODE is supposedly not unaffected by education, premorbid intellectual ability, or by proficiency in the culture's dominant language.⁴ On the other hand, because the IQCODE relies on the information provided by a caregiver or a close relative of the patient, scores may be biased by the informant's cognitive and emotional state (such as depression and anxiety) or personality traits, and by the other factors affecting the relationship between the informant and the patient. These factors can both minimize or maximize the actual deficits. The IQCODE has been shown to have a good sensitivity and specificity to identify cases of dementia,⁵ but very few studies have so far addressed the diagnostic properties of the IQCODE in the diagnostic screening of MCI.

Previous reports showed that the association of cognitive screening tests improves the potential of a single test to differentiate individuals with Alzheimer's disease (AD) from unimpaired elderly patients.⁶ Therefore, the aim of this work is to ascertain whether the association of two widely used cognitive screening tests, i.e. the MMSE and the IQCODE, is more accurate than each test alone to discriminate cognitively unimpaired elderly subjects from patients with MCI and AD. We hypothesize that the association of these two screening instruments have a higher accuracy in the identification of cognitive impairment than either test alone.

Method

1. Patients

One hundred and ninety-one elderly patients (67 controls, 65 MCI, and 59 AD) were assessed at a university-based memory clinic from December 2001 to May 2006. Age and educational level for patients and controls are presented in Table 1. Patients were recruited from community sources for the assessment of suspected cognitive decline in São Paulo, Brazil. This study was approved by the Ethical Committee of Hospital das Clínicas University of São Paulo (protocol number: 1032/07), and was performed in accordance with the Helsinki Declaration.

2. Clinical assessment

The cognitive assessment and the diagnostic procedures in this cohort are described in details in previously published studies.⁷ In brief, patients were examined by clinicians specialized in the evaluation of cognitive disorders. The clinical assessment included the Cambridge Mental Disorders of the Elderly Examination (CAMDEX) interview, which yields the Cambridge Cognitive Examination (CAMCOG) and the MMSE scores. The IQCODE was further administered to a close relative or caregiver of each subject. The 21-item Hamilton Depression Scale⁸ was administered to rule out depressive symptoms. Evidence of functional decline was suggested by the informant's or the patient's report of difficulties performing activities of daily living. The neuropsychological evaluation was carried out as the gold standard for the diagnosis of MCI and AD as described in previous studies.⁹ It included the Rivermead Behavioral Memory Test (RBMT), the Fuld Object-Memory Evaluation (FOME), the Trail Making Test (TMT) A and B, the Short Cognitive Test (SKT) and the Wechsler Adult Intelligence Scale-Revised (WAIS-R) Vocabulary and Block Design tests. The neuropsychologists responsible for the assessments were blinded to the MMSE and IQCODE scores when the tests were administered. For the sake of conciseness, only the scores for the IQCODE and the MMSE will be reported in this paper (Table 1).

3. Diagnosis

Clinical diagnoses were reached by consensus at expert multi-disciplinary meetings, taking into account clinical, neuropsychological, laboratorial, and neuroimaging data. The MMSE and IQCODE scores per se were not taken into account for establishing the clinical diagnosis. Dementia was diagnosed according to the DSM-IV criteria.¹⁰ In this study, we included patients that fulfilled the diagnostic criteria for AD according to the NINCDS-ADRDA¹¹ or for MCI according to Petersen et al.,¹² in addition to cognitively unimpaired age-matched controls. Patients with evidence of depressive or anxiety disorders were excluded from the study, as well as patients with other dementia syndromes, such as vascular dementia and other neurodegenerative diseases, diagnosed according to the currently accepted diagnostic criteria.^13-15

4. Statistical analysis

Non-parametric statistics were performed to assess median differences between the diagnostic groups (Kruskall Wallis test), while analysis of variance (ANOVA) was performed to control for the variable age. Backward stepwise logistic regression analysis was performed to assess whether the two tests used together provided any additional information for predicting MCI over that given by either test used alone. Receiving Operating Characteristics (ROC) analyses were performed for each test comparing the diagnostic group in a two-by-two fashion. The accuracy of each test, and of their association, is represented by the area under the curve (AUC). In addition, cutoff values and the respective sensitivity and specificity values were drawn by ROC analyses. ROC curves were also constructed for the combination of the tests for MCI accuracy according to the Mackinnon and Mulligan's weighted sum rule.¹⁶ 'Logit (case)' is the logarithm of the odds of a subject being a case of dementia. The IQCODE and the MMSE are the test score values, and the derivation of these values is influenced by the characteristics of the sample.

Weighted sum formula: Logit (case) = 1.4508 + 1.5819 x IQCODE-0.3119 x MMSE.

All statistical analyses were performed using SPSS v14.0 for Windows (SPSS Inc., Chicago, IL) and statistic significance was set at α = 5%.

Results

ROC curve analysis performed in the subsample of patients with MCI and controls indicated that the cutoff scores that yielded the best diagnostic accuracy were 28 for the MMSE (area under the ROC curve) AUC: 0.70 (p < 0.001); sensitivity, SS: 73.5%; specificity, SP: 61.5%; positive predictive value, PPV: 49.6%; negative predictive value, NPV: 81.8%; 3.11 for the IQCODE, AUC: 0.62 (p < 0.001); SS: 69.2%; SP: 49.3%; PPV: 41.3%; NPV: 75.6%. The weighted sum (MMSE combined with IQCODE) indicated a diagnostic accuracy similar to the MMSE alone, and slightly better than the IQCODE alone [AUC: 0.70 (p < 0.001); SS: 73.8%; SP: 62.7%; PPV: 50.5%; NPV: 82.2%]. Additionally, the cutoff scores that best differentiated patients with dementia from controls in the current sample were 25 for the MMSE [AUC: 0.975 (p < 0.001); SS: 88.7%; SP: 94.8%; PPV: 93.2%; NPV: 91.4%] and 3.3 for the IQCODE [AUC: 0.847 (p < 0.001); SS: 83.6%; SP: 80.6%; PPV: 72.3%; NPV: 84.2%].

In the logistic regression analysis for the MCI and Control group, using the IQCODE and the MMSE co-variables together, the more the IQCODE scores the higher is the risk of having MCI comparing to lower scores (IQCODE: Odds Ratio (OR): 2.34; p: 0.192; Confidence Interval (CI): 0.65-8.44), but p value is not statistically significant and the wide CI gives less credibility to the results. Only the MMSE was statistically significant in this analysis showing that the lower the score, the higher the risk of having MCI (OR: 0.725; p: 0.001; CI: 0.604-0.871).

Discussion

Our results showed that although the MMSE and the IQCODE are adequate instruments for the diagnostic screening for dementia, neither test had a good diagnostic accuracy for identification of cases of MCI. In addition, the combined use of these tests did not increase the aforementioned diagnostic properties. Our results are in accordance with previously published studies in which these tests used alone or in combination in the screening for dementia,¹⁷ but are in disagreement with a recent publication by Isella et al.,¹⁸ who suggested that the IQCODE had a good discriminative power between cognitively unimpaired older adults and patients with MCI. In this study, with similar cutoff values (28 for the MMSE and 3.19 for the IQCODE), both tests had a similar discriminative accuracy, but higher than in the current sample (AUC: 0.86 and 0.84, respectively). Differences in diagnostic procedures and definitions of cognitive decline may account for the differences in the latter and the current study. In the present sample, the diagnosis of MCI was based on the output of a comprehensive neuropsychological assessment, which probably results in the inclusion of patients with milder degrees of cognitive impairment.

This study highlights two important aspects that may have practical implications for the diagnostic work-up of cognitive disorders. Firstly, the combination of the MMSE and the IQCODE does not seem sufficiently accurate for the identification of cases of MCI, although it is a frequent procedure in clinical practice. Secondly, the diagnosis of MCI still depends on the information provided by a comprehensive neuropsychological assessment. The complexity of this procedure is a limitation for the large-scale diagnosis of MCI, particularly in primary care settings. The diagnosis of MCI is undoubtedly associated with an increased risk of developing dementia in the long-term outcome. However, it is by no means a synonymous of any given neurodegenerative condition. From the clinician's perspective, a careful judgment based on detailed clinical and neuropsychological data, in addition to reliable information on the patient's functional status, preferentially corroborated by longitudinal reassessments, is still the best approach for the detection of relevant cognitive impairment, and to estimate the actual risk of the dementia outcome.¹⁹

Conclusion

In summary, the IQCODE, the MMSE, and their combined use had low sensitivity for the screening of MCI patients. Therefore, the development of new cognitive screening strategies are mandatory for the identification of such cases and further analyses, in this population, are underway to evaluate which of the 26 IQCODE questions or clusters of them better characterize the MCI patients and its subtypes.

Acknowledgements

The statistical orientation of Mr Jony Arrais Pinto Júnior of the Institute of Mathematics and Statistics of Universidade de São Paulo (USP) is greatly appreciated.

Submitted: January 12, 2008

Accepted: May 12, 2008

• 1. Dubois B, Feldman HH, Jacova C, Dekosky ST, Barberger-Gateau P, Cummings J, Delacourte A, Galasko D, Gauthier S, Jicha G, Meguro K, O'brien J, Pasquier F, Robert P, Rossor M, Salloway S, Stern Y, Visser PJ, Scheltens P. Research criteria for the diagnosis of Alzheimer's disease: revising the NINCDS-ADRDA criteria. Lancet Neurol 2007 6(8):734-46.
• 2. Winblad B, Palmer K, Kivipelto M, Jelic V, Fratiglioni L, Wahlund LO, Nordberg A, Bäckman L, Albert M, Almkvist O, Arai H, Basun H, Blennow K, de Leon M, DeCarli C, Erkinjuntti T, Giacobini E, Graff C, Hardy J, Jack C, Jorm A, Ritchie K, van Duijn C, Visser P, Petersen RC. Mild cognitive impairment beyond controversies, towards a consensus: report of the International Working Group on Mild Cognitive Impairment. J Intern Med 2004;256(3):240-6.
• 3. Jorm AF, Jacomb PA. The Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE): socio-demographic correlates, reliability, validity and some norms. Psychol Med 1989;19(4):1015-22.
• 4. Jorm AF. The Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE): a review. Int Psychogeriatr 2004;16(3):275-93.
• 5. Jorm AF. Assessment of cognitive impairment and dementia using informant report. Clin Psychol Rev 1996;16:51-73.
• 6. Bustamante E, Bottino C, Lopes M, Azevedo D, Hototian SR, Litovoc J, Jacob Filho W. Instrumentos combinados na avaliação de demência em idosos: resultados preliminares. Arq Neuropsiquiatr 2003;61(3):601-6.
• 7. Nunes PV, Diniz BSO, Radanovic M, Dutra I, Borelli D, Yasuda M, Forlenza OV. CAMCOG as a screening tool for diagnosis of Mild Cognitive Impairment and Dementia in a Brazilian clinical sample of moderate to high education. Int J Geriatr Psychiatry. 2008; [Epub ahead of print]
• 8. Dratcu L, Ribeiro LC, Calil HM. Escalas de avaliação da depressão e sua utilidade clínica: Hamilton, Montgomery Åsberg e visual análoga do humor. Rev Ass Bras Psiq 1985;7(1):59-65.
• 9. Flaks MK, Yassuda MS, Regina AC, Cid CG, Camargo CH, Gattaz WF, Forlenza OV. The Short Test (SKT) -cognitive performance: a preliminary study of its psychometric properties in Brazil. Int Psychogeriatr. 2006;18(1):121-33.
• ¹⁰
  American Psychiatric Association. Diagnostic and statistic manual of mental disorders 4th ed. Washington (DC): American Psychiatric Association; 1994.
• 11. McKhaan G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimer's disease: report of the NINCDS-ADRDA work group under the auspices of Department of Health and Human Services Task Force on Alzheimer's disease. Neurology 1984;34(7):939-44.
• 12. Petersen RC, Smith GE, Waring SC, Ivnik RJ, Tangalos EG, Kokmen E. Mild cognitive impairment: clinical characterization and outcome. Arch Neurol 1999;56(3):303-8.
• 13. Román GC. Defining dementia: clinical criteria for the diagnosis of vascular dementia. Acta Neurol Scand Suppl 2002;178:6-9.
• 14. McKeith IG, Galasko D, Kosak K, Perry EK, Dickson DW, Hansen LA, Salmon DP, Lowe J, Mirra SS, Byrne EJ, Lennox G, Quinn NP, Edwardson JA, Ince PG, Bergeron C, Burns A, Miller BL, Lovestone S, Collerton D, Jansen EN, Ballard C, de Vos RA, Wilcock GK, Jellinger KA, Perry RH. Consensus guidelines for the clinical and pathologic diagnosis of Dementia with Lewy Bodies (DLB): report of the consortium on DLB international workshop. Neurology 1996;47(5):1113-24.
• 15. Hodges JR, Miller B. The classification, genetics and neuropathology of frontotemporal dementia. Introduction to the special topic papers: Part I. Neurocase 2001;7(1):31-5.
• 16. Mackinnon A, Mulligan R. Combining cognitive testing and informant report to increase accuracy in screening for dementia. Am J Psychiatry 1998;155(11):1529-35.
• 17. Knafelc R, Lo Giudice D, Harrigan S, Cook R, Flicker L, Mackinnon A, Ames D. The combination of cognitive testing and an informant questionnaire in screening for dementia. Age Ageing 2003;32(5):541-7.
• 18. Isella V, Villa L, Russo A, Regazzoni R, Ferrarese C, Appollonio IM. Discriminative and predictive power of an informant report in mild cognitive impairment. J Neurol Neurosurg Psychiatry 2006;77(2):166-71.
• 19. Portet F, Ousset PJ, Visser PJ, Nobili F, Scheltens P, Vellas B, Touchan J; MCI Working Group of the European Consortium on Alzheimer's Disease. Mild cognitive impairment (MCI) in medical practice: a critical review of the concept and new diagnostic procedure. Report of the MCI Working Group of the European Consortium on Alzheimer's Disease. J Neurol Neurosurg Psychiatry 2006;77(6):714-8.

Publication Dates

• Publication in this collection
  17 Dec 2008
• Date of issue
  Dec 2008

History

• Accepted
  12 May 2008
• Received
  12 Jan 2008