The Cognitive Telephone Screening Instrument (COGTEL): a reliable and valid tool for the assessment of cognitive functioning in the Brazilian elderly

Abstract Objectives: To study the reliability/stability of the Cognitive Telephone Screening Instrument (COGTEL) for the assessment of cognitive functions, and to investigate the concurrent validity (that is, the relationship between the COGTEL scores and external variables, such as level of education and MMSE results) in a pilot study of elderly persons residing in the community in the municipal regions of Apuí, Fonte Boa and Manaus (Amazonas, Brazil). Method: This pilot study included 90 elderly persons (29 men and 61 women) aged 60-85 years of age [68.2 (± 6.7)]. The COGTEL, the MMSE and socio-economic survey were applied in the form of two interviews, a week apart and under the same conditions. Results: The test-retest intraclass correlation coefficient of the COGTEL total score (and respective six subtests), MMSE and educational level ranged from acceptable to high (0.708 < R <0.946). There was a strong positive correlation between the total score of the COGTEL with the MMSE (r =0.682; p <0.001), as well as with educational level (r =0.604; p <0.001). Conclusion: This study presents preliminary evidence of the reliability/stability and concurrent validity of the COGTEL in the evaluation of cognitive functions in elderly persons residing in the community. The results of this study support the use of COGTEL as a short, reliable and valid instrument for analyzing differences in cognitive functioning in inter-individual studies with elderly persons.


INTRODUCTION
The Mini Mental State Examination (MMSE) is one of the most widely used triage or screening tests for the assessment of changes in cognitive functions in both epidemiological studies and clinical settings. It includes orientation, memory, attention and calculation, language and constructive capacity tests 1 , and has been successively improved by considering variables such as age, schooling and the definition of specific regional cutoff points. Generally, the MMSE is recognized as a valid instrument that is easy and quick to apply 2 . This is one of reason why it has been extensively included in many epidemiological studies, as well as in studies of aging that seek to assess cognitive functions. However, the use of MMSE alone may result in limitations, particularly in studies with individuals residing in the community who have aged healthily in terms of cognitive functioning 3,4 . This means that the test may not be sensitive enough to differentiate individual performance levels, as it is restricted by its ceiling effect, which makes it difficult to assess and monitor interindividual differences in cognitive functioning 5,6 .
The Cognitive Telephone Screening Instrument (COGTEL) 3 battery of tests may be a useful alternative in this respect, as it allows the detailed evaluation of performance in six cognitive domains (prospective, short-term, long-term and working memory, verbal fluency and inductive reasoning) using tests adopted from well-established neuropsychological instruments such as the Wechsler scales, and additionally including a total score that is indicative of general cognitive functioning 3,4 .
Thus, the COGTEL can be widely applied in the evaluation of cognitive functions, as it differentiates total cognitive function performance, and identifies not only cognitive deficits but also interindividual differences in cognitive functioning within the range of healthy performance 4 . In addition, the COGTEL can be applied in a flexible manner through either face-to-face or telephone interviews, and takes 10 to 15 minutes to complete 3 .
The presentation and evaluation of the psychometric properties of the COGTEL described in this article include the application of the instrument with a sample of elderly adults living in the community in Brazil. To evaluate the reliability of the test, we asked the participants to undertake the COGTEL at two different times (test-retest method). Concurrent assessment was examined, calculating the degree of association between COGTEL scores, MMSE score and level of schooling. Most neuropsychological investigations of the impact of sociodemographic factors on cognitive ability have focused on the role of education 7 , as it has been shown to be an important determinant of cognitive performance 8 .
The objectives of the present study were: (1) to study the reliability/stability of the COGTEL instrument in the evaluation of cognitive functions, and (2) to investigate concurrent validity (i.e. the relationship between COGTEL scores and external variables such as schooling and total MMSE score) in a pilot study in elderly adults living in the community in the state of Amazonas, Brazil.

Study design and sample
The present pilot study included 90 elderly adults (29 men and 61 women) aged 60-85 years [68.2(±6. 7)]. The present study is part of the "Health, Lifestyle and Aptitude in Elderly Adults from Amazonas" (SEVAAI) research project. The elderly adults were residents of the community from Apuí, Fonte Boa and Manaus (municipal districts in the state of Amazonas, Brazil). To test the reliability of the assessment instrument, 90 elderly adults were reassessed for all the variables seven days after the first evaluation 9 . The evaluations took place in each of the municipal districts in partnership with the local institutions: the Amazonas Federal Institute of Education Science and Technology; the Open University of the Third Age of Amazonas State University; the Social Care Reference Center; the Oscar de Paulo Portela Municipal Library and the Paulista Social Center for the Elderly.
The study field team comprised 21 members: four students from the Master's in Physical Activity and Sport course of the University of Madeira (UMa), Portugal; 15 undergraduate students of the Licentiate degree course in Physical Education of Amazonas State University (UEA); a student from the Uninorte Physical Therapy course; and an undergraduate degree student from the Federal University of Amazonas (UFAM).
To maximize the consistency of the evaluations, training sessions were conducted with all members of the field team. Firstly, a theoretical explanation of the evaluation protocols and their tests was created. Secondly, the tests and questionnaires were applied among the members of the field team. This training phase lasted one month, with three training sessions per week, each lasting three hours. Thirdly, evaluations were conducted with elderly individuals to calculate the protocol administration times. On the day of the pilot study tests, all the research protocols were explained so that the elderly persons fulfilled the requested tasks.

Inclusion and exclusion criteria
The inclusion criteria of the sample considered in this study were: be a male or female resident in the community aged between 60 and 90 years of age; be autonomous and independent in the accomplishment of activities of daily living, and to have no reported health problems considered absolute contraindications to the practice of physical activity 10 .
The following exclusion criteria were used in the present study: to have a low level of physical functionality (assessed by the physical functionality questionnaire 11 ; physical functionality questionnaire score <12/24 points); have severe cognitive deficits (assessed by MMSE) 1 , MMSE score <15/30 points 2 , or severe hearing loss; have a co-morbidity that could compromise participation in physical activities (acute illness, progressive neurological diseases, stroke, unstable chronic conditions) 10 , or any of the following conditions: (1) individuals who had been advised by their doctor not to exercise because of medical conditions; (2) those with congestive heart failure; (3) people who had experienced joint pain, chest pain, dizziness or who suffered angina pectoris, and (4) people with uncontrolled blood pressure (greater than 160/100) 11 .
The participants were individually tested in faceto-face sessions by field team members trained in the application of the COGTEL, MMSE and the socioeconomic questionnaire.

Mini Mental State Exam (MMSE)
The Mini Mental State Examination (MMSE) 1 was used to assess mental status. This questionnaire allows a summary evaluation of cognitive functions. The test consists of five subsections covering orientation (0-10 points), immediate and recent memory (0-3 points each), attention capacity and counting backwards calculation (0-5 points), language and constructive capacity (0-9 points). A total score is derived from the sum of the scores of the five subsections.

Level of schooling
Level of schooling was estimated from the socioeconomic questionnaire of the Brazilian Association of Research Companies (ABEP) 12 . To this end, participants were asked about their level of schooling. The following scores were considered for this purpose: 0 = Illiterate/Incomplete primary; 1 = Complete primary/Incomplete junior high; 2 = Complete junior high/Incomplete secondary; 3 = Complete secondary/incomplete higher; 4 = Complete higher.

Cognitive Telephone Screening Instrument (COGTEL)
COGTEL, originally constructed with the dual purpose of being applied by telephone and in face-toface interviews, is composed of six subtests that cover important domains of cognitive functioning. All the procedures related to the application of COGTEL can be consulted in detail in an earlier publication by the authors of the instrument, Kliegel et al. 3 . Prospective Memory: Prospective memory is evaluated using the "event-based task" system. In this test, the task of executing the intended action is triggered by the presentation of specific external information. At the beginning of the questionnaire, the participants were instructed to say their date of birth without being asked for it at a certain point during the questionnaire. The prospective memory score was 1 if the participant correctly stated their date of birth at the right time. Otherwise, the score was 0.
Short-Term Verbal Memory: In this subtest, eight pairs of words are given to the participants (four or which are semantically related and the remainder of which are unconnected). After giving an example, the field team member reads aloud all the pairs, which the participants memorize. The short-term Verbal Memory score is the number of words correctly associated with the pairs (min = 0; max = 8).
Working Memory: Working Memory is evaluated using the backward digit-span test (saying a sequence of numbers from back to front). Participants hear the sequence of numbers and immediately repeat what they have heard in reverse order. The Working Memory score is the total number of correctly reproduced sequences (min = 0; max = 12).
Inductive Reasoning: In Inductive Reasoning, the member of the field team presents the participants with a sequence of five numbers constructed according to a mathematical rule between each of them. The participants must add the final number to the sequence in order to complete it. The inductive reasoning score is the total number of correct sequences (min = 0; max = 8).
Verbal Fluency: Verbal Fluency (executive functioning) is assessed using two tests: (1) Letter Fluency -participants are instructed to produce words that begin with the letter "A" for 60 seconds; and (2) Category Fluency -participants are instructed to state all the different types of profession they can think of in 60 seconds. The total verbal fluency score is the sum of the "letter fluency" test score + the "category fluency" test score.
Long-Term Verbal Memory: Long-Term Verbal Memory is evaluated in the same way as Short-Term Verbal Memory, using the same word pairs. The Long-Term Verbal Memory score is the number of words correctly associated with the pairs (min = 0, max = 8).
The total COGTEL score is derived from the sum of the scores of each of the six subtests, weighted accordingly, using the following formula: Total COGTEL score = 7.2 x prospective memory + 1.0 x short-term verbal memory + 0.9 × long-term verbal memory + 0.8 x working memory + 0.2 x verbal fluency + 1.7 x inductive reasoning.

COGTEL translation and retranslation procedure
The COGTEL translation team consisted of a committee of five researchers, including the lead author of the instrument (Matthias Kliegel; MK) who published the first English version in 2007 (Kliegel et al.) 3 . Initially, each subtest of the instrument was discussed with its author and the translation from English to Portuguese was performed by a native speaker. Later, the same procedure was carried out from French to Portuguese by a Portuguesedescended researcher, as COGTEL has also been translated into French. In both cases, each final version was back-translated and a final revision of both back-translation versions was made.
The translation, synthesis and back-translation procedures were carried out without difficulties and the modifications of the committee were aimed at guaranteeing the semantic, idiomatic, cultural and conceptual equivalence of the translated instrument with the original instrument. Finally, the instrument was tested in the community with different age groups (young people, young adults, adults and elderly adults). The purpose of the application of COGTEL in this preliminary phase was to simulate the application of the instrument in a real field context, to calculate application times and gather information relating to the degree of difficulty of the instrument. Prior to the pilot study, the committee reconvened and the version was again submitted for review and adjustment by the author of the instrument (MK), until it was considered ready.

Statistical Analysis
Descriptive statistics (mean and standard deviation) were used to describe the characteristics of the sample. The reliability/stability and concurrent validity of the COGTEL was assessed as follows: first, the test-retest reliability of the total COGTEL score (as well as the six subtests separately) was assessed among 90 elderly adults from the three regions of Amazonas studied using an intraclass correlation coefficient. Next, concurrent validity was evaluated by analyzing the relationship between total COGTEL score and total MMSE score using bivariate correlations (Pearson correlation coefficient). Finally, using the same procedure, the relationship between the total COGTEL score and level of schooling (number of years of schooling) was assessed.
The level of significance was set at p<0.05. Analysis was performed using the SPSS statistical program, version 23.0.

Test-retest reliability
The intraclass correlation coefficient (R) and the confidence interval (CI 95%) between the total test and retest COGTEL scores (and the scores of the six subtests), the MMSE scores and levels of schooling are shown in Table 1. A high test-retest reliability was found for the total COGTEL score (R=0.946). In the case of the six COGTEL subtests, reliability ranged from acceptable to high (Table  1). High reliability was also found for total MMSE score, as well as level of schooling, R=0.899 and R=0.985, respectively.

Concurrent validity
The relationship between the total COGTEL score (as well as for each of the six subtests) and MMSE and level of schooling was investigated using the Pearson correlation coefficient. Preliminary analyzes were carried out to ensure assumptions of normality, linearity and homoscedasticity. There was a positive correlation between total COGTEL score and MMSE (r=0.682, p<0.001), as well as level of schooling (r=0.604, p<0.001). The correlations in the remaining cognitive function subtests and MMSE ranged from weak for prospective memory (r=0.237; p<0.05) to strong for working memory (r=0.655; p<0.001) ( Table 2).

DISCUSSION
The present study sought to evaluate the reliability and concurrent validity of the COGTEL instrument in a pilot study with 90 elderly adults from the Amazon region. Test-retest reliability was high for the total COGTEL score and acceptable-to-high for the remaining six subtests of the instrument. Similar test-results were found for MMSE score and level of schooling. The intraclass correlation coefficient is the most commonly used measure to study the stability of the scores in the two tests 13 .
The reliability values reported in our pilot study are comparable to those found for other scales. For example, on the Wechsler scale (intelligence assessment), reliabilities ranging from 0.38 to 0.87 were found for young adults and adult-adults 14,15 . When considering the reliability measures of by the MMSE 16 , meanwhile, the values generally range between 0.80 and 0.95. The test-retest reliability reported in the present pilot study for the MMSE and the level of schooling was equally high, at 0.899 and 0.985, respectively.
These results indicate that COGTEL (as well as its subtests), the MMSE and level of schooling (evaluated from the socioeconomic questionnaire proposed for the Brazilian population) have an acceptable degree of reliability/stability, taking into account the points mentioned by Thomas and Nelson 13 . This means that these tests can be reliably used for assessing cognitive functioning and level of schooling in elderly persons. These results are in agreement with a recent study published in Dementia and Geriatric Cognitive Disorders Extra by the research team that translated and developed the COGTEL for the Portuguese language, Ihle et al. 4 . COGTEL is thus proposed as a quick and reliable assessment tool for cognitive functions, which can be used in epidemiological studies with elderly adults.
The present study also found a strong positive correlation between total COGTEL score and MMSE score, as well as level of schooling. Concurrent validity involves a measuring instrument and an evaluation criterion administered at the same time 13 . In the present study, MMSE was considered as a popular criterion measure, validated and widely accepted for the evaluation of cognition, particularly in elderly adults, and COGTEL was the instrument to be validated for this population. Ihle et al. 4 , in a Brazilian sample of 361 elderly men and 507 elderly women living in the community, with a mean age of 70.1 (± 6.8) and a variation of 57-92 years, established the correlations between COGTEL and MMSE. A substantial correlation was found between total COGTEL and MMSE score (r = 0.65, p <0.001). These results are very similar to those found in the present pilot study (r = 0.68, p <0.001).
Kliegel et al. 3 , when evaluating the simultaneous validity of the COGTEL instrument, calculated the Pearson correlations between total COGTEL score and level of schooling. Higher total COGTEL score values were associated with more schooling, r=0.47. The results of the present pilot study support the results of the authors of the COGTEL, Kliegel et al. 3 , obtaining even higher correlation values (r=0.60).
As previously suggested by Kliegel et al. 3 , in the present study the concurrent validation of COGTEL was performed by comparing the results of this instrument with those of other cognitive evaluation instruments, such as the MMSE. As described in literature, Creavin et al. 2 corroborates the finding that MMSE is one of the most used tests, both in epidemiological studies and in the context of clinical practice, mainly due to its established validity, as well as its quick and easy application. We are, therefore, dealing with an appropriate criterion. Since there was a strong positive correlation between total COGTEL score and the MMSE (r=0.682; p <0.001), we can say that there is concurrent validity in these instruments.
In contrast, the application of the COGTEL instrument presents some advantages over the MMSE in the evaluation of cognition. Firstly, in the quantification of total COGTEL score, six subtests are considered (prospective memory, shortterm verbal memory, working memory, inductive reasoning, verbal fluency and long-term verbal memory) with different weightings in the final COGTEL equation. In the case of the MMSE, all the tasks have the same weighting in the final calculations. Secondly, the MMSE has exhibited limitations in studies in individuals living in the community with healthy cognitive functioning aging. This means that the MMSE is not sensitive enough to differentiate individual performance levels, as it is restricted by its ceiling effect 3,4 . In other words, it does not allow the monitoring of interindividual differences in cognitive functioning in individuals who have already reached the total score of the scale 5,6 .
In addition, in order to study concurrent validation, we have supported our findings with previous studies 16,17,3 in which the associations between the tests of cognitive functions and level of schooling were calculated. This conceptual framework is based on the assumption that schooling, usually defined as the number of years of formal study completed, has proven to be an important determinant of cognitive performance 8 . Therefore, there is some unanimity in the idea that intellectual capacity and schooling contribute to the development of cognitive reserve, behind which lies an ability to attenuate the effects of neural impairment on the cognitive abilities resulting from the aging process 2,18,19 . These results support the use of COGTEL as a valid instrument for assessing cognitive function, which can be used in epidemiological studies with elderly adults residing in the community.
The present study, however, presents some limitations that must be considered when interpreting the results. Firstly, the participants were essentially voluntary, and therefore, may be generally healthier than those who did not participate. Second, survival bias, especially among men in older age groups, can not be ruled out as a potential confounding factor, particularly in comparisons between genders. Finally, although cases of severe hearing loss were identified, small auditory deficits were not totally controlled, which may have led to difficulties in understanding the tasks to be performed by the elderly.

CONCLUSION
The present study presents preliminary evidence of the reliability/stability and the concurrent validity of COGTEL in the evaluation of cognitive functions in elderly adults living in the community. The results of the present study support the use of COGTEL as a quick, reliable and valid instrument to analyze interindividual differences in cognitive functioning in studies with elderly adults.