Assessing executive functions in older adults: a comparison between the manual and the computer-based versions of the Wisconsin Card Sorting Test

CLINICAL AND EXPERIMENTAL NEUROPSYCHOLOGY

Assessing executive functions in older adults: a comparison between the manual and the computer-based versions of the Wisconsin Card Sorting Test

Gabriela Peretti Wagner; Clarissa Marceli Trentini

Instituto de Psicologia, Universidade Federal do Rio Grande do Sul, Brazil

^{Correspondence regarding this article should be directed to}

ABSTRACT

Executive functions (EF) are a group of high-level cognitive processes that control and direct lower-level abilities in order to produce goal-directed behavior. Because these functions are a multidimensional entity, they can be assessed using different tests. One of the tests often used to evaluate EF is the Wisconsin Card Sorting Test (WCST). The WCST is a task that involves hypothesis testing, identification of concepts, resistance to interference, switching categories, and inhibition. Two versions of the task have been used in neuropsychological testing, but little is known about their equivalence. In this study, we investigated the performance of two groups of elderly people in two versions of the task: manual (cards) and computer-based. Fifty-four healthy elderly participants took part in this study; half of the sample was assessed with the computer-based version of the WCST, while the other half performed the manual version. There were no differences between the two versions of the task in our sample of elderly participants (total number correct, perseverative errors, percent conceptual level responses, and number of categories completed). In this sense, the results provide evidence for the equivalence of both versions in the assessment of EF in healthy elderly participants.

Keywords: Wisconsin Card Sorting Test, executive functions, aging.

Introduction

Executive functions involve a variety of higher-order cognitive processes. These processes include cognitive flexibility, planning, judgment, decision making, initiation, and hypothesis generation (Strauss, Sherman, & Spreen, 2006). Based on this concept, one might conclude that a single cognitive test is not enough to evaluate each process of this multidimensional construct (Stuss & Levine, 2002).

One well-known measure of executive functions is the Wisconsin Card Sorting Test (WCST – Heaton, Chelune, Taley, Kay, & Curtiss, 1993). This test involves mainly the assessment of abilities related to abstract reasoning and shifting cognitive strategies in response to environmental contingencies. It also requires strategic planning, organization, and directing behavior toward a goal (Heaton et al., 1993).

One factor that can influence the psychometric properties of psychological tests, including the WCST, is the variability in application procedures (Anastasi & Urbina, 1997). The increasing use of computer versions of traditional tests and scales implies the necessity to evaluate the equivalence between paper and electronic versions. Concerning the WCST, however, only one study comparing its different versions has been published to date (Tien, Spevack, Jones, Pearlson, Schlaepfer, & Strauss, 1996). The scarceness of evidence related to automated versions of the WCST, together with the necessity of transcultural adaptation, led us to design a study to compare traditional and automated Portuguese versions of the WCST in Brazil.

Method

Participants

A total of 54 (70.3% females) participants took part in the study following a transversal design. Participants were aged 60-82 years. Volunteers were divided in two groups: 27 participants performed the manual version (cards) and 27 participants were assessed using the computer-based version. The two groups were matched for age and years of education. Demographic variables are displayed in Table 1.

Participants were excluded when they had past or current history of neurological and/or psychiatric conditions (self-reported). In order to take part in the study, individuals had to be older than 60 years, have at least four years of formal education, and a score of 24 or more on the Mini-Mental State Examination (MMSE – Folstein, Folstein, & McHugh, 1975).

Instruments

- Participant information sheet: Individuals had to fill out an information sheet where they provided information regarding age, gender, years of education, and medical conditions, including neurological and/or psychiatric disorders.

- Mini-Mental State Examination (MMSE): This instrument is a cognitive screening tool for deficits in attention, orientation, memory, language, and praxis. It was first published by Folstein and colleagues (1975). In the present study, the adaptation from Chaves and Izquierdo (1992) was used. Participants who scored less than 24 were excluded because this value is considered to be the cut-off score for demential conditions.

- Wisconsin Card Sorting Test – manual version: This version consists of a set of four stimuli cards and two decks of 64 response cards. The stimuli cards display one red triangle, two green stars, three yellow printed crosses, or four blue circles. The 128 response cards have pictures combining variations of forms (triangles, stars, crosses, and circles), colors (red, blue, yellow, and green) and numbers of figures printed (one, two, three or four). Each one of the response cards can match a stimulus card in one, two or three parameters. The participant's task is to try to match the response cards, one by one, to the stimuli cards. According to these combinations, a series of indexes can be calculated. For more details, please, see Heaton and colleagues (1993).

- Wisconsin Card Sorting Test – computer-based version: In the computer version of the WCST, the cards are displayed on a computer screen (top left part). The four stimuli cards are the same as in the version with cards, but the participant uses the keyboard to respond (numerical keys 1, 2, 3, and 4). One response card per trial appears in the middle of the screen, and the participant has to try to match it to one of the four stimuli cards. The numerical keys on the keyboard are equivalent to the four positions of the stimuli cards, which appear on the top left side of the screen.

Procedure

The experimental protocol was evaluated and approved by the local ethics committee in accordance with Bylaw 196/96 of the Brazilian National Health Council (Conselho Nacional de Saúde). Participants gave their voluntary consent to take part in the study. Test sessions were individual and lasted about one hour. After providing their consent, individuals filled the participant information sheet and were assessed with the MMSE (Chaves & Izquierdo, 1992; Folstein et al., 1975). Finally, half of the sample was presented with the manual version of the WCST, while the other half performed the computer-based version of the test.

Results and Discussion

The normality of the sample was tested using Kolmogorov-Smirnov's test. The distribution of the performance on the test was normal. Regarding demographic variables, the results suggest that the differences between the two groups of elderly were not significant, according to the t test for independent samples. As seen in Table 1, both groups were equivalent in regard to age and years of formal education, which are important variables in neuropsychological assessment (Strauss et al., 2006). Firstly, aging can impair cognitive abilities such as executive functions, for example (MacPherson, Phillips, & Della Sala, 2002). When Strauss and colleagues (2006) compared the performance of three age groups in the WCST, they found a poorer performance of the oldest group in the WCST and other measures of executive functions. Second, years of education appear to influence cognitive abilities (Strauss et al., 2006), which means that the more time a person spends at school, the better the performance on some cognitive tests (Ardila, Ostrosky-Solis, Rosseli, & Gómez, 2000). However, in our sample, both groups were matched in terms of age and years of education. As there were no significant differences in these variables, one might say that possible differences in WCST performances are unlikely to be due to the time spent at school or to chronological age.

For the MMSE, the t test for independent samples showed significant differences [t (43.611) = 2.193, p = .03]. Despite achieving normal scores on the task (MMSE >24 – Chaves & Izquierdo, 1992), which indicates that participants of both groups do not show signs of severe cognitive deficits, those elderly who performed the manual version of the WCST had a lower mean score on the MMSE. The MMSE is an instrument that taps general cognition and is used for broad deficits in demential states. In the present study, both groups had a normal performance on the task, which means that participants with typical cognitive impairments characteristic of dementia were not included. So, despite the fact that there was a significant difference in performance on the MMSE between the two groups, this difference was not clinically relevant. Scores within the range of 24-30 are considered normal, while scores between 0 and 23 suggest demential disorder.

Participants' performance on both versions of the WCST was compared. For this analysis, four of all WCST scores were used, including total number of correct responses, perseverative errors, percent conceptual level responses, and number of categories completed. Groups' performance was compared using the t test for independent samples. As seen in Table 2, there were no significant differences between the groups on these four scores. Rhodes (2004) conducted a meta-analysis investigating studies where the WCST was used, especially those in which the goal was to investigate the effects of aging. The author concluded that there is strong evidence that chronological age is an intervening variable on WCST performance. The groups of the present study were age-equivalent, and perhaps this can explain the absence of differences in the performance of both versions of the WCST.

It is important to mention that there was a trend toward a statistically significant difference in the number of categories completed, as seen in Table 2. In order to investigate this in depth, a new analysis was conducted. One could say that the significant difference in the MMSE - although not clinically relevant - could explain the trend to significance in the number of categories completed in the WCST. In other words, despite the normal performance in the MMSE (MMSE > 24), a new data analysis was conducted in order to check if the performance in the MMSE would affect the WCST results. This time the authors considered only those participants who scored more than 28 in the MMSE from both groups. Out of the 54 participants, 12 were excluded according to this criterion. This means that 23 participants were allocated to the computer-based version and 19 participants performed the manual version. The differences between the groups in the MMSE disappeared [t(40) = 1.161; p = .253]. After this analysis, the same pattern of results was found in the number of categories completed – the same trend to significance [t(40) = -1.931; p = .061]. Based on these findings, it is possible to say that both groups are equivalent in terms of cognitive performance, which means that there were no differences between the two versions of the WCST (manual and computer-based).

Finally, Tien and colleagues (1996) found no differences between the manual and computer-based versions of the WCST in their sample. The findings of this study support their conclusions, speaking in favor of the use of computer-based tests in the neuropsychological assessment of executive functions. A similar assumption is defended by Schatz and Browndyke (2002), who believe that computer-based tests can offer benefits over paper-and-pencil evaluations of cognitive functions during neuropsychological assessment.

Conclusion

The authors believe that the current study provides evidence for the equivalence of manual and computer-based versions of the WCST when assessing healthy participants. Based on the presented results, it seems that the administration type does not affect participant's performance.

The authors suggest two alternatives to contribute to these results in further investigations. First, a larger sample should be recruited in order to test the equivalence hypothesis. Larger samples tend to be adequate because there are higher chances that variables will follow a Gaussian distribution. Also, as seen in Table 2, the group that performed the manual version tended to have a better performance in the WCST. In other words, the group that performed the computer-based version of the WCST had less correct responses, less categories completed, less conceptual level responses, and more perseverative errors. It is important to investigate why those participants who have higher scores in the MMSE (23 out of 27 elderly scored more than 28) and were assessed through a possibly more difficult task (the computer-based one) have lower means in the WCST when compared to those who performed the manual task. The authors believe that increasing the sample size could, perhaps, answer this question.

As a second alternative, the assessment of younger participants might provide new data regarding comparisons between manual and computer-based versions of the WCST. As the scientific literature shows, age affects WCST performance, and only elderly people were assessed in the present study. A 2 x 2 design (two types of WCST administration x two age groups) could offer more evidence for researchers and clinicians.

Received 24 October 2009

received in revised form 23 November 2009

accepted 23 November 2009

Available on line 29 December 2009

• Anastasi, A., & Urbina, S. (1997). Psychological Testing. New Jersey: Prentice-Hall.
• Ardila, A., Ostrosky-Solis, F., Rosseli, M., & Gómez, C. (2000). Age related cognitive decline during normal aging: The complex effect of education. Archives of Clinical Neuropsychology, 15, 495-513.
• Conselho Nacional de Saúde (1996). Diretrizes e normas regulamentadoras de pesquisas envolvendo seres humanos. Resolução 196/96, Brasília.
• Chaves, M.L.F., & Izquierdo, I. (1992). Differential diagnosis between dementia and depression: A study of efficiency increment. Acta Neurologica Scandinavica, 85, 378-382.
• Folstein, M.F., Folstein, S.E., & McHugh, P.R. (1975). Mini-mental state. A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatry Resources, 12, 189-198.
• Heaton, R.K., Chelune, G.J., Taley, J.L., Kay, G.G., & Curtiss, G. (1993). Wisconsin Card Sorting Test Manual (Revised and expanded). Odessa: Psychological Assessment Resources.
• MacPherson, S.E., Phillips, L.H., & Della Sala, S. (2002). Age, executive function, and social decision making: A dorsolateral prefrontal theory of cognitive aging. Psychology and Aging, 17, 598-609.
• Rhodes, M.G. (2004). Age-related differences in performance on the Wisconsin Card Sorting Test: A meta-analytic review. Psychology and Aging, 19(3), 482-494.
• Schatz, P., & Browndyke, J. (2002). Applications of computer-based neuropsychological assessment. Journal of Head Trauma Rehabilitation, 17, 395-410.
• Strauss, E., Sherman, E.M.S., & Spreen, O. (2006). A compendium of neuropsychological tests. Oxford: Oxford University Press.
• Stuss, D.T., & Levine, B. (2002). Adult clinical neuropsychology: Lessons from studies of the frontal lobes. Annual Reviews of Psychology, 53, 401-433.
• Tien, A.Y., Spevack, T.V., Jones, D.W., Pearlson, G.D., Schlaepfer, T.E., & Strauss, M.E. (1996). Computerized Wisconsin Card Sorting Test: Comparison with manual administration. Kaohsiung Journal of Medical Sciences, 12, 479-485.

Publication Dates

• Publication in this collection
  17 Jan 2011
• Date of issue
  Dec 2009

History

• Accepted
  23 Nov 2009
• Received
  24 Oct 2009
• Reviewed
  23 Nov 2009