Validity evidence for the Cognitive Screening Test in Stroke Patients

Evidências de Validade para o Instrumento Triagem Cognitiva em Pacientes com Acidente Vascular Cerebral

Evidencias de Validez del Instrumento de Cribado de Deterioro Cognitivo en Pacientes con Accidente Cerebrovascular

Abstract

Cognitive deficits are common among post-stroke patients. Cognitive impairments of this sort are mediated by age and education. In Brazil, the only specific cognitive screening tool designed for post-stroke patients is the Cognitive Screening Test (Triagem Cognitiva - TRIACOG). The goal of this study was to investigate validity evidence related to external variables for the TRIACOG. Our sample included 153 adults and elderly people (M = 60.08, SD = 9.61) from Porto Alegre and metropolitan area, comprising 87 post-stroke patients and 66 healthy individuals. Three-way ANOVAs were used to assess main effects and interactions between the variables group (clinical/control), age and education. An influence of group and age on scores in the TRIACOG was found. We emphasize the relevance of these results to the selection of cut-off points for the tasks and cognitive functions assessed by the instrument, considering education and age, so as to allow more accurate identification of deficits in post-stroke patients.

Keywords:
cerebrovascular accident; neuropsychological assessment; test validity; schooling; age groups

Resumo

Déficits cognitivos são comuns em pacientes após acidente vascular cerebral (AVC). O prejuízo cognitivo causado por esse evento é mediado por variáveis etárias e de escolaridade. No Brasil, o único instrumento de rastreio cognitivo específico para o pós-AVC é a Triagem Cognitiva (TRIACOG). O objetivo deste estudo é investigar evidências de validade relacionadas a variáveis externas da TRIACOG. Participaram do estudo 153 adultos e idosos (M = 60,08; DP = 9,61) de Porto Alegre e região metropolitana, sendo 87 pacientes pós-AVC e 66 saudáveis. Three-way ANOVA foi utilizada para indicar os efeitos e interações entre variáveis de grupo, etárias e educacionais. Observou-se a influência dos fatores de grupo e idade nos escores da TRIACOG. Ressalta-se a relevância dos resultados para a construção de pontos de corte para tarefas e funções do instrumento, considerando aspectos educacionais e etários, aumentando a precisão na identificação de déficits em pacientes pós-AVC.

Palavras-chave:
acidente cerebrovascular; avaliação neuropsicológica; validade do teste; escolaridade; grupos etários

Resumen

Los déficits cognitivos son comunes en pacientes después de un accidente cerebrovascular. El deterioro cognitivo causado por este evento está mediado por variables de edad y educación. En Brasil, la única herramienta de detección cognitiva específica para después de un accidente cerebrovascular es el Cribado de Deterioro Cognitivo (TRIACOG). El propósito de este estudio fue investigar evidencias de validez relacionadas con las variables externas de TRIACOG. Participaron en el estudio un total de 153 adultos y ancianos (M = 60.08; DS= 9.61) de Porto Alegre y región metropolitana, de los cuales, 87 eran pacientes posictus y 66 eran sanos. Se utilizó Three-way ANOVA para indicar los efectos y las interacciones entre las variables de grupo, edad y escolarización. Se observó la influencia de factores de grupo y edad en las puntuaciones del TRIACOG. Se enfatiza la relevancia de los resultados para la construcción de puntos de corte para tareas y funciones del instrumento, teniendo en cuenta aspectos educativos y de edad, aumentando la precisión en la identificación de déficits en pacientes posictus.

Palabras clave:
accidente cerebrovascular; evaluación neuropsicológica; validación de test; escolarización; grupos por edad

Introduction

Post-stroke cognitive impairments have heterogeneous manifestations, with a variety of neuropsychological profiles occurring among patients (Milinavičienė et al., 2011). Individual clinical profiles comprise several variables that influence post-stroke cognitive and functional outcomes (Sagnier et al., 2019). Such variables include stroke-specific neurological characteristics (such as stroke type and location; Wei et al., 2015), post-event cognitive profile (Sagnier et al., 2019), and sociodemographic aspects, such as age and education (Bento-Torres et al., 2017). A number of studies show an effect of age and schooling in performance and results of post-stroke neuropsychological assessments. Hence, those variables must be considered when constructing and investigating the psychometric properties of new instruments (Bento-Torres et al., 2017; Pinto et al., 2018).

Low education is related to increased incidence of cerebrovascular conditions and poor results in neuropsychological assessments, especially when associated with socioeconomic factors. Studies show that education is protective factor in neurological and neurodegenerative conditions, possibly due to the association between high educational level and greater cognitive reserve, which promotes greater cognitive adaptation capacity following neurological events (Bento-Torres et al., 2017; Stern, 2009). Individuals with low educational levels also have lower access to information about risk factors and stroke prevention, resulting in a higher frequency of stroke in this population (Stelmach et al., 2004).

As for age, younger individuals affected by cerebrovascular disease show better cognitive outcomes in neuropsychological assessments, as well as more successful recovery (Tang et al., 2018). Some studies point out that age can be a proxy for accumulated lifetime experiences that affect cognitive function (Patel, Coshall, Rudd &Wolfe, 2002).

Analyses of post-stroke cognitive deficits reveal heterogeneous profiles of cognitive impairment, including a variety of neuropsychological deficits, the most common of which are: Impaired processing speed, hemispatial neglect, attention deficits, aphasia, apraxia, and memory impairment (Saa et al., 2019). Up to 80% adults who suffer a stroke exhibit neuropsychological alterations in cognitive domains such as memory, language, and executive functions (Sun, Tan & Yu, 2014), with a fraction of these (around 40%) showing no improvement in neuropsychological profile after the stroke (Blackburn et al., 2013).

When assessing post-stroke cognitive deficits, it’s important to conduct a comprehensive neuropsychological assessment, investigating each cognitive function that may be relevant to the clinical profile (Barker-Collo & Feigin, 2006). However, administering assessment instruments is impractical in some contexts, such as neuropsychological evaluation of in-bed post-stroke patients, due to factors such as limited period during which many patients are hospitalized, and the time required for that type of assessment, which may compromise the results of such assessments (Bento-Torres et al., 2017). In such cases, Nys et al. (2005) recommended the administration of cognitive screening instruments.

Instruments for post-stroke cognitive screening used in Brazil include the Montreal Cognitive Assessment - MoCA (Nasreddine et al., 2005) and the Mini-Mental State Examination (MMSE) (Folstein, 1999). However, these instruments are appropriate only for severely impaired patients (Stolwyk et al., 2014). Even if some cognitive domains measured by these tests are compatible with deficits observed in stroke patients, specific instruments to assess this condition are still needed (Kosgallana et al., 2019).

The Cognitive Screening Test was developed to fill this gap (TRIACOG; Rodrigues, Bandeira & Salles, 2021). The TRIACOG was constructed based on theoretical models in which neuropsychological functions rely on a broad and integrated network of interrelated cognitive functions (Rodrigues, Bandeira, & Salles, 2020). The instrument comprises tasks that measure multiple cognitive functions impacted by stroke. It also includes tasks to assess subprocesses of cognitive domains such as language and executive functions, which are seldom measured by the most commonly administered tests (i.e., MMSE and MoCA; Rodrigues, Bandeira & Salles, 2020).

The TRIACOG comprises 21 tasks that can be completed in approximately 25 minutes, and assess eight cognitive functions grouped into five domains: Memory Domain (orientation, attention/working memory, episodic-semantic verbal memory, and visual memory); Praxis Domain (constructive praxis and ideomotor praxis); Executive Functions Domain (verbal fluency, rapid serial naming, and processing speed); Language Domain (listening, writing, naming, vocabulary, reading, inference processing, repetition, and dictated writing tasks); and Numerical Processing Domain (transcoding and calculation).

To investigate how external variables influence results in the TRIACOG, as well to support interpretations of those results, psychometric properties are needed, including validity evidence (American Educational Research Association [AERA], American Psychological Association [APA] & National Council on Measurement in Education [NCME], 2014; Pasquali, 2010). Validity evidence indicates the extent to which interpretations of test results are supported by evidence and theory, considering how the instrument was conceived to be used. Currently, validity is no longer characterized as a property of tests, but rather of test interpretations (AERA, APA & NCME, 2014; Pacico, Hutz, Schneider, & Bandeira, 2015). For Valentini and Damásio (2016), validity evidence for a test can be obtained through systematic research that confirms or adds empirical evidence or supports its use. It is preferable to have multiple sources of validity evidence when investigating whether interpretations of an instrument are valid; instruments supported exclusively by psychometric analyses of comparisons between clinical and non-clinical groups, for example, are considered lacking (AERA, APA & NCME, 2014). Specifically, for post-stroke cognitive conditions and their heterogeneous manifestations, such evidence should also include age and variables related to education, identified in other studies as relevant in neuropsychological assessments (Pawlowski, Segabinazi, Wagner, & Bandeira, 2013).

In a previous study, interpretations of the TRIACOG showed validity evidence and significant performance differences in all tasks between post-stroke patients and healthy individuals. Patients’ performance was worse compared to a group of healthy individuals, with effect sizes ranging from medium to large for most tasks (Rodrigues, Salles, & Bandeira, 2020). That study controlled for the effects of age and education on participants’ performance in comparative analyses of performance differences between post-stroke patients and the control group (Rodrigues, Salles, & Bandeira, 2020).

Considering the influence of age and education in post-stroke cognitive outcome (Bento-Torres et al., 2017; Pinto et al., 2018), the current study aims to obtain further validity evidence for the TRIACOG, expanding the sample studied by Rodrigues, Salles, and Bandeira (2020) and analyzing the effect of external variables on test results. This was done by assessing main effects and interactions between the variables group type (clinical/control), age (groups of individuals between 40-59 and 60-75 years of age), and education (3-7 and 8-11 years of formal study).

Methods

Participants

The study was conducted with 153 participants, 88 females (57.5%) and 65 males (42.5%). This sample comprised 87 adults who suffered stroke and 66 neurologically healthy adults from the city of Porto Alegre and its metropolitan region, between 40 and 75 years of age (M = 60.08; SD = 9.61). Patients were invited to the study at the Special Care Unit of Hospital de Clínicas de Porto Alegre (HCPA). The clinical group included individuals who had suffered a stroke of any type (diagnosed by neuroimaging exams), in either hemisphere or any brain region. To obtain data on post-stroke specific cognitive functions without interference from other variables, we excluded illiterate patients with a history of learning impairments in school, neurological injury - such as traumatic brain injury or brain tumor (according to neuroimaging tests) - and individuals who reported use of illicit drugs.

Control participants were members of the general community and were recruited following the same steps as those employed by Rodrigues et al. (2020) in the construction and investigation of validity evidence for the TRIACOG. Illiterate individuals, those with poor performance in the Mini Mental State Examination - MMSE (using a cut-off point adjusted for education: 22 points for individuals with up to 7 years of formal education and 23 points for those with 8 to 11 years of formal education; Kochhann et al., 2010), and those who scored above 9 points on the BDI-II screening test for depressive symptoms (Gorenstein et al., 2011) were excluded from the control group. The sample was split into two groups by age, and two groups by years of formal education, as shown in Table 1.

Table 1
Sociodemographic Data for Post-Stroke Patients and Controls, by Age and Education

Variables		Groups
Age groups		40 - 59		60 - 75
Education		3 - 7	8 - 11	3 - 7	8 - 11
N	P	13	23	37	14
	C	14	23	16	13
Age (M, SD)	P	53.1 (4.7)	52.3 (5.3)	68.4 (4.3)	67.4 (3.9)
	C	51.6 (6.6)	50.1 (6.4)	69.4 (5.0)	64.2 (3.8)
Years of formal education (M, DP)	P	5.2 (1.7)	9.3 (1.4)	4.43 (1.3)	9.14 (1.4)
	C	5.1 (1.1)	9.7 (1.5)	4.5 (0.7)	9.8 (1.2)

• Note. M = mean; SD = Standard deviation; P = Patients; C = Controls.

This study was approved by the Research Ethics Committee and by the Ethics Committee. All individuals or their guardians signed an informed consent form before participating in the study.

Instruments and procedures

The instruments and questionnaires used in this study were administered to the clinical group in the hospital bed, sitting on the bed or on a chair, with the help of a table and clipboard to answer the tasks. For the control group, data collection occurred in reserved rooms at the Federal University of Rio Grande do Sul (UFRGS). The following instruments and questionnaires were employed:

1. Questionnaire on health conditions and sociocultural aspects for the clinical group: this instrument includes questions about age, education, premorbid clinical history, as well as specific information about the stroke (type of stroke and lesion location) and qualitative information on post-injury cognitive-behavioral and motor aspects;

2. Questionnaire on health conditions and sociocultural aspects for the control group: this instrument collects data on age, education and clinical history;

3. Cognitive Screening (TRIACOG; Rodrigues, Bandeira, & Salles, 2021) for both groups: a screening test composed of tasks that assess eight main neuropsychological functions, allowing to determine, in a short time, whether a patient has cognitive deficits and requires neuropsychological assessment and intervention. The instrument consists of 21 subtests encompassing eight main neuropsychological functions: orientation (time); episodic-semantic verbal memory (immediate and delayed recall); praxis (constructive and ideomotor); visual memory; auditory attention/working memory; executive functions (verbal fluency, processing speed, inhibition, and alternation); language (listening and listening, vocabulary, reading, inference processing, repetition, and dictated writing); and numerical processing (transcoding and calculation).

4. Beck Depression Scale (BDI-II) (Gorenstein et al., 2011) administered only to the control group: this instrument, adapted for the Brazilian context, is a self-report questionnaire with 21 items. For each item, four alternatives can be selected, with scores ranging from 0 to 3, except in items 16 and 18, for which there are seven alternatives, which does not change the score. Respondents select the alternative that best describes how they have been feeling in the past two weeks. The items refer to progressively increasing levels of depression severity. The overall score is the sum of individual items, with a maximum of 63 points. This score is categorized as mild, moderate, or severe levels of depression intensity. The BDI-II has a satisfactory reliability estimate based on Cronbach’s alpha with a value of .86, indicating that the instrument is sufficiently accurate to measure the intensity of depressive symptoms (Paranhos, Argimon & Werlang, 2010).

5. Mini-Mental State Examination - MMSE (Folstein, 1999) administered only to the control group: a screening instrument that assesses cognitive functions grouped into seven categories: spatial orientation, temporal orientation, word registration, attention and calculation, word recall, language and visual constructive skills. With a score ranging from 0 to 30, the MMSE detects cognitive impairment when a score below 23 points is obtained. The choice of this cut-off point considered a study by Kochhann et al. (2010) and Lourenço and Veras (2006) on the psychometric characteristics of the MMSE, prioritizing greater sensitivity.

Data analysis

Descriptive statistics (mean and standard deviations) were computed for the sociodemographic data of the total sample. To evaluate the effects of the variables group (clinical or control), age, and years of formal education on cognitive function and TRIACOG subtests, a three-way analysis of variance (ANOVA) was performed, with post-hoc pairwise Bonferroni comparisons to analyze interactions. Statistical analyses were carried out using the Statistical Package for the Social Sciences (SPSS) for Windows, version 24.0 (IBM Inc., 2016).

Results

Clinical condition influenced performance in all cognitive functions, with healthy individuals performing better than patients (Table 2, 3, 4, and 5). Age had a significant effect on the performance of both the clinical and control groups, with older individuals showing reduced performance on the TRIACOG. This variable significantly influenced scores on tasks for almost all cognitive functions (with the exception of calculation) and on most subtests, except for clock drawing, listening, naming, reading/inference processing, and written comprehension (Table 4 and 5). Education influenced performance of both groups - clinical and control - on all cognitive functions and half of the subtests, except for orientation and calculation. Specifically, lower education levels were associated with lower scores on the instrument (Table 2, 3, 4 and 5).

Table 2
TRIACOG performance of patients and healthy, by age group and years of study

Cognitive Domais/Tasks		Groups
Age		40 - 59			60 - 75
Schooling		3 - 7	8 - 11	Sig.	3 - 7	8 - 11	Sig.
Age (M; SD)	C	53,1 (4,7)	52,3 (5,3)	0,66	68,4 (4,3)	67,4 (3,9)	0,42
	H	51,6 (6,6)	50,1 (6,4)	0,51	69,4 (5,0)	64,2 (3,8)	0,01*
Sig		0,51	0,21		0,49	0,008*
Years of study	C	5,2 (1,7)	9,3 (1,4)	< 0,001*	4,43 (1,3)	9,14 (1,4)	< 0,001*
	H	5,1 (1,1)	9,7 (1,5)	< 0,001*	4,5 (0,7)	9,8 (1,2)	< 0,001*
Sig		0,80	0,42		0,91	0,21
Orientation (M; SD; Min-Max)	C	1,78 (0,55) (0 - 2)	1,91 (0,30) (1 - 2)	0,70	1,37 (0,76) (0 - 2)	1,50 (0,80) (0 - 2)	0,98
	H	2,0 (a.) (2 - 2)	1,96 (0,21) (1 - 2)	0,44	1,93 (0,26) (1 - 2)	2,0 (a.) (2 - 2)	0,33
Sig.		0,15	0,48		0,01*	0,01*
Attention/Working Memory (M; SD; Min-Max)	C	4,82 (1,47) (3 - 7)	5,17 (2,50) (1 - 10)	0,75	2,17 (1,49) (0 - 6)	4,17 (2,52) (0 - 8)	0,01*
	H	5,29 (2,52) (1 - 10)	6,22 (2,81) (1 - 10)	0,32	4,87 (2,33) (1 - 8)	6,23 (2,39) (2 - 10)	0,11
Sig		0,13	0,47		0,02*	0,19
Episodical-Semantic Verbal Memory (M; SD; Min-Max)	C	5,09 (2,21) (1 - 10)	5,56 (2,38) (1 - 12)	0,93	3,1 (1,65) (0 - 6)	4,42 (2,02) (1 - 9)	0,05*
	H	6,29 (1,86) (4 - 10)	6,39 (1,88) (3 - 11)	0,87	4,07 (1,28) (2 - 7)	6,08 (2,02) (3 - 10)	0,002*
Sig.		0,86	0,04*		0,02*	0,006*
Immediate (M; SD; Min-Max) Max)	C	4,0 (1,22) (1 - 6)	4,17 (1,23) (1 - 6)	0,69	2,83 (1,39) (0 - 5)	3,58 (1,24) (1 - 6)	0,11
	H	4,50 (0,86) (3 - 6)	4,57 (0,95) (2 - 6)	0,83	3,60 (0,99) (2 - 5)	4,62 (1,12) (3 - 6)	0,01*
Sig.		0,23	0,15		0,01*	0,12
Late Memory (M; SD; Min-Max)	C	1,13 (1,42) (0 - 6)	1,23 (1,42) (0 - 6)	0,83	0,27 (0,52) (0 - 2)	0,83 (1,80) (0 - 6)	0,35
	H	1,79 (1,53) (0 - 5)	1,83 (1,40) (0 - 5)	0,94	0,47 (0,92) (0 - 3)	1,46 (1,20) (0 - 3)	0,01*
Sig.		0,10	0,008*		0,20	0,25
Visual Memory (M; SD; Min-Max)	C	9,64 (4,72) (0 - 17)	11,39 (7,70) (0 - 20)	0,74	5,97 (6,20) (0 - 20)	6,50 (8,13) (0 - 23)	0,65
	H	15, 36 (6,02) (0 - 22)	19,04 (3,81) (7 - 23)	0,05*	11,87 (5,97) (0 - 18)	16,92 (3,88) (9 - 24)	0,01*
Sig.		< 0,001*	0,08		< 0,001*	< 0,001*	< 0,001
Praxis (M; SD; Min-Max)	C	22,55 (4,48) (15 - 29)	24,61 (6,48) (8 - 33)	0,75	16,47 (10,02) (0 - 31)	21,25 (10,93) (2 - 33)	0,23
	H	30,36 (2,17) (27 - 34)	30,96 (1,58) (27 - 33)	0,38	28,47 (2,29) (23 - 32)	30,15 (3,00) (23 - 33)	0,07
Sig.		< 0,001*	0,17		0,004*	0,02*
Figure Copy (M; SD; Min-Max)	C	15,08 (5,90) (2 - 23)	16,65 (6,75) (0 - 24)	0,49	12,33 (7,97) (0 - 23)	15,33 (8,45) (0 - 24)	0,29
	H	21,79 (1,76) (19 - 24)	22,30 (0,97) (20 -24)	0,33	21,00 (1,13) (19 - 23)	21,54 (2,18) (17 - 24)	0,34
Sig.		0,01*	0,41		0,01*	0,02*
Ideomotor Praxis (M; SD; Min-Max)	C	0,96 (0,21) (0 - 1)	0,98 (0,21) (0 - 1)	0,46	0,83 (0,38) (0 - 1)	0,83 (0,39) (0 - 1)	0,99
	H	1,0 (a.) (1 - 1)	1,0 (a.) (1 - 1)	1,00	0,93 (0,26) (0 - 1)	1,0 (a.) (1 - 1)	0,38
Sig.		0,33	0,32		0,008*	0,35
Clock Draw (M; SD; Min-Max)	C	4,83 (3,08) (0 - 9)	5,46 (1,66) (3 - 7)	0,43	3,30 (2,49) (0 - 8)	5,08 (2,81) (1 - 9)	0,14
	H	7,57 (0,94) (6 - 9)	7,65 (1,91) (5 - 9)	0,83	6,53 (2,26) (1 - 9)	7,62 (1,04) (5 - 9)	0,10
Sig.		< 0,001*	0,14		0,002*	0,05*
Executive Functions (M; SD; Min-Max)	C	18,82 (6,31) (5 - 26)	21,61 (6,60) (0 - 26)	0,44	18,33 (6,24) (8 - 27)	19,23 (5,12) (11 - 27)	0,88
	H	24,43 (2,47) (18 - 28)	25,09 (2,02) (21 - 28)	0,38	24,27 (1,98) (21 - 27)	25,62 (1,12) (23 - 27)	0,11
Sig		0,01*	0,33		< 0,001*	0,15

• Note. * p ≤ 0,05; M = Mean; SD = Standard Deviation; C = Clinical; H = Healthy.

Table 3
TRIACOG performance of patients and healthy by age group and years of study

		Cognitive Domais/Tasks			Groups
Age		40 - 59			60 - 75
Schooling		3 - 7	8 - 11	Sig.	3 - 7	8 - 11	Sig.
Verbal Fluency (M; SD; Min-Max)	C	1,0 (0,91) (0 - 2)	1,26 (0,86) (0 - 3)	0,40	0,83 (1,03) (0 - 3)	0,90 (0,99) (0 - 3)	0,98
	H	1,93 (0,92) (1 - 4)	2,3 (1,02) (1 - 4)	0,70	1,67 (0,98) (0 - 3)	2,23 (0,73) (1 - 4)	0,16
Sig.		0,01*	0,76		< 0,001*	0,22
Processing Speed (M; SD; Min-Max)	C	68,02” (32,85) (34” - 132”)	41,88” (20,14) (16” - 106”)	0,03*	84,08” (44,49) (25” - 227”)	69,13” (65,23) (27 “- 190”)	0,45
	H	39,67” (15,43) (21” - 81”)	32,48” (10,03) (14” - 55”)	0,09	45,56” (21,02) (23” - 100”)	32,63” (8,25) (22” - 51”)	0,04*
Sig.		0,01*	0,71		0,01*	0,02*
Serial Naming (M; SD; Min-Max)	C	17,31 (5,98) (5 - 24)	19,09 (7,69) (0 - 24)	0,48	17,50 (5,71) (8 - 24)	18,33 (4,73) (10 - 24)	0,87
	H	22,50 (2,24) (16 - 24)	22,78 (1,81) (18 - 24)	0,68	22, 60 (1,96) (19 - 24)	23,38 (1,33) (20 - 24)	0,21
Sig.		0,03*	0,01*		0,001*	0,14
Language (M; SD; Min-Max)	C	30,10 (4,44) (20 - 35)	30,67 (6,81) (4 - 35)	0,79	25,80 (7,95) (5 - 34)	30,17 (2,89) (26 - 34)	0,01*
	H	33,21 (1,31) (31 - 35)	34,39 (1,12) (31 - 36)	0,01*	32,60 (2,77) (26 - 36)	33,46 (3,38) (23 - 36)	0,43
Sig.		0,08	0,37		0,004*	0,01*
Oral comprehension (M; SD; Min-Max)	C	0,92 (0,28) (0 - 1)	0,78 (0,42) (0 - 1)	0,24	0,93 (0,25) (0 - 1)	0,83 (0,39) (0 - 1)	0,49
	H	1,0 (a.) (1 - 1)	1,0 (a.) (1 - 1)	1,00	1,0 (a.) (1 - 1)	1,0 (a.) (1 - 1)	1,00
Sig.		0,02*	0,02*		0,08	0,35
Naming (M; SD; Min-Max)	C	3,38 (1,04) (1 - 4)	3,57 (1,04) (0 - 4)	0,62	3,03 (1,27) (0 - 4)	3,00 (0,95) (2-4)	0,69
	H	3,86 (0,36) (3 - 4)	3,91 (0,29) (3 - 4)	0,61	3,8 (0,56) (2 - 4)	4,0 (a.) (4 - 4)	0,19
Sig.		0,13	0,31		0,002*	0,35
Vocabulary/Semantic Memory (M; SD; Min-Max)	C	1,15 (0,56) (0 - 2)	1,13 (0,63) (0 - 2)	0,91	0,93 (0,64) (0 - 2)	0,92 (0,52) (0 - 2)	0,35
	H	1,29 (0,47) (1 - 2)	1,74 (0,45) (1 - 2)	0,01*	1,27 (0,59) (0 - 2)	1,62 (0,51) (1 - 2)	0,09*
		< 0,001*	0,12		0,001*	0,05*
Reading/Inference processing (M; SD; Min-Max)	C	12,85 (3,51) (3 - 16)	12,65 (4,62) (0 - 16)	0,90	11,23 (5,07) (0 - 16)	14,17 (1,11) (12 - 16)	0,001*
	H	15,14 (0,95) (14 - 16)	15,17 (0,83) (13 - 16)	0,92	14,67 (1,29) (12 - 16)	14,92 (0,95) (13 - 16)	0,49
Sig.		0,02*	0,53		0,05*	0,12
Writing Comprehension (M; SD; Min-Max)	C	0,87 (0,34) (0 - 1)	0,92 (0,28) (0 - 1)	0,64	0,90 (0,31) (0 - 1)	0,92 (0,29) (0 - 1)	0,54
	H	1,0 (a.) (1 - 1)	1,0 (a.) (1 - 1)	1,00	1,0 (a.) (1 - 1)	1,0 (a.) (1 - 1)	1,00
Sig.		0,08	0,08		0,34	0,17
Repetition/Dictated writing (M; SD; Min-Max)	C	10, 61 (2,50) (0 - 12)	11,00 (0,91) (9 - 12)	0,59	8,77 (3,09) (0 - 12)	10,33 (1,37) (8 - 12)	0,26
	H	10,93 (0,83) (9 - 12)	11,57 (0,51) (11 - 12)	0,01*	10,87 (1,19) (8 - 12)	10,92 (2,43) (3 - 12)	0,95
Sig.		0,08	0,64		0,03*	0,15
Calculations (M; SD; Min-Max)	C	5,18 (1,66) (2 - 7)	5,39 (2,15) (0 - 7)	0,77	3,40 (1,98) (0 - 7)	4,58 (2,61) (0 - 7)	0,20
	H	6,0 (1,24) (4 - 7)	6,26 (1,10) (4 - 7)	0,51	6,13 (1,25) (3 - 7)	6,31 (0,95) (5 - 7)	0,49
Sig.		0,07	0,61		0,01*	0,05*

• Note. * p ≤ 0,05; M = Mean; SD = Standard Deviation; C = Clinical; H = Healthy.

Table 4
Effects and interactions of group (patients/controls), age, and education on performance in the TRIACOG

Cognitive Domain/Task	P/C (F)	Age	Education (F)	P/C & Age (F)	P/C & Education (F)	Age & Education (F)	P/C & Age & Education (F)
Orientation	8.92*	5.11*	0.05	5.13*	0.09	0.17	0.04
Episodic-Semantic Verbal Memory	16.48*	13.57*	3.72*	0.12	0.77	2.02	0.28
Immediate	7.0*	10.50*	5.58*	1.78	0.09	1.10	0.14
Delayed	14.39*	7.89*	0.70	1.12	1.23	1.93	0.39
Attention/Working Memory	10.21*	4.84*	8.29*	1.86	0.13	0.61	0.64
Praxis	25.84*	8.76*	4.30*	5.16*	0.73	0.62	0.72
Figure Copying	21.79*	7.39*	3.52*	4.96*	1.10	0.40	0.90
Ideomotor Praxis	1.70	3.74*	0.14	2.33	0.14	0.25	0.04
Clock Drawing	20.15*	6.22	3.75*	2.64	0.10	1.53	0.25
Attention/Working Memory	10.21*	4.84*	8.29*	1.86	0.13	0.61	0.64
Executive Functions	11.19*	5.18*	4.71*	2.50	1.60	0.53	0.88
Verbal Fluency	3.71*	6.97*	12.19*	2.77	3.24*	1.03	0.29
Rapid Serial Naming	10.01*	3.63*	2.63*	1.92	0.99	0.42	0.79
Processing Speed	9.61*	7.01*	4.66*	3.60*	0.61	0.21	0.44

• Note. * p ≤ .05; P/C = Patients/Controls.

Table 5
Effects and interactions of group (clinical/control), age, and education on performance in the TRIACOG

Cognitive Domain/Task	P/C (F)	Age	Education (F)	P/C & Age (F)	P/C & Education (F)	Age & Education (F)	P/C & Age & Education (F)
Language	7.66*	3.84*	3.54*	2.27	1.22	0.75	1.00
Listening Comprehension	3.18	0.77	0.53	0.79	0.57	0.17	0.22
Naming	5.66*	2.25	2.41	2.18	1.18	0.24	0.25
Vocabulary/Semantic memory	3.84*	4.24*	3.06*	3.04*	2.52	2.63*	0.23
Reading and inference processing	4.18*	0.51	2.59	0.39	2.75	1.04	1.08
Written comprehension	2.97	1.09	0.14	1.10	0.02	0.70	0.94
Repetition and dictated writing	3.46	5.96*	1.76	3.01*	0.05	0.30	0.94
Calculation	15.11*	2.69	2.57	3.37*	0.22	0.37	0.52
Visual memory	44.64*	15.62*	5.10*	2.43	2.76	0.15	0.81

• Note. * p ≤ .05; P/C = Patients/Controls

Interactions were observed between group (clinical vs control) and age in the cognitive functions of orientation, praxis, calculation, and processing speed; and in the subtests figure copying, vocabulary, and repetition/dictated writing. The variables group and education interacted only in the verbal fluency subtest (Table 4 and 5), with the worst performance being found in participants with post-stroke cognitive impairment and low educational level. Patients with fewer years of formal education, in both age groups, had significantly lower verbal fluency scores than controls.

Discussion

This study investigated how external variables influence results on the TRIACOG. Careful consideration of the variability and interpretation of the results (therefore, of whether these results are useful to hypothesis testing or generation) is an essential step in validating an instrument (AERA, APA & NCME, 2014). The specific type of validity evidence discussed here is based on the relationship with other variables, that is, on the extent to which performance on the TRIACOG is related to presence or absence of stroke, age, and education.

All three variables affected performance on the TRIACOG. We expected such results, since those factors are often found to influence neuropsychological assessments (Bento-Torres et al., 2017; Mancuso et al., 2016). As shown by the interactions, age and education have distinct effects on performance depending on group (clinical vs control).

In this study, all cognitive functions and 70% of the subtests in the TRIACOG showed an effect of group (clinical/control), with the exception of the subtests of ideomotor praxis, listening comprehension, writing comprehension and repetition/dictated writing. This effect consisted in a decrease in the performance of post-stroke individuals. This result was already expected, considering previous meta-analyses describing poorer performance in cognitive screening of patients following cerebrovascular events (Kuzma et al, 2018; Makin et al., 2013) as well as studies by Rodrigues (2017) and Rodrigues, Salles and Bandeira (2021), which provided evidence for the sensitivity of TRIACOG to differentiate patients from healthy individuals.

Age also had an effect on all cognitive functions. Elderly patients had a larger number of impaired functions compared to healthy individuals of the same age group, in both education groups. Additionally, older individuals with fewer years of formal education had a significantly lower performance compared to those with more years of formal education.

Several studies point out that, with advanced age, widespread deficits are observed in multiple cognitive functions. In healthy aging, this reduction in cognitive performance occurs, but does not compromise individuals’ everyday routine (Coco, Lopez & Corrao, 2016; Umarova et al., 2019). Thus, even in the absence of dementia-which substantially impairs the ability to carry out everyday activities - aging alters cognitive abilities, reducing the efficiency of functions such as episodic memory, long-term memory, attention, working memory and executive functions, which are generally the first cognitive functions to show deficits in elderly people (Coco, Lopez & Corrão, 2016).

In this study, strong interactions between group (clinical/control) and age were observed in the domains of orientation, praxis; and in the subtests of figure copying, vocabulary, repetition/dictated writing, calculation, and processing speed. On the other hand, functions such as visual memory, episodic-semantic verbal memory, executive functions, attention/working memory, and language - described above as those that decline earliest in healthy aging - showed the largest main effects of group (clinical/control) and age, with no interactions. Thus, the cognitive functions that generally decline in healthy aging were most strongly affected by the group factor - that is, they were impaired more severely by stroke - regardless of age.

These findings are in line with studies that describe effects of aging on human cognition (Bento-Torres et al., 2017; Bettio, Rajendran & Gil-Mohapel, 2017). Additionally, a combined effect of cerebrovascular event and age is often reported; this was also observed in the current study. More severe and widespread deficits usually affect older individuals, aggravating preexisting impairments and increasing the risk for dementia (Lee et al., 2014). Thus, based on the current results, we hypothesize that cognitive impairment caused by stroke affects more prominently cognitive functions that are more vulnerable to the aging process. In elderly patients, stroke seems to more extensively affect cognitive functions that are already impaired by age-related decline, accentuating deficits that may already exist and compromising other functions, as shown by the larger number of cognitive functions and subtests significantly impaired in older patients compared to younger ones.

A combined effect of education and cerebrovascular condition on executive functions and language tasks has also been reported before (Sarno et al., 2005; Schmidt et al., 2017). The analysis of the verbal fluency subtest of the TRIACOG, as well as of the subprocesses required by this subtest, clearly shows that this task engages cognitive processes such as lexical access, inhibition, and processing speed, skills that are coordinated mainly by executive functions and language (Pereira et al., 2018). Since both of these functions are often impaired by cerebrovascular conditions, our results corroborate earlier findings that demonstrate a relationship between schooling and tasks that require both executive and language skills, such as verbal fluency (Beckenkamp et al., 2019).

We also found that education had an effect on the performance of cognitively healthy participants. Statistically significant differences between individuals with distinct educational levels, in both age groups, were found in a larger number of tasks in the control group, compared to the patient group. This is in line with previous findings that education has a sizeable effect, sometimes larger than the effect of age, on the performance of adult participants in neuropsychological tests (Bento-Torres et al., 2017). These results are also in agreement with those of Parente et al. (2009) and Umarova et al. (2019), who showed that cerebrovascular disease normalizes the effects of higher education, making the results of patients with different educational levels more similar. Although education had a larger effect in the control group, patients with higher levels of education were impaired on a smaller number of cognitive functions compared to those with lower educational level, indicating that more of years of formal studies are an important protective factor for post-stroke cognition (Bento-Torres et al., 2017).

The variables age and education proved to be relevant to understand cerebrovascular conditions, corroborating hypotheses about how cognitive function may be altered by factors other than neurological damage. In addition to reiterating the need for cognitive assessment instruments specific to stroke patients, our data show that diagnosis of post-stroke cognitive deficits can be more accurate and exhaustive when performed using cognitive screening tools that measure performance in each cognitive domain affected. Such instruments allow evaluators to investigate a greater number of neuropsychological profiles and minimizes the rate of false negatives by detecting impaired skills even before the patient leaves the hospital. This represents an investment in primary care of stroke patients, providing accurate data in relatively little time, with low costs and avoiding exposure of patients to risk (such as premature return to work). Moreover, the data obtained with such assessments are highly relevant for later rehabilitation treatments.

Conclusions

Neuropsychological studies have shown that multivariate analyses must be employed to evaluate cognitive deficits after neurological events, considering specific aspects of the injury as well as sociodemographic characteristics, for example. Therefore, it is essential to investigate the validity of the TRIACOG based on the relationship with external variables, since such variables are well established as influential in post-stroke cognitive outcomes.

Our results confirm the need to understand the role of age and, particularly in countries with educational differences such as Brazil, schooling in the variability cognitive deficits caused by cerebrovascular events. Considering how these contextual factors influence performance increases assessment accuracy, thus reducing underdiagnosis.

In this study, participants were selected by convenience, particularly those in the clinical group, with data in this group being collected from patients available at the ECU. This leads to some limitations in terms of gender (studies show a higher incidence of stroke in men, but several aspects such as epidemiological factors, risk factors and cognitive outcomes still need investigation; Bushnell et al., 2018). Furthermore, other variables not controlled for may interfere with performance in this type of instrument. For example, in preliminary studies by Rodrigues (2017), patients who suffered more than one cerebrovascular event displayed worse performance compared to those who had suffered their first stroke.

Thus, further studies are needed to investigate how performance in the cognitive functions and subtests of this instrument changes as a function of neurological variables involved in stroke, as well as associated psychological factors (Santos, Rodrigues, & Salles, 2019). We expected that the TRIACOG will be employed by health professionals to identify patients at risk for vascular cognitive impairment and for other neurological conditions to which this instrument may apply.

References

• American Educational Research Association. (2014). Standards for educational and psychological testing. American Educational Research Association American Psychological Association National Council on Measurement in Education.
• Barker-Collo, S., & Feigin, V. (2006). The impact of neuropsychological deficits on functional stroke outcomes. Neuropsychology Review, 16(2), 53-64. doi: 10.1007/s11065-006-9007-5
  » https://doi.org/10.1007/s11065-006-9007-5
• Beckenkamp, C. L., Becker, N., De Carvalho Rodrigues, J., & Fumagalli de Salles, J. (2019). Estratégias de clustering e switching na evocação lexical de adultos após acidente vascular cerebral nos hemisférios direito e esquerdo. Avances En Psicología Latinoamericana, 37(2), 375. doi: 10.12804/revistas.urosario.edu.co/apl/a.6073
  » https://doi.org/10.12804/revistas.urosario.edu.co/apl/a.6073
• Bento-Torres, N. V. O., Bento-Torres, J., Tomás, A. M., Costa, V. O., Corrêa, P. G. R., Costa, C. N. M., ... & Picanco-Diniz, C. W. (2017). Influence of schooling and age on cognitive performance in healthy older adults. Brazilian Journal of Medical and Biological Research, 50(4). doi: 10.1590/1414-431x20165892
  » https://doi.org/10.1590/1414-431x20165892
• Bettio, L. E., Rajendran, L., & Gil-Mohapel, J. (2017). The effects of aging in the hippocampus and cognitive decline. Neuroscience & Biobehavioral Reviews, 79, 66-86. doi: /10.1016/j.neubiorev.2017.04.030
  » https://doi.org/10.1016/j.neubiorev.2017.04.030
• Blackburn, D. J., Bafadhel, L., Randall, M., & Harkness, K. A. (2013). Cognitive screening in the acute stroke setting. Age and Ageing, 42(1), 113-116. doi: 10.1093/ageing/afs116
  » https://doi.org/10.1093/ageing/afs116
• Bushnell, C. D., Chaturvedi, S., Gage, K. R., Herson, P. S., Hurn, P. D., Jiménez, M. C., … Rundek, T. (2018). Sex differences in stroke: Challenges and opportunities. Journal of Cerebral Blood Flow & Metabolism, 38(12), 2179-2191. doi:10.1177/0271678x18793324
  » https://doi.org/10.1177/0271678x18793324
• Coco, D. L., Lopez, G., & Corrao, S. (2016). Cognitive impairment and stroke in elderly patients. Vascular health and risk management, 12, 105. doi: 10.2147/vhrm.s75306
  » https://doi.org/10.2147/vhrm.s75306
• Folstein, M. F. (1999). The Mini Mental State Examination (MMSE). Director (Cincinnati, Ohio), 7(2), 62. doi:10.1016/j.jclinepi.2006.06.009
  » https://doi.org/10.1016/j.jclinepi.2006.06.009
• Gorenstein, C., Pang, W. Y., Argimon, I. L., & Werlang, B. S. G. (2011). Manual do Inventário de depressão de Beck-BDI-II. São Paulo: Editora Casa do Psicólogo.
• Kochhann, R., Varela, J. S., Lisboa, C. S. D. M., & Chaves, M. L. F. (2010). The Mini Mental State Examination Review of cutoff points adjusted for schooling in a large Southern Brazilian sample. Dementia & Neuropsychologia, 4(1), 35-41. doi: 10.1590/s1980-57642010dn40100006
  » https://doi.org/10.1590/s1980-57642010dn40100006
• Kosgallana, A., Cordato, D., Chan, D. K. Y., & Yong, J. (2019). Use of cognitive screening tools to detect cognitive impairment after an ischaemic stroke: a systematic review. Comprehensive Clinical Medicine, 1(4), 255-262. doi: 10.1007/s42399-018-0035-2
  » https://doi.org/10.1007/s42399-018-0035-2
• Kuźma, E., Lourida, I., Moore, S. F., Levine, D. A., Ukoumunne, O. C., & Llewellyn, D. J. (2018). Stroke and dementia risk: a systematic review and meta-analysis. Alzheimer’s & Dementia, 14(11), 1416-1426. doi: 10.1016/j.jalz.2018.06.3061
  » https://doi.org/10.1016/j.jalz.2018.06.3061
• Lee, M. J., Seo, S. W., Na, D. L., Kim, C., Park, J. H., Kim, G. H., ... & Yoon, C. W. (2014). Synergistic effects of ischemia and β-amyloid burden on cognitive decline in patients with subcortical vascular mild cognitive impairment. JAMA psychiatry, 71(4), 412-422. doi: 10.1016/j.jalz.2017.07.438
  » https://doi.org/10.1016/j.jalz.2017.07.438
• Lourenço, R. A., & Veras, R. P. (2006). Mini-Exame do Estado Mental: características psicométricas em idosos ambulatoriais. Revista de Saúde Pública, 40(4), 712-719. doi:10.1590/s0034-89102006000500023
  » https://doi.org/10.1590/s0034-89102006000500023
• Makin, S. D. J., Turpin, S., Dennis, M. S., & Wardlaw, J. M. (2013). Cognitive impairment after lacunar stroke: systematic review and meta-analysis of incidence, prevalence and comparison with other stroke subtypes. J Neurol Neurosurg Psychiatry, 84(8), 893-900. doi: 10.1136/jnnp-2012-303645
  » https://doi.org/10.1136/jnnp-2012-303645
• Milinavičienè, E., Rastenytė, D., Kriščiūnas, A., Milinavičienè, E., Rastenytė, D., & Kriščiūnas, A. (2011). Effectiveness of the Second-Stage Rehabilitation in Stroke Patients With Cognitive Impairment. Medicina, 47(9), 74. doi: 10.3390/medicina47090074
  » https://doi.org/10.3390/medicina47090074
• Mancuso, M., Varalta, V., Sardella, L., Capitani, D., Zoccolotti, P., Antonucci, G., & Italian OCS Group. (2016). Italian normative data for a stroke specific cognitive screening tool: the Oxford Cognitive Screen (OCS). Neurological Sciences, 37(10), 1713-1721. doi:10.1007/s10072-016-2650-6
  » https://doi.org/10.1007/s10072-016-2650-6
• Nasreddine, Z. S., Phillips, N. a., Bedirian, V., Charbonneau, S., Whitehead, V., Collin, I., … Chertkow, H. (2005). The Montreal Cognitive Assessment, MoCA: A Brief Screening Tool for Mild Cognitive Impairment. Journal of the American Geriatrics Society, 53(4), 695-699. doi: 10.1111/j.1532-5415.2005.53221.x
  » https://doi.org/10.1111/j.1532-5415.2005.53221.x
• Nys, G. M. S., Van Zandvoort, M. J. E., Van Der Worp, H. B., De Haan, E. H. F., De Kort, P. L. M., & Kappelle, L. J. (2005). Early depressive symptoms after stroke: Neuropsychological correlates and lesion characteristics. Journal of the Neurological Sciences , 228(1), 27-33. doi: 10.1016/j.jns.2004.09.031
  » https://doi.org/10.1016/j.jns.2004.09.031
• Pacido, J. C., Hutz, C. S., Schneider, A. M. A., & Bandeira, D. R. (2015). Validade. In C. S. Hutz, D. R. Bandeira, & C. M. Trentini (orgs.). Psicometria (pp. 70-83). Porto Alegre: ArtMed.
• Paranhos, M. E., de Lima Argimon, I. I., & Werlang, B. S. G. (2010). Propriedades psicométricas do Inventário de Depressão de Beck-II (BDI-II) em adolescentes. Avaliação Psicológica, 9(3), 383-392. Recuperado de http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712010000300005
  » http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712010000300005
• Parente, M. A. D. M. P., Scherer, L. C., Zimmermann, N., & Fonseca, R. P. (2009). Evidências do papel da escolaridade na organização cerebral. Neuropsicologia Latinoamericana, 1(1), 72-80. Recuperado de http://pepsic.bvsalud.org/scielo.php?script=sci_abstract&pid=S2075 -94792009000100009&lng=pt&nrm=i#:~:text=Dentre%20os%20principais%20dados%20com,cerebrais%20estruturais%20volum%C3%A9tricas%20e%20funcionais
  » http://pepsic.bvsalud.org/scielo.php?script=sci_abstract&pid=S2075 -94792009000100009&lng=pt&nrm=i#:~:text=Dentre%20os%20principais%20dados%20com,cerebrais%20estruturais%20volum%C3%A9tricas%20e%20funcionais
• Pereira, A. H., Gonçalves, A. B., Holz, M., Gonçalves, H. A., Kochhann, R., Joanette, Y., … Fonseca, R. P. (2018). Influence of age and education on the processing of clustering and switching in verbal fluency tasks. Dementia & Neuropsychologia , 12(4), 360-367. doi:10.1590/1980-57642018dn12-040004
  » https://doi.org/10.1590/1980-57642018dn12-040004
• Patel, M. D., Coshall, C., Rudd, A. G., & Wolfe, C. D. A. (2002). Cognitive Impairment after Stroke: Clinical Determinants and Its Associations with Long-Term Stroke Outcomes. Journal of the American Geriatrics Society , 50(4), 700-706. doi:10.1046/j.1532-5415.2002.50165.x
  » https://doi.org/10.1046/j.1532-5415.2002.50165.x
• Pinto, T. C., Machado, L., Bulgacov, T. M., Rodrigues-Junior, A. L., Costa, M. L. G., Ximenes, R. C., & Sougey, E. B. (2018). Influence of age and education on the performance of elderly in the Brazilian version of the Montreal Cognitive Assessment battery. Dementia and geriatric cognitive disorders, 45, 290-299. doi: 10.1159/000489774
  » https://doi.org/10.1159/000489774
• Rodrigues, J. D. C., Bandeira, D. R., & de Salles, J. F. (2021). TRIACOG: Triagem Cognitiva. São Paulo: Vetor.
• Rodrigues, J. C., Salles, J. F., & Bandeira, D. R. (2021). Evidence of validity based on the relation to other variables and interrater reliability of the Cognitive Screening (TRIACOG) for adults with cere-brovascular diseases, Applied Neuropsychology: Adult, DOI: 10.1080/23279095.2021.1931219
  » https://doi.org/10.1080/23279095.2021.1931219
• Rodrigues, J. D. C., Bandeira, D. R., & de Salles, J. F. (2020). Cognitive Screening (TRIACOG) for adults with cerebrovascular diseases: Construction process and validity evidence. Psychology & Neuroscience. doi:10.1037/pne0000197
  » https://doi.org/10.1037/pne0000197
• Rodrigues, J. C. (2017). Triagem cognitiva nas doenças cerebrovasculares: Processo de construção e propriedades psicométricas do instrumento TRIACOG. Tese de Doutorado não publicada. Programa de Pós-gradução em Psicologia. Universidade Federal do Rio Grande do Sul - UFRGS.
• Saa, J. P., Tse, T., Baum, C., Cumming, T., Josman, N., Rose, M., & Carey, L. (2019). Longitudinal evaluation of cognition after stroke-A systematic scoping review. PloS one, 14(8), e0221735.doi:10.1371/journal.pone.0221735
  » https://doi.org/10.1371/journal.pone.0221735
• Sagnier, S., Munsch, F., Bigourdan, A., Debruxelles, S., Poli, M., Renou, P., & Sibon, I. (2019). The influence of stroke location on cognitive and mood impairment. A voxel-based lesion-symptom mapping study. Journal of Stroke and Cerebrovascular Diseases, 28(5), 1236-1242. doi:10.1016/j.jstrokecerebrovasdis.2019.01.010
  » https://doi.org/10.1016/j.jstrokecerebrovasdis.2019.01.010
• Santos, D. P., Rodrigues, J. D. C., & Salles, J. F. (2019). Influencia de los síntomas de depresión en las funciones neuropsicológicas después del accidente cerebrovascular. Ciencias Psicológicas, 56. doi:10.22235/cp.v13i1.1809
  » https://doi.org/10.22235/cp.v13i1.1809
• Sarno, M. T., Postman, W. A., Cho, Y. S., & Norman, R. G. (2005). Evolution of phonemic word fluency performance in post-stroke aphasia. Journal of communication disorders, 38(2), 83-107. doi: 10.1016/j.jcomdis.2004.05.001
  » https://doi.org/10.1016/j.jcomdis.2004.05.001
• Schmidt, C. S., Schumacher, L. V., Römer, P., Leonhart, R., Beume, L., Martin, M., ... & Kaller, C. P. (2017). Are semantic and phonological fluency based on the same or distinct sets of cognitive processes? Insights from factor analyses in healthy adults and stroke patients. Neuropsychologia, 99, 148-155. doi: 10.1016/j.neuropsychologia.2017.02.019
  » https://doi.org/10.1016/j.neuropsychologia.2017.02.019
• Stelmach, W., Kaczmarczyk-Chałas, K., Bielecki, W., Stelmach, I., & Drygas, W. (2004). How income and education contribute to risk factors for cardiovascular disease in the elderly in a former Communist country. Public health, 118(6), 439-449. doi:10.1016/j.puhe.2003.12.012
  » https://doi.org/10.1016/j.puhe.2003.12.012
• Stern, Y. (2009). Cognitive reserve. Neuropsychologia, 47(10), 2015-2028. doi:10.1016/j.neuropsychologia.2009.03.004
  » https://doi.org/10.1016/j.neuropsychologia.2009.03.004
• Stolwyk, R. J., O’Neill, M. H., McKay, A. J., & Wong, D. K. (2014). Are cognitive screening tools sensitive and specific enough for use after stroke? A systematic literature review. Stroke, 45(10), 3129-3134. doi: 10.1161/strokeaha.114.004232
  » https://doi.org/10.1161/strokeaha.114.004232
• Sun, J. H., Tan, L., & Yu, J. T. (2014). Post-stroke cognitive impairment: epidemiology, mechanisms and management. Annals of translational medicine, 2(8). doi: 10.3978/j.issn.2305-5839.2014.08.05
  » https://doi.org/10.3978/j.issn.2305-5839.2014.08.05
• Tang, E. Y., Amiesimaka, O., Harrison, S. L., Green, E., Price, C., Robinson, L., ... & Stephan, B. C. (2018). Longitudinal effect of stroke on cognition: a systematic review. Journal of the American Heart Association, 7(2), e006443. doi: 10.1161/jaha.117.006443
  » https://doi.org/10.1161/jaha.117.006443
• Umarova, R. M., Sperber, C., Kaller, C. P., Schmidt, C. S., Urbach, H., Klöppel, S., ... & Karnath, H. O. (2019). Cognitive reserve impacts on disability and cognitive deficits in acute stroke. Journal of neurology, 266(10), 2495-2504. doi: 10.1007/s00415-019-09442-6
  » https://doi.org/10.1007/s00415-019-09442-6
• Valentini, F., & Damásio, B. F. (2016). Variância Média Extraída e Confiabilidade Composta: Indica-dores de Precisão. Psicologia: Teoria e Pesquisa, 32(2), e322225. Epub 27 de outubro de 2016. doi: 10.1590/0102-3772e322225
  » https://doi.org/10.1590/0102-3772e322225
• Wei, N., Yong, W., Li, X., Zhou, Y., Deng, M., Zhu, H., & Jin, H. (2015). Post-stroke depression and lesion location: a systematic review. Journal of Neurology, 262(1), 81-90. doi:10.1007/s00415-014-7534-1
  » https://doi.org/10.1007/s00415-014-7534-1

Publication Dates

• Publication in this collection
  06 Jan 2023
• Date of issue
  Oct-Dec 2022

History

• Received
  13 Dec 2020
• Reviewed
  26 Mar 2021
• Accepted
  05 July 2021