Methodological Criteria for Scoring Clustering and Switching in Verbal Fluency Tasks

Critérios Metodológicos para Análises de Clustering e Switching em tarefas de Fluência Verbal

Criterios Metodológicos para Análisis de Estrategias de Agrupación en Tareas de Fluidez Verbal

Abstract

The objective of this study was to describe an adaptation to Brazilian Portuguese of the methodological criteria for analysis of clustering and switching in semantic verbal fluency (SVF) and phonemic verbal fluency (PVF) tasks. The adaptation process consisted of six steps, including the selection of the clustering and switching variables based on data from a sample of 419 children and the analysis of inter-rater reliability (six raters). The following variables were scored: the total number of words generated the raw number of clusters, the mean cluster size, and the raw number of switches. There was a significant association between raters (intra-class correlation coefficients between 0.95 and 0.99), showing that the analytical method was reliable. Our study provides an evaluation of SVF and PVF tasks that goes beyond the overall score, making it possible to investigate the cognitive processes underlying this neuropsychological function.

Keywords:
semantic verbal fluency; phonemic verbal fluency; clustering; switching

Resumo

Este artigo objetiva apresentar uma adaptação ao português brasileiro dos critérios metodológicos para análises de clustering e switching em tarefas de fluência verbal semântica (FVS) e ortográfica (FVO). São descritas as seis etapas realizadas no processo de adaptação destes critérios, incluindo o processo de extração das variáveis de clustering e switching a partir de dados de amostras de crianças (N=419) e a análise da fidedignidade da concordância entre juízes (N=6). As variáveis avaliadas são número total de palavras evocadas, número de clusters, média do tamanho dos clusters e número de switches. Houve forte associação entre juízes (Coeficientes de Correlação Intraclasse entre 0,95 a 0,99) indicando fidedignidade do método de análise. Este estudo contribui com a avaliação da FVS e FVO para além do escore geral, permitindo investigar os processos cognitivos subjacentes a esta função neuropsicológica.

Palavras-chave:
fluência verbal semântica; fluência verbal ortográfica; agrupamentos

Resumen

Este artículo presenta una adaptación para el portugués brasileño de criterios metodológicos de análisis de estrategias de agrupación (clustering y switching) en tareas de fluidez verbal semántica (FVS) y fluidez verbal fonológica (FVF). El proceso de adaptación de estos criterios consiste en seis pasos, incluyendo el proceso de extracción de las variables de clustering y switching de una muestra formada por 419 niños y el análisis de fiabilidad entre seis jueces. Se investigaron las siguientes variables: número total de palabras generadas, número de grupos de palabras (clusters), tamaño medio de los grupos y número de saltos (switches). Hubo una fuerte asociación entre jueces (coeficientes de correlación intraclase entre 0,95 y 0,99), indicando así la fiabilidad del método analítico. Este estudio contribuye a la evaluación de la FVS y FVF más allá de la puntuación global, lo que permite investigar los procesos cognitivos subyacentes a esta función neuropsicológica.

Palabras-clave:
fluencia semántica; fluencia fonológica; estrategias de agrupamientos; clustering; switching

Our study describes the methodological criteria adapted from (^{Troyer, Moscovitch & Winocour, 1997}), (^{Troyer, 2000}), and (^{Lopes, Brucki, Giampaoli, & Mansur, 2009}) for the evaluation of the underlying cognitive components (clustering and switching) in the overall performance on verbal fluency (VF) tasks based on the cognitive neuropsychology approach. VF tasks have been used in psychology for a long time. These tasks were first used by (Thurstone, 1938, cited by ^{Miller, 1984}) and his tests of "primary mental abilities". The overall score of these tasks is widely used in clinical and experimental contexts in individuals of different ages and educational levels (^{Oberg & Ramírez, 2006}) as a diagnostic tool. These tasks are also used in research on normal aging (^{Mayr, 2002}).

VF tasks consist of retrieving as many words as possible based on a phonemic criterion for 60 seconds (^{Strauss, Sherman, & Spreen, 2006}). In PVF tasks, these words should begin with a specific letter. Several different letters can be used. In the literature, the most frequently used letters are: F, A, and S; or C, F, and L; or even P, R, and W (^{Oberg & Ramírez, 2006}). In SVF tasks, the words retrieved should belong to a certain category, such as fruits, animals and clothes; whereas animals is the most commonly used category in the literature (^{Charchat-Fichman, Oliveira, & Silva, 2011}). Most Brazilian studies on VF include SVF tasks using the animal category (^{Araujo et al., 2011}; ^{Brucki, Malheiros, Okamoto, & Bertolucci, 1997}; ^{Brucki & Rocha, 2004}; ^{Caramelli, Carthery-Goulart, Porto, Charchat-Fichman, & Nitrini, 2007}; ^{Lopes et al., 2009}; ^{Silva, Yassuda, Guimarães, & Florindo, 2011}). There is a small number of studies using fruits (^{Paula et al., 2010}), body parts (^{Malloy-Diniz et al., 2007}), and clothes (^{Fonseca, Parente, Côté, Ska, & Joanette, 2008}; ^{Moraes et al., 2014}; ^{Zimmermann, Parente, Joanette, & Fonseca, 2014}). PVF tasks are also mentioned in national studies using the letters F, A, and S (^{Charchat-Fichman et al., 2011}; ^{Fonseca, Salles, & Parente, 2009}; ^{Machado et al., 2009}), the letter P (^{Fonseca et al., 2008}; ^{Moraes et al., 2014}; ^{Zimmermann et al., 2014}), and the letter M (^{Charchat-Fichman et al., 2011}; ^{Salles et al., in press}).

VF tasks, both PVF and SVF tasks, evaluate language skills, such as lexical retrieval, vocabulary size, and lexical access. That is, the ability to access the semantic lexicon and retrieve the semantic and formal information of the words (^{Jaichenco, Wilson, & Ruiz, 2007}). These tasks also include the evaluation of the access to the working and semantic memories (^{Lezak, Howeison, & Loring, 2004}). In addition, these tasks assess speed and facilitation of verbal production and response, mental organization, search strategies, and ability to initiate a behavior as a response to a new activity. These characteristics are related to the following components of executive functions: volition, flexibility, and inhibitory control.

VF performance improves throughout childhood and adolescence, but the peak of development varies according to the type of task. In terms of this variation, international studies have found significant differences in the SVF performance between 7-8 years old and 9-10 years old; whereas, when it comes to PVF, the number of words retrieved significantly increases starting at 11-12 years (^{Sauzéon, Lestage, Raboutet, Kaoua, & Claverie, 2004}; ^{Tallberg, Carlsson, & Liberman, 2011}). The literature shows that both adults and children have a better performance in SVF tasks in comparison with PVF tasks (^{Sauzéon et al., 2004}; ^{Strauss et al., 2006}). This performance pattern is explained by the different cognitive components recruited during the tasks. PVF tasks are associated with increased use of search strategies to retrieve words; therefore, these tasks depend more on the executive functions (such as cognitive flexibility and switching), which require greater cognitive effort (^{Sauzéon et al., 2004}). As for SVF tasks, mental imaging strategies (imagining scenarios based on the category of words) facilitate response retrieval. In addition, responses to SVF seem to be more related to the lexical-semantic networks that are part of the semantic memory (^{Charchat-Fichman et al., 2011}), which develops prior to the executive functions. Thus, the improvement of SVF is more apparent in early childhood. The subsequent occurrence of changes in the performance of PVF tasks compared with SVF can be explained in terms of development of strategic search skills, cognitive flexibility, switching, and inhibitory control. All these components are dependent on executive functions and the maturation of the frontal lobes, which reaches its peak between 10 and 12 years old (^{Tallberg et al., 2011}). In addition, PVF is also dependent on educational level and growth of the orthographic lexicon (^{Charchat-Fichman et al., 2011}).

Studies of patients with brain lesions have suggested that impaired performances on PVF tasks are related to frontal lobe lesions, particularly in the left hemisphere (^{Birn et al., 2010}; ^{Troyer, Moscovitch, Winocur, Alexander, & Stuss, 1998}; ^{Troyer, Moscovitch, Winocur, Leach, & Freedman, 1998}). In contrast, SVF tasks require the use of a smaller number of sets of words, because these tasks recruit words within a particular semantic category (^{Troyer et al., 1997}). In this case, studies using the lesion paradigm have shown impaired performances on these tasks associated with lesions in the temporal lobes (^{Birn et al., 2010}; ^{Troyer, Moscovitch, Winocur, Alexander et al., 1998}; ^{Troyer, Moscovitch, Winocur, Leach et al., 1998}).

The score analyzed in VF tasks usually is the total number of correct words generated within a certain amount of time (^{Strauss et al., 2006}). However, this score does not provide much information on the cognitive processes involved in the performance of fluency. Additionally, it does not answer the question of why a particular group of patients has impaired performance on the task (^{Troyer et al., 1997}). One of the reasons why fluency is a multifactorial process can be related to the theories of lexical access (^{Levelt, Roelofs, & Meyer, 1999}; ^{Rapp & Goldrick, 2006}). Briefly, these theories propose that the word production originates from a complex process that involves the activation of three representational levels: semantic, lexical, phonological levels. These levels interact simultaneously translating a concept into a set of phonemes with mediation of lexical forms. Therefore, changes in any of these levels could lead to impaired performance on VF tasks. Hence, the overall score of the PVF and SVF tasks does not show which of these levels are impairing the overall performance. Only an analysis of the cognitive strategies used during retrieval is able to reveal the deficient processes.

Some authors have suggested the use of qualitative assessment of the performance on VF tasks for the analysis of the cognitive processes underlying these tasks (^{Troyer, 2000}; ^{Troyer et al., 1997}). In addition to the qualitative analysis of types of errors (e.g., perseverative vs. non perseverative errors), (^{Troyer et al., 1997}) proposed another method of qualitative and quantitative analysis: analyses of clustering and switching. The analysis of clustering involves the phonemic analysis in PVF, whereas the analysis of switching involves the semantic categorization in SVF. Both the clustering and switching processes are relatively automatic. Clustering is related to semantic memory, especially lexical storage of words. Switching involves search processes and cognitive flexibility in order to switch from one subcategory to another. This is a controlled process related to executive functions. (^{Troyer, 2000}) argues that, in order to achieve good overall performance on VF tasks, the individual needs to generate words within the same subcategory and only switch to the next category after using all available words in the previous category. In the case of PVF using the phonemic criterion of letter "M", the process would consist of using the words starting with "MO" followed by those starting with "MA", as shown in this sequence: "moon, mother, maple, mate, marmot, mango, mail, married, and mad." In the SVF task using the semantic category of animals, an example containing balancing of strategies included the following retrieved words: "giraffe, alligator, ladybug, butterfly, eagle, and dove." This pattern would correspond to switching between subcategories of wild animals, insects, and birds.

Several studies have been conducted using this method. There are international studies of children samples (^{Hurks et al., 2010}; ^{Nieto, Galtier, Barroso, & Espinosa, 2008}; ^{Sauzéon et al., 2004}; ^{Tallberg, et al., 2011}) and adult samples (^{Clark et al., 2012}; ^{Raboutet et al., 2010}; ^{Troyer, 2000}; ^{Troyer et al., 1997}) with typical development. There are studies using samples of neurological patients with Parkinson's disease (^{Anzak et al., 2011}; ^{Troyer et al., 1998}), Alzheimer's disease (^{Troyer, Moscovitch, Winocur, Leach et al., 1998}; ^{Zhao, Guo, & Hong, 2013}), Huntington's disease (^{Rich, Troyer, Bylsma, & Brandt, 1999}), stroke (^{Vivas & Naveira, 2010}), and brain lesions without specific causes (^{Okruszek, Rutkowska, & Wilinska, 2013}; ^{Stuss, et al., 1998}; ^{Tröster et al., 1998}; ^{Troyer et al., 1998}).

There are fewer Brazilian studies using this method in VF tasks. We could find studies of adults (^{Brucki & Rocha, 2004}) and healthy elderly (^{Silva et al., 2011}), elderly with mild cognitive decline (^{Bertola et al., 2014}), patients with Alzheimer's disease (^{Lopes et al., 2009}), patients with lesion in the right hemisphere (^{Becker, Muller, Rodrigues, Villavicencio, & Salles, 2014}), and children with anxiety disorders (^{Toazza et al., 2014}). However, we could not find specific studies discussing the criteria used to make up clusters and switching analysis between them. The original studies by Troyer et al. (^{Troyer et al., 1997}; ^{Troyer, 2000}) established some criteria for the formation of phonemic and semantic subcategories; however, they are adapted to the Canadian context (English), instead of the Brazilian context. Brazilian studies have shortly explained how the categories were determined, but failed to list the items included in each category.

Therefore, the objective of this study was to describe an adaptation to Brazilian Portuguese of the analysis of clustering and switching in VF tasks according to the proposal designed by (^{Troyer et al., 1997}), (^{Troyer, 2000}), and (^{Lopes et al., 2009}), and to investigate its reliability. Specifically, this paper aims to: 1) describe the selection of the clustering and switching variables based on data from a children sample; and 2) investigate the inter-rate reliability. Therefore, we tried to describe the criteria adapted to the Brazilian context for the analysis of clustering and switching in PVF and SVF tasks for clinical use in childhood.

Method

Participants, instrument, and procedures will be described step by step. The selection of the clustering and switching variables was according to the following steps: 1) development of a database of words generated by the sample of children; 2) determination of the categories for PVF and SVF; 3) scoring words and selection of the variables by the raters independently; 4) analysis of inter-rater reliability; 5) new scoring of discordant variables; and 6) final version of the scoring variables.

Step 1: Development of a Database of Generated Words

The selection of the clustering and switching variables was based on the words generated by 419 children on a PVF task and a SVF task. These participants were from the standardization sample of the "Child Brief Neuropsychological Assessment Battery - NEUPSILIN-INF" (^{Salles, Fonseca, Cruz-Rodrigues, Mello, Barbosa, & Miranda, 2011}; ^{Salles, Sbicigo, Machado, Miranda, & Fonseca, 2014}), they were 6-12-year primary school students from public and private schools located in the state of Rio Grande do Sul, Brazil. This project was approved by the Research Ethics Committee (information removed by the journal), and the children's parents allowed them to participate in the study by signing a written consent form.

The VF tasks used in our study were according to the Child Brief Neuropsychological Assessment Battery - NEUPSILIN-INF. The PVF task used the letter M as the phonemic criterion, whereas the SVF task used the category animals as the semantic criterion. Both tasks used 60 seconds for word retrieval. All the words retrieved by each participant were entered into an Excel spreadsheet in the order in which they were generated, including errors (repetitions and intrusions).

Step 2: Determination of the categories for PVF and SVF

Six raters participated in the selection of the clustering and switching variables. Four of them were undergraduate students of psychology, one was a Master's degree student in psychology with training in neuropsychology, and one was a university professor, who was a speech therapist with expertise in cognitive neuropsychology and language. Based on the criteria established in the study by (^{Troyer et al., 1997}) and (^{Lopes et al., 2009}) for the formation of clustering and switching variables, the raters discussed the necessary adaptations to the Brazilian context for the PVF and SVF tasks. Initially, the raters independently scored the words generated by 50 participants in each task. The goal was to adapt the categories created by (^{Troyer et al., 1997}) for the PVF and SVF tasks and the categories created by (^{Lopes et al., 2009}) for the SVF task based on the retrieval patterns of the sample. The initial scoring included the original categories of the studies. Next, the raters discussed which of these categories would actually be used in the study and which new items would be included in each category.

Step 3: Scoring Words and Selection of Variables

After defining the categories, four raters independently scored the words generated by 419 participants in each VF task. Three variables were selected for each VF task related to the components of clustering and switching: the raw number of clusters, the mean cluster size, and the raw number of switches. In addition, we also analyzed the classical measure: the total number of words generated according to the standards determined by (^{Strauss et al., 2006}). For each participant, three raters scored the variables independently.

Step 4: Analysis of Inter-Rater Reliability

Intraclass correlation coefficient (ICC) analyses were performed to investigate the reliability of each variable analyzed by the four raters. We used the statistical package SPSS 20.0. The significance level was set at 5%. ICC values higher than or equal to 0.75 were considered to be excellent correlations (^{Shrout & Feliss, 1979}).

Step 5: New Scoring of Discordant Items

For items where there was no agreement between at least two raters, the scoring was performed again by three raters independently. With that purpose, the raters were trained again by the principal researcher. This procedure was based on the original study by (^{Troyer et al., 1997}) in which the principal researcher was also considered the gold standard rater.

Step 6: Final Version of the Scoring Variables

After performing all these steps, we determined the criteria for scoring each of the six variables of analysis (the raw number of clusters, the mean cluster size, and the raw number of switches in the PVF and SVF tasks). Three categories were established for the formation of clustering and switching on the PVF task and six categories on the SVF task.

Results and Discussion

Steps 1 and 2: Development of Database of Generated Words and Determination of the Categories for PVF and SVF

Based on the database developed in Step 1, we conducted Step 2 (determination of the categories for PVF and SVF). In the original study by ^{Troyer et al. (1997}), four categories were used to select the clustering and switching variables in the PVF task: words starting with the same two letters (e.g.: arm and art), words that rhyme (e.g.: sand and stand), words that differ only by a vowel sound (e.g.: sat and seat and soot), and homonymous words (e.g.: sum and some). In the PVF task, there was not need to implement many changes to the patterns used in the Canadian study; we only removed the category of homonymous words because this type of words is not common in Portuguese and there were not cases in this sample. Thus, the following chosen categories were: words starting with the same two letters, words that rhyme, and words starting and ending with the same sounds, differing by a vowel sound (examples in Table 1).

Table 1
Criteria for Phonemic Clustering Score in the PVF Task

Phonemic Criteria	Description	Example
Same two letters	Words starting with the same two letters	mail, maple, mango melon, mentor, memory
Rhymes	Words that rhyme	mansion, melon smaller, bigger
Starting and ending with the same sounds	Words that differ only in vowel sounds, keeping the first and last letters	meal, mail mesh, mash

The SVF task required greater changes to the original categories, mainly because of the use of unusual animals in the Brazilian context when compared to the original Canadian study. The Canadian study (^{Troyer et al., 1997}) used the following groups: living environment (Africa, Australia, Arctic/Far North, Farm, North America, Water), human use (beasts of burden, fur, pets), and zoological categories (bird, bovine, canine, deer, feline, fish, insect, insectivores, primate, rabbit, reptile or amphibian, rodent and weasel). Many of these categories did not appear in the generations (such as weasels, Australian animals, etc.) in our study sample, or these categories were too broad. In these cases, the whole sequence of the participant's retrieval fit in only one category (one cluster). There was not variation of patterns (variations of clusters, that is, switching between clusters or single words), as it is the case for the zoological categories, which includes almost all kinds of animals. Therefore, this would not be an illustrative analysis of strategies underlying lexical retrieval.

(^{Lopes et al., 2009}) studied a Brazilian sample of adults and adapted and refined the categories proposed by (^{Troyer et al., 1997}) because of this problem. These authors used the following categories: wild, domestic environment, breeding, small (for arthropods and such like), winged and aquatic animals. We discussed the use of these categories, and we further refined them because we could not find sequences involving the pattern of small animals in our data. Also, we chose to include the category insects because this was a common pattern in our sample. Therefore, we included the living environment animals mentioned by (^{Troyer et al., 1997}) in the wild, and we removed animals that are unusual in the Brazilian context, such as bison and musk ox. The following categories were the same as those in the study by (^{Lopes et al., 2009}): domestic environment, breeding, winged, and aquatic animals (see Table 2 for the final scoring).

Table 2
Criteria for Semantic Clustering Score in the SVF Task

Semantic Criteria	Description	Examples
Wild animals	Animals that live preferably in free environments and can be land or water animals.	Elk, tapir, baboon, sloth, buffalo, camel, chameleon, kangaroo, beaver, cheetah, chimpanzee, snake, rabbit, coyote, crocodile, dinosaur, elephant, emu, squirrel, seal, ferret, skunk, giraffe, gorilla, hippo, hyena, alligator, ocelot, python, leopard, lion, lemur, lynx, wolf, monkey, platypus, porcupine, manatee, penguin, puma, panther, fox, rhino, snake, anteater, tiger, bear, polar bear, deer, zebra.
Aquatic animals	Animals that live in aquatic environments and can be sea animals or not.	Whale, shrimp, crab, seal, dolphin, lobster, sea lion, wolf-fish, otter, calamari, oyster, hedgehog, fish, manatee, swordfish, penguin, octopus, salamander, salmon, toad, turtle, trout, shark.
Breeding animals	Animals that assist people to make their living. They are usually found in rural environments, such as farms.	Ox, goat, donkey, buffalo, goat, camel, sheep, horse, rabbit, chicken, rooster, boar, donkey, llama, mule, sheep, duck, turkey, pig, bull, cow.
Domestic environment animals	Found in the domestic environment (pets or those living in the home garden).	Spider, dog, turtle, cockatiel, canary, snail, rabbit, ant, cat, hamster, tortoise, lizard, snail, worm, fly, parrot, frog, parakeet, guinea pig, flea, rat, mouse, turtle, three-banded armadillo.
Winged animals	Animals that belong to the zoological class of birds.	Eagle, parrot, bird, ostrich, great kiskadee, canary, condor, hawk, goose, heron, mallard, parrot, bird, duck, peacock, pelican, parakeet, turkey, woodpecker, penguin, chick, southern lapwing, finch, toucan, vulture.
Insects	Animals that belong to the zoological class of insects.	Bee, cockroach, beetle, butterfly, ladybug, praying mantis, hornet, fly, mosquito.

Step 3: Scoring Words and Selection of Variables

The criteria for variable selection were the same as that of the original study by (^{Troyer et al., 1997}). See below the description of the selection process for each variable.

Raw number of clusters (categories): sum of all clusters of each participant. Clusters are groups of words generated successively and belonging to the same category. There is a cluster when at least two words are successively generated in the same category (e.g., macaroni, mango, and mittens - the first two words make up a cluster and the latter is considered a single word). Single words are not considered to be a cluster.

Mean cluster size: sum of the words of a cluster starting with the second word generated (e.g., model, morning, and motion is a cluster of two words), dividing this number by the total number of clusters generated by the participant.

Raw number of switches: sum of the number of switches between the clusters, also including exchanges between single words, which are not included in the two previous scores. The sequence mail, machine, madam, meal, mule, music has two clusters (made up by words beginning with ma and mu and a single word between them, with three switches, i.e., three strategy changes).

Errors and repetitions are included in the calculation of these scores because they provide information about the cognitive processes underlying the generated words (^{Troyer et al., 1997}). The score total number of words generated (quantitative analysis) was also analyzed independently by the raters using the criteria shown below.

Total number of correct words generated: sum of the words retrieved during the limit of time set for the task, excluding errors and repetitions. For both tasks, errors are considered to be morphological derivatives of the same word, such as gender variations (e.g.: boy, girl / rooster, chicken), variations of number without representing collectives (e.g.: boy, boys / rooster, roosters), variations of size (e.g.: boy, little boy / cat, kitten), different forms of the same verb (e.g.: bite, bites, bit) and names (^{Salles et al., in press}).

Step 4: Analysis of Inter-Rater Reliability

Table 3 shows the ICCs for the raw number of clusters, the mean cluster size, the raw number of switches, and the total number of words generated in the PVF and SVF tasks evaluated by the four raters independently. The ICC for the PVF task showed high values (the lowest value for the confidence interval was 0.91), demonstrating excellent inter-rater reliability. The ICC values for the same variables in the SVF task were also satisfactory because the lowest value for the confidence interval was 0.90.

Table 3
Results of the Intraclass Correlation Coefficient Analysis in the Scores of PVF and SVF Tasks

Task	Variables	Intraclass correlation coefficient	95% CI	p
PVF	Total no. of words generated	0.99	0.990-0.997	< 0.001
	No. of clusters	0.98	0.97-0.99	< 0.001
	Mean cluster size	0.99	0.997-0.999	< 0.001
	No. of switches	0.99	0.996-0.999	< 0.001
SVF	Total no. of words generated	0.99	0.98-0.99	< 0.001
	No. of clusters	0.97	0.95-0.99	< 0.001
	Mean cluster size	0.95	0.90-0.97	< 0.001
	No. of switches	0.98	0.97-0.99	< 0.001

• Note. Total nº. of words generated = total number of words generated; Nº of clusters = raw number of clusters; Nº of switches = raw number of switches; CI = confidence interval.

Step 5: New Scoring of Discordant Items

After performing the ICC analyses, the raters participated in additional training sessions with the purpose of scoring the items again when there was no agreement between at least two raters (considering the three raters who analyzed each task). In the PVF task, there was no need to discuss the rules because two out of three raters agreed on the values of variables during the independent analysis.

In the SVF task, it was necessary to revise some items, mainly in terms of difference between wild animals - the broadest category of the rules - and aquatic and winged, because of the overlapping of animals included in the latter categories when compared to the former category. There was disagreement also regarding the category domestic environment and winged, such as, for example, the sequence dog, cat, birdie, bird, and ostrich. In this case, the word birdie should be included in two clusters; in the first cluster of domestic environment and in the second cluster of winged animals. In order to resolve, the items showing disagreement were reviewed by the research team, the rules for scoring were discussed again and the score were determined independently. The final value of each variable was the one agreed upon by at least two raters after this second analysis.

Step 6: Final version of the scoring variables

Criteria for Cluster Scoring

According to (^{Troyer et al., 1997}) and (^{Troyer, 2000}), clusters are groups of words retrieved successively and belonging to the same category. A cluster must include at least two words retrieved successively belonging to the same category. For example, model, morning, and motion is a cluster of two words (because the number of words per cluster is always considered starting with the second word), considering the PVF task.

When two clusters overlap, i.e., some of the words retrieved in the sequence may be included in more than one category, and other words exclusively belong to one of the categories, the words that overlap are considered twice in each of the corresponding clusters. In other words, these words will be counted twice, the first time in the first cluster and again in the second cluster, that is, in the second category. For instance, in the sequence mail, mass, mansion, melon; the word mansion is counted in the cluster of mail, mass, mansion (words starting with the same two letters) and in the cluster mansion, melon (words that rhyme), scoring two clusters, one with two words and another one with one word.

In cases where smaller clusters are encompassed by larger clusters, only the largest cluster should be considered, encompassing all words. For example, in the sequence hippo, monkey, camel, and rabbit the last two words could form a cluster of breeding animals, but because they may also belong to the group of wild animals, they are considered only for the latter, totaling a cluster of three words. Thus, in these cases, we should always choose the category that encompasses more words (^{Troyer et al., 1997}).

1. Phonemic verbal fluency: phonemic clusters are considered successively retrieved words that share the categories presented in Table 1.

2. Semantic verbal fluency: semantic clusters are considered successively retrieved words belonging to the same semantic categories. These words were arranged as described in Table 2. Importantly, there are words that can belong to more than one category, such as camel, which appears in the categories wild and breeding animals. In such cases, the words preceding and following the generation of the word camel in the sequence should be taken into account to decide whether that word will be counted in wild or breeding animals, or even in both, as explained above in the criteria for cluster formation.

Final Considerations

The present study describes the steps included in the analyses of clustering and switching in PVF and SVF tasks adapted to the Brazilian context. We also assessed the inter-rater reliability of this adaptation. The results of inter-rater agreement were satisfactory for both VF tasks (PVF and SVF), demonstrating that the adaptation performed to determine the clustering and switching scoring was consistent, at least considering the context of data collection using a sample of children. We attempted to contribute to the analysis of the cognitive components underlying VF performance to assist in refining the diagnosis and prognosis of neuropsychological performances. Up to date, the Brazilian studies using this method (^{Becker et al., 2014}; ^{Bertola et al., 2014}; ^{Brucki & Rocha, 2004}; ^{Lopes et al., 2009}; ^{Silva et al., 2011}; ^{Toazza et al., 2014}) to answer the research questions did not describe the scoring rules in detail so as to enable accurate replication.

The original study by (^{Troyer et al., 1997}) is available in the international literature, providing a detailed description of this method development. The authors described the procedures required for the analysis of the following variables: the raw number of clusters, the mean cluster size, and the raw number of switches, as well as the categories considered for the phonemic analysis in the PVF task and the semantic analysis in the SVF task. However, because the study was conducted in Canada, the categories used required adaptation to the Brazilian context because cultural factors have an influence on the performance of VF tasks (^{Oberg & Ramírez, 2006}).

Other authors have sought to expand the qualitative scoring system created by (^{Troyer et al., 1997}). (^{Abwender, Swan, Bowerman, & Connolly, 2001}) extended the system used by (^{Troyer et al., 1997}) including other categories and using phonemic and semantic clusters in PVF tasks. Furthermore, these authors argued that switching between single words (hard switching) reflects only an inability to perform clustering, whereas switching between clusters (cluster switching) reflects cognitive flexibility and switching skills, thus distinguishing between these two scores. However, (^{Ross et al., 2007}) compared the two methods (^{Troyer et al., 1997}; ^{Abwender et al., 2001}) to check the reliability and validity of the qualitative analysis methods. The authors found good reliability rates related to the results of the original study by (^{Abwender et al., 2001}); however, there was no evidence that this method of analysis was better than the method proposed by (^{Troyer et al., 1997}). Therefore, we chose to keep the scoring used in the original method. Nevertheless, future studies should further investigate the reliability and validity of the method because no consensus has been reached (^{Ross et al., 2007}).

Our study describes in detail the steps required for the analysis of clustering and switching scoring adapted to Brazilian Portuguese with strong inter-rater correlation, demonstrating the reliability of the method. However, there are some limitations in our adaptation. It is worth mentioning that our results should be used with caution in adult populations, considering that the data we used for the analysis of clustering and switching components are derived from a children sample aged 6-12 years. These children are still developing their language skills (especially in terms of vocabulary) and the maturation of brain structures that mediate the processes related to VF and language. In addition, phonemic analysis in the SVF task and semantic analysis in the PVF could also have been explored in order to adapt to the Brazilian context. International studies have also used this type of analysis when investigating the components of clustering and switching (^{Sauzéon et al., 2004}; ^{Tallberg et al., 2011}). However, the adaptation of the method carried out in our study is the initial step to reinforce the use of PVF and SVF tasks, which is important to stimulate other researchers to further develop adaptations of appropriate methods for the Brazilian population.

It is noteworthy that VF tasks consist of a quick and cheap method for clinical use; nevertheless these tasks offer in-depth analyzes showing the functioning of complex cognitive processes. In that sense, there is need of analyses that go beyond the overall score of the task, and the clustering and switching scores proves helpful to achieve deep understanding of the task results (^{Troyer, Moscovitch, Winocur, Alexander et al., 1998}). Information about the cognitive processes underlying the overall performance can contribute to further studies because it is possible to understand normal cognitive functioning and brain-behavior relations, thus providing better diagnosis and treatment of neuropsychological deficits (^{Rapp & Goldrick, 2006}).

In future studies, we intend to compare the results of this adaptation with computer programming in order to get a new measure of reliability, a process that could facilitate the work of researchers when scoring clustering and switching variables. In order to achieve that, a syllable separation system is used, i.e., the syllables of each emission are separated automatically using the system proposed by (^{Oliveira, 2007}). After this step, two different approaches are used to accomplish clustering. In the first approach, the researchers check if the first syllable of an emission is equal to the first syllable of the following emission. In the second approach, the researchers check if there is rhyme between two consecutive emissions. Also, the data can be modeled as a network by analyzing graphs, using computing techniques (^{Albert & Barabasi, 2002}). This method has been used to analyze word association (^{Steyvers & Tenenbaum, 2005}; ^{Zortea, Menegola, Villavicencio, & Salles, 2014}), as well as in studies of clustering and switching analysis in the SVF tasks of adults with right hemisphere lesions (^{Becker et al., 2014}) because it provides greater understanding of the semantic associations between words and lexical-semantic differences between individuals.

References

• Abwender, D. A., Swan, J. G., Bowerman, J. T., & Connolly, S. W. (2001). Qualitative analysis of verbal fluency output: Review and comparison of several scoring methods. Assessment, 8(3), 323-336. doi: 10.1177/107319110100800308
  » https://doi.org/10.1177/107319110100800308
• Albert, R., & Barabasi, A. (2002). Statistical mechanics of complex networks. Reviews of Modern Physics, 74, 47-97. doi: 10.1103/RevModPhys.74.47
  » https://doi.org/10.1103/RevModPhys.74.47
• Anzak, A., Gaynor, L., Beigi, M., Limousin, P., Hariz, M., Zrinzo, L., Foltynie, T., Brown, P. et al. (2011). A gamma band specific role of the subthalamic nucleus in switching during verbal fluency tasks in Parkinson's disease. Experimental Neurology, 232, 136-142. doi: 10.1016/j.expneurol.2011.07.010
  » https://doi.org/10.1016/j.expneurol.2011.07.010
• Araujo, N. B., Barca, M. L., Engedal, K., Coutinho, E. S. F., Deslandes, A. C., & Lacks, J. (2011). Verbal Fluency in Alzheimer's disease, Parkinson's disease and major depression. Clinics, 66(4), 623-627. doi: 10.1590/S1807-59322011000400017
  » https://doi.org/10.1590/S1807-59322011000400017
• Becker, N., Muller, J. L., Rodrigues, J. C., Villavicencio, A., & Salles, J. F. (2014). Estratégias de evocação lexical com critério semântico em adultos após Acidente Vascular Cerebral no hemisfério direito. Letrônica, 7, 325-347. doi: 10.15448/1984-4301.2014.1
  » https://doi.org/10.15448/1984-4301.2014.1
• Bertola, L., Lima, M. L. C., Romano-Silva, M. A., Moraes, E. N., Diniz, D. S., & Malloy-Diniz, L. F. (2014). Impaired generation of new subcategories and switching in a semantic verbal fluency test in older adults with mild cognitive impairment. Frontiers in Aging Neuroscience, 6, 1-6. doi:10.3389/fnagi.2014.00141
  » https://doi.org/10.3389/fnagi.2014.00141
• Birn, R. M., Kenworthy, L. K., Case, L., Caravella, R., Jones, T. B., Bandettini, P. A., & Martin, A. (2010). Neural systems supporting lexical search guided by letter and semantic cattegory cues: A self-paced overt response fMRI study of verbal fluency. NeuroImage, 49, 1099-1107. doi: 10.1016/j.neuroimage.2009.07.036
  » https://doi.org/10.1016/j.neuroimage.2009.07.036
• Brucki, S. M. D., Malheiros, S. M. F., Okamoto, I. H., & Bertolucci, P. H. F. (1997). Dados normativos para o teste de fluência verbal categoria animais em nosso meio. Arquivos de Neuropsiquiatria, 55(1), 56-61. Retrieved from http://www.scielo.br/scielo.php?pid=%22S0004-282X1997000100009%22&script=sci_arttext
  » http://www.scielo.br/scielo.php?pid=%22S0004-282X1997000100009%22&script=sci_arttext
• Brucki, S. M. D., & Rocha, M. S. G. (2004). Category fluency test: effects of age, gender and education on total scores, clustering and switching in Brazilian Portuguese-speaking subjects. Brazilian Journal of Medical and Biological Research, 37, 1771-1777. Retrieved from http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0100-879X2004001200002
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0100-879X2004001200002
• Caramelli, P., Carthery-Goulart, M. T., Porto, C. S., Charchat-Fichman, H., & Nitrini, R. (2007). Category fluency as a screening test for Alzheimer disease in illiterate and literate patients. Alzheimer Disease Associations Disorders, 21(1), 65-67. doi: 10.1097/WAD.0b013e31802f244f
  » https://doi.org/10.1097/WAD.0b013e31802f244f
• Charchat-Fichman, H., Oliveira, R. M., & Silva, A. M. (2011). Performance of Brazilian children on phonemic and semantic verbal fluency tasks. Dementia & Neuropsychologia, 5(2), 78-84. Retrieved from http://www.demneuropsy.com.br/detalhe_artigo.asp?id=269
  » http://www.demneuropsy.com.br/detalhe_artigo.asp?id=269
• Clark, L. R., Scheihser, D. M., Weissberger, G. H., Salmon, D. P., Delis, D. C., & Bondi, M. W. (2012). Specific measures of executive function predict cognitive decline in older adults. Journal of The International Neuropsychological Society, 18(1), 118-127. doi: 10.1017/S135561771100152
  » https://doi.org/10.1017/S135561771100152
• Fonseca, R. P., Parente, M. A. M. P., Côté, H., Ska, B., & Joanette, Y. (2008). Bateria Montreal de Avaliação da Comunicação - Bateria MAC. São Paulo: Pró-Fono.
• Fonseca, R. P., Salles, J. F., & Parente, M. A. M. P. (2009). Instrumento de Avaliação Neuropsicológica Breve NEUPSILIN. São Paulo: Vetor.
• Hurks, P. P. M., Schrans, D., Meijs, C., Wassenberg, R., Feron, F. J. M., & Jolles, J. (2010). Developmental changes in semantic verbal fluency: Analyses of word productivity as a function of time, clustering, and switching. Child Neuropsychology, 16(4), 366-387. doi: 10.1080/09297041003671184
  » https://doi.org/10.1080/09297041003671184
• Jaichenco, V., Wilson, M., & Ruiz, A. (2007). Evaluación del lenguaje. In D. I. Burin, M. A. Drake & P. Harris (Eds.). Evaluación neuropsicológica en adultos (pp. 213-241). Buenos Aires: Paidós.
• Levelt, W. J. M., Roelofs, A., & Meyer, A. S. (1999). A theory of lexical access in speech production. Behavioral and Brain Sciences, 22, 1-75. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.104.7511&rep=rep1&type=pdf
  » http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.104.7511&rep=rep1&type=pdf
• Lezak, M. D., Howieson, D. B., & Loring, D. W. (2004). Neuropsychological Assessment. Oxford: Oxford University Press.
• Lopes, M., Brucki, S. M. D., Giampaoli, V., & Mansur, L. L. (2009). Semantic Verbal Fluency test in dementia: preliminary retrospective analysis. Dementia & Neuropsychologia, 3(4), 315-320. Retrieved from http://www.demneuropsy.com.br/detalhe_artigo.asp?id=187
  » http://www.demneuropsy.com.br/detalhe_artigo.asp?id=187
• Machado, T. H., Charchat-Fichman, H., Santos, E. L., Carvalho, V. A., Fialho, P. P., Koenig, A. M., ... Fernandes, C. S. (2009). Normative data for healthy elderly on the phonemic verbal fluency task - FAS. Dementia & Neuropsychologia, 3(1), 55-60. Retrieved from http://www.demneuropsy.com.br/detalhe_artigo.asp?id=150
  » http://www.demneuropsy.com.br/detalhe_artigo.asp?id=150
• Malloy-Diniz, L. F., Bentes, R. C., Figueiredo, P. M., Brandão-Bretas, D., Costa-Abrantes, Parizzi, A. M., Borges-Leite, W., & Salgado, J. V. (2007). Normalización de una batería de tests para evaluar las habilidades de comprensión del lenguaje, fluidez verbal y denominación en niños brasileños de 7 a 10 años: Resultados preliminares. Revista de Neurología, 44(5), 275-280. Retrieved from http://www.neurologia.com/pdf/Web/4405/x050275.pdf
  » http://www.neurologia.com/pdf/Web/4405/x050275.pdf
• Mayr, U. (2002). On the dissociation between clustering and switching in verbal fluency: comment on Troyer, Moscovitch, Winocour, Alexander and Stuss. Neuropsychologia, 40, 562-566. doi: 10.1016/S0028-3932(01)00132-4
  » https://doi.org/10.1016/S0028-3932(01)00132-4
• Miller, E. (1984). Verbal fluency as a function of a measure of verbal intelligence and in relation to different types of cerebral pathology. British Journal of Clinical Psychology, 23, 53-57. Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/j.2044-8260.1984.tb00626.x/pdf
  » http://onlinelibrary.wiley.com/doi/10.1111/j.2044-8260.1984.tb00626.x/pdf
• Moraes, A., Guimaraes, L. S. P., Joanette, Y., Fonseca, R. P., Parente, M. A. M. P., & Almeida, R. M. M. (2014). Effect of aging, education, reading and writing, semantic processing and depression symptoms on verbal fluency. Psicologia: Reflexão e Crítica, 26, 680-690. Retrieved from http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0102-79722013000400008&lng=pt&nrm=iso&tlng=en
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0102-79722013000400008&lng=pt&nrm=iso&tlng=en
• Nieto, A., Galtier, I., Barroso, J., & Espinosa, G. (2008). Fluencia verbal en niños espeñoles en edad escolar: Estúdio normativo piloto y análisis de las estratégias organizativas. Revista de Neurología, 46(1), 2-6. Retrieved from http://www.revneurol.com/sec/resumen.php?or=pubmed&id=2007390
  » http://www.revneurol.com/sec/resumen.php?or=pubmed&id=2007390
• Oberg, G., & Ramírez, M. (2006). Cross-linguistic meta-analysis of phonological fluency: Normal performance across cultures. International Journal of Psychology, 41(5), 342-347. doi: 10.1080/00207590500345872
  » https://doi.org/10.1080/00207590500345872
• Okruszek, L., Rutkvowska, A., & Wilinska, J. (2013). Clustering and switching strategies during the semantic and fluency task in men with frontal lobe lesions and in men with schizophrenia. Psychology of Language and Communication, 17(1), 94-100. doi: 10.2478/plc-2013-0006
  » https://doi.org/10.2478/plc-2013-0006
• Oliveira, H. R. G (2007). Geração de texto com base em rítmo (Dissertação de mestrado). Retrieved from http://hdl.handle.net/10316/15404
  » http://hdl.handle.net/10316/15404
• Paula, J. J., Schlottfeldt, C. G., Moreira, L., Cotta, M., Bicalho, M. A., Romano-Silva, M. A., & Malloy-Diniz, L. F. (2010). Propriedades psicométricas de um protocolo neuropsicológico breve para uso em populações geriátricas. Revista de Psiquiatria Clínica, 37(6), 251-255. Retrieved from http://www.scielo.br/pdf/rpc/v37n6/a02v37n6.pdf
  » http://www.scielo.br/pdf/rpc/v37n6/a02v37n6.pdf
• Raboutet, C., Sauzéon, H., Corsini, M., Rodrigues, J., Langevin, S., & N'Koua, B. (2010). Performance on a semantic verbal fluency task across time: Dissociation between clustering, switching, and categorical exploration processes. Journal of Clinical Experimental Neuropsychology, 32(3), 268-280. doi: 10.1080/13803390902984464
  » https://doi.org/10.1080/13803390902984464
• Rapp, B., & Goldrick, M. (2006). Speaking words: Contributions of cognitive neuropsychological research. Cognitive Neuropsychology, 23(1), 39-73. doi: 10.1080/02643290542000049
  » https://doi.org/10.1080/02643290542000049
• Rich, J. B., Troyer, A. K., Bylsma, F. W., & Brandt, J. (1999). Longitudinal analysis of phonemic clustering and switching during word list generation in Huntington's disease. Neuropsychology, 13, 525-531. Retrieved from http://dx.doi.org/10.1037/0894-4105.13.4.525
  » http://dx.doi.org/10.1037/0894-4105.13.4.525
• Ross, T. P., Calhoun, E., Cox, T., Wenner, C., Kono, W., Pleasant, M. (2007). The reliability and validity of qualitative scores for the Controlled Oral Word Association Test. Archives of Clinical Neuropsychology, 22, 475-488. doi:10.1016/j.acn.2007.01.026
  » https://doi.org/10.1016/j.acn.2007.01.026
• Salles, J. F., Fonseca, R. P., Miranda, M. C., Berlin, C., Rodrigues, C., & Barbosa, T. (in press). Instrumento de Avaliação Neuropsicológica Breve Infantil - NEUPSILIN-Inf. (Manual). São Paulo: Vetor Editora.
• Salles, J. F., Fonseca, R. P., Rodrigues, C. C., Mello, C. B., Barbosa, T., & Miranda, M. C. (2011). Desenvolvimento do Instrumento de Avaliação Neuropsicológica Breve Infantil NEUPSILIN-INF.
• Salles, J. F., Sbicigo, J. B., Machado, W. L., Miranda, M. C., & Fonseca, R. P. (2014). Análise fatorial confirmatória do Instrumento de Avaliação Neuropsicológica Breve Infantil - NEUPSILIN - Inf. Psico-USF, 19(1), 119-130. Retrieved from http://dx.doi.org/10.1590/S1413-82712011000300006
  » http://dx.doi.org/10.1590/S1413-82712011000300006
• Sauzéon, H., Lestage, P., Raboutet, C., Kaoua, B. N., & Claverie, B. (2004). Verbal fluency output in children aged 7-16 as a function of the production criterion: qualitative analysis of clustering, switching processes and semantic network exploitation. Brain and Language, 89, 192-202. doi:10.1016/S0093-934X(03)00367-5
  » https://doi.org/10.1016/S0093-934X(03)00367-5
• Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420-428. Retrieved from http://www.aliquote.org/cours/2012_biomed/biblio/Shrout1979.pdf
  » http://www.aliquote.org/cours/2012_biomed/biblio/Shrout1979.pdf
• Silva, T. B. L., Yassuda, M. S., Guimarães, V. V., & Florindo, A. A. (2011). Fluência Verbal e variáveis sociodemográficas no processo de envelhecimento: Um estudo epidemiológico. Psicologia: Reflexão e Crítica, 24(4), 739-746. doi: 10.1590/S0102-79722011000400014
  » https://doi.org/10.1590/S0102-79722011000400014
• Strauss, E., Sherman, E. M. S., & Spreen, O. (2006). A compendium of neuropsychological tests: Administration, norms and commentary. New York: Oxford University Press.
• Steyvers, M., & Tenenbaum, J. (2005). The large-scale structure of semantic networks: Statistical analyses and a model of semantic growth. Cognitive Science, 29(1), 41-78. Retrieved from http://web.mit.edu/cocosci/Papers/03nSteyvers.pdf
  » http://web.mit.edu/cocosci/Papers/03nSteyvers.pdf
• Stuss, D. T., Alexander, M. P., Hamer, L., Palumbo, C., Dempster, R., Binns, M., ... Levine, B. (1998). The effects of focal anterior and focal posterior brain lesions on verbal fluency. Journal of the International Neuropsychological Society, 4, 265-278. Retrieved from http://journals.cambridge.org/article_S1355617798002653
  » http://journals.cambridge.org/article_S1355617798002653
• Tallberg, I. M., Carlsson, S., & Liberman, M. (2011). Children's word fluency strategies. Scandinavian Journal of Psychology, 52, 35-42. doi: 10.1111/j.1467-9450.2010.00842.x
  » https://doi.org/10.1111/j.1467-9450.2010.00842.x
• Toazza, R., Salum, G. A., Flores, S. M., Jarros, R. B., Pine, D. S., Salles, J. F., & Manfro, G. G. (2014). Phonemic verbal fluency is associated with pediatric anxiety disorders: Evidence from a community study. Journal of Child and Adolescent Psychopharmacology, 24(3), 149-157. doi: 10.1089/cap.2013.0086
  » https://doi.org/10.1089/cap.2013.0086
• Tröster, A. I., Fields, J. A., Testa, J. A., Paul, R. H., Blanco, C. R., Hames, K. A., ... Salmon, D. P. (1998). Cortical and subcortical influences on clustering and switching in the performance of verbal fluency tasks. Neuropsychologia, 36(4), 295-304. doi:10.1016/S0028-3932(97)00153-X
  » https://doi.org/10.1016/S0028-3932(97)00153-X
• Troyer, A. K. (2000). Normative data for Clustering and Switching on Verbal Fluency Tasks. Journal of Clinical and Experimental Neuropsychology, 22(3), 370-378. doi: 10.1076/1380-3395(200006)22:3;1-V;FT370
  » https://doi.org/10.1076/1380-3395(200006)22:3;1-V;FT370
• Troyer, A. K., Moscovitch, M., & Winocur, G. (1997). Clustering and Switching as two components of Verbal Fluency: Evidence from younger and older healthy adults. Neuropsychology, 11(1), 138-136. doi: 10.1037/0894-4105.11.1.138
  » https://doi.org/10.1037/0894-4105.11.1.138
• Troyer, A. K., Moscovitch, M., Winocur, G., Alexander, & M. P., Stuss, D. (1998). Clustering and Switching on verbal fluency: the effects of focal frontal - and temporal - lobe lesions. Neuropsychologia, 36(6), 499-504. doi: 10.1016/S0028-3932(97)00152-8
  » https://doi.org/10.1016/S0028-3932(97)00152-8
• Troyer, A. K., Moscovitch, M., Winocur, G., Leach, L., & Freedman, M. (1998). Clustering and Switching on verbal fluency tests in Alzheimer's and Parkinson's disease. Journal of the International Neuropsychological Society, 4, 137-143. Retrieved from http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=70021&fulltextType=RA&fileId=S1355617798001374
  » http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=70021&fulltextType=RA&fileId=S1355617798001374
• ^207-212
  Vivas, L., & Naveira, L. (2010). Generación de agrupamientos semánticos en una tarea de fluidez verbal en pacientes víctimas de un accidente cerebro vascular y controles sin patología cerebral. Revista Chilena de Neuropsicología (En Línea), 5(3), . Retrieved from http://www.neurociencia.cl/dinamicos/articulos/670936-rcnp2010vol5n3-4.pdf
  » http://www.neurociencia.cl/dinamicos/articulos/670936-rcnp2010vol5n3-4.pdf
• Zhao, Q., Guo, Q, & Hong, Z. (2013). Clustering and switching during a semantic verbal fluency test contribute to a differential diagnosis of cognitive impairment. Neuroscience Bull, 29(1), 75-82. doi: 10.1007/s12264-013-1301-7
  » https://doi.org/10.1007/s12264-013-1301-7
• Zimmermann, N., Parente, M. A. M. P., Joanette, Y., & Fonseca, R. P. (2014). Unconstreined, phonemic and semantic verbal fluency: age and education effects, norms and discrepancies. Psicologia Reflexão e Crítica, 27(1), 1-9. doi: 10.1590/S0102-79722014000100007
  » https://doi.org/10.1590/S0102-79722014000100007
• Zortea, M., Menegola, B., Villavicencio, A, & Salles, J. F. (2014). Graph analysis of semantic word association between children, adults, and the elderly. Psicologia: Reflexão e Crítica (Impresso), 27(1), 1-10. doi: 10.1590/S0102-79722014000100011
  » https://doi.org/10.1590/S0102-79722014000100011

• Nota das autoras: Agradecimentos: Este trabalho é parte da Dissertação de Mestrado da primeira autora, o qual foi desenvolvido com Bolsa de Mestrado do Conselho Nacional em Pesquisa (CNPq). As autoras agradecem a colaboração das alunas de Iniciação Científica: Ana Claudia Araújo de Souza, Carolina Beckencamp, Daniele Pioli, Débora de Bitencourt Fél.

Publication Dates

• Publication in this collection
  Sep-Dec 2016

History

• Received
  28 July 2015
• Reviewed
  08 Oct 2015
• Accepted
  20 Oct 2015