SciELO - Scientific Electronic Library Online

 
vol.3 número1Normative data and construct validity of the Rey Auditory Verbal Learning Test in a Brazilian elderly populationColor Trails Test: a Brazilian normative sample índice de autoresíndice de assuntospesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados

Journal

Artigo

Indicadores

Links relacionados

Compartilhar


Psychology & Neuroscience

versão On-line ISSN 1983-3288

Psychol. Neurosci. (Online) vol.3 no.1 Rio de Janeiro jan./jun. 2010

http://dx.doi.org/10.3922/j.psns.2010.1.011 

CLINICAL AND EXPERIMENTAL NEUROPSYCHOLOGY

 

The Rey Auditory Verbal Learning Test: normative data for the Brazilian population and analysis of the influence of demographic variables

 

 

Sabrina S. Magalhães; Amer C. Hamdan

Universidade Federal do Paraná, Brazil

Correspondence regarding this article should be directed to

 

 


ABSTRACT

The Rey Auditory Verbal Learning Test (RAVLT) is an efficient neuropsychological instrument for testing episodic declarative memory. Performance on this test can be influenced by demographic and cultural variables. The objective of the present study was to analyze the influence of demographics variables, such as age, gender, and education, on RAVLT performance in young and elderly adults. A Portuguese version of the RAVLT was administered to 302 healthy participants. Subjects were from both genders, ranging in age from 17 to 85 years (mean 50.6 ± 15.9 years) and in education from 1 to 20 years (mean 11.3 ± 3.7 years). Participants were grouped by age: 17-34 years old, 34-49 years old, 50-64 years old, and 65-85 years old. Pearson's Correlation analysis showed a significant association between RAVLT performance and both age and education (p < .001), but not between RAVLT performance and gender. Two-way analysis of variance revealed significant effects of age on all RAVLT measurements and significant effects of years of education on all measurements, with the exception of recognition. The present data indicate that participant age and education significantly influence performance on the RAVLT.

Keywords: memory, neuropsychological tests, neuropsychology.


 

 

Introduction

The Rey Auditory Verbal Learning Test (RAVLT) is an efficient neuropsychological instrument for evaluating episodic declarative memory. It provides scores for assessing immediate memory, new verbal learning, susceptibility to interference (proactive and retroactive), retention of information after a period of time, and memory recognition (Malloy-Diniz, Lasmar, Gazinelli, Fuentes, & Salgado, 2007; Spreen & Strauss, 1998; Van der Elst, Van Boxtel, Van Breukelen, & Jolles, 2005).

The RAVLT is sensitive to neurological impairment and verbal memory deficits in a variety of patient groups (Mitrushina, Boone, Razani, & D'Elia, 2005; Spreen & Strauss, 1998), including those suffering from left temporal epilepsy (Loring et al., 2008; Spreen & Strauss, 1998), Alzheimer's disease (Hamdan & Bueno, 2005; Schoenberg et al., 2006; Spreen & Strauss, 1998; Tierney et al., 1994), schizophrenia (Torres, Flashman, O'Leary, & Andreasen, 2001), Parkinson's disease (Spreen & Strauss, 1998; Tierney et al., 1994), autism (Bowler, Limoges, & Mottron, 2009), depression (Spreen & Strauss, 1998), dementias (Hamdan & Bueno, 2005; Mitrushina et al., 2005; Schoenberg et. al., 2006; Spreen & Strauss, 1998), specific language impairment (Spreen & Strauss, 1998), closed-head injury (Spreen & Strauss, 1998), alcoholic amnesia (Mitrushina et al., 2005; Spreen & Strauss, 1998), traumatic and vascular amnesia (Mitrushina et al., 2005), and impairments related to normal aging (Hamdan & Bueno, 2005; Tierney et al., 1994). This sensitivity of the RAVLT contributes to its clinical utility and the ability to make clinical distinctions between these disorders. Scores of verbal learning and verbal episodic declarative memory on the RAVLT also correlate strongly with executive function (Duff, Schoenberg, Scott, & Adams, 2005; Fuentes, Malloy-Diniz, Camargo, & Cosenza, 2008; Hamdan & Bueno, 2005).

RAVLT performance can be affected by demographic variables, including age, gender, education, and intelligence level or IQ, and cultural influences resulting from translations and adaptations to local languages and cultures (Malloy-Diniz et al., 2007; Mitrushina et al., 2005; Uchiyama et al., 1995; Van der Elst et al., 2005). Although no consensus has been reached about the magnitude of the influence of these factors, the influence of age has been suggested to be more pronounced than the influences of gender and education (Mitrushina et al., 2005; Spreen & Strauss, 1998; Van der Elst et al., 2005). Van der Elst et al. (2005) found that age was the strongest predictor of performance on all RAVLT scores; they also found that females and highly educated participants scored significantly higher than males and their less educated counterparts on most measurements. Gender differences on the RAVLT have been reported in several studies (Malloy-Diniz et al., 2007; Mitrushina et al., 2005; Teruya, Ortiz, & Minett, 2009), but other studies did not demonstrate an effect of gender on any of the RAVLT measures (Mitrushina et al., 2005).

Mitrushina et al. (2005) performed a meta-analysis on the influence of demographic variables on RAVLT performance and concluded the following: (i) age had a consistent effect on RAVLT performance, (ii) unclear was the effect of education level, suggesting that this factor did not significantly contribute to the results, (iii) intelligence level (IQ) was associated with RAVLT performance, although discrepancies were found in the effect on this variable, and (iv) in contrast to the literature, the effect of gender was found to improve female performance more than male performance. However, the authors emphasized that the data could not be generalized because of potential problems with the sample composition. They concluded that more studies are needed to evaluate the effect of demographic variables on RAVLT performance.

Previous studies have suggested that age negatively correlates with RAVLT performance, whereas education level shows a positive correlation (Mitrushina et al., 2005; Spreen & Strauss, 1998). These studies suggest that the ability of memory storage and recovery decreases with age. Some studies have indicated that primarily the ability to recall newly obtained information appears to decline with age. However, higher education levels may favor the ability to store information (Malloy-Diniz et al., 2007; Spreen & Strauss, 1998; Van der Elst et al., 2005).

Few studies in Brazil (Malloy-Diniz, Cruz, Torres, & Cosenza, 2000; Malloy-Diniz et al., 2007; Teruya et al., 2009) have investigated the effect of demographic variables on RAVLT performance. Malloy-Diniz et al. (2000) were at the forefront on this topic when they developed the Brazilian Portuguese version of the RAVLT. The test was administered to 218 participants, ranging in age from 16 to 93 years. Significant differences were found with respect to age, but no gender effect was observed. In another study, Malloy-Diniz et al. (2007) evaluated a sample of 223 elderly participants, ranging in age from 60 to 89 years. The results were consistent with those described in the international literature, indicating that age, gender, and education are important factors when assessing participant performance on the RAVLT. Age and education negatively and positively correlate, respectively, with general performance on the test. These authors also found better performance in women on some measures: A1, A3, A5, and the Sum (i.e., sum of recalled words for A1 through A5). These authors indicated that their data have limitations with regard to generalization because the sample consisted of subjects with high education levels, which may explain the differences. The Malloy-Diniz et al. (2007) pilot study used only elderly subjects, and other age groups were not evaluated.

Teruya et al. (2009) also published data on the performance of Brazilian subjects on the RAVLT. The sample from this study was more expansive than the Malloy-Diniz et al. (2007) study and included participants ranging in age from 34 to 85 years. They found a positive correlation between education level and RAVLT performance and a negative correlation with age. No significant effects of gender were found, with the exception of the delay recall trial in which women outperformed men. They applied a version of the RAVLT that was distinct from Malloy-Diniz et al. (2007), which makes comparisons between the studies difficult, including generalizing the data and compiling normative data for the test (Mitrushina et al., 2005; Spreen & Strauss, 1998; Van der Elst et al., 2005). Therefore, uniform normative data do not exist for RAVLT performance in the Brazilian population across the entire age spectrum, precluding an accurate analysis of the influence of demographic and cultural variables.

The objective of the present study was to analyze the effect of age, education, and gender as demographic variables on RAVLT performance and collect normative data on performance in a healthy Brazilian population.

 

Methods

Participants

The participants consisted of 302 healthy individuals, ranging in age from 17 to 85 years. Participation in the study was voluntary, and all subjects were informed of the confidentiality of the data. Among the 302 participants, 113 were male (62.6%) and 189 were female (37.4%). Ages ranged from 17 to 81 years (mean = 50.6 years; standard deviation [SD] = 15.9 years.

The education variable was computed using the number of years the subject studied according to the Brazilian school system. The average education level was 11.3 years (SD = 3.7; range = 1 to 20 years), demonstrating that they had at least completed high school. The median education level was 11 years, indicating that half of the participants had received formal schooling for less than 11 years and the other half received formal schooling for more than 11 years.

The sample was distributed by age into the following groups: 17-34 years old, 35-49 years old, 50-64 years old, 65-85 years old. According to their educational level, individuals were also classified into two groups: secondary school completion (> 10 years of education) and non-secondary school completion (< 10 years of education).

Procedure

The following instruments were used: (i) Volunteer Participation Consent and Confidentiality Terms (formal consent of the volunteer to participate and to guarantee to the subjects that participation in the study was voluntary and that all information provided by the subject to the research group was confidential), (ii) structured neuropsychological interview (to obtain general information about the participant's health and personal data), (iii) Mini Mental State Exam (MMSE; to screen for cognitive impairment), (iv) Geriatric Depression Scale (GDS; to evaluate the participant's affective state and differentiate those with and without depressive symptoms), and (v) RAVLT (to evaluate verbal episodic memory). A Portuguese version of the RAVLT was used (Malloy-Diniz et al., 2007). This version consists of orally presenting a list of 15 concrete nouns (List A) to the participants. The list was read aloud in the same order five consecutive times. After each reading, a free-recall test of the words presented was given. Participants were free to say the words as they remembered them. After the fifth reading, participants were presented with a distractor list (List B), with 15 different concrete nouns, followed by a free-recall test from List B. Immediately afterward, delayed recall of List A was tested without repeating the list to the participants. After a 20 minute interval, participants were asked again to recall the words from List A (delayed recall). Finally, the final task was orally presenting to the participants a list of 50 nouns, including those from Lists A and B, and 20 phonologically or semantically similar words to Lists A and B. The participants needed to identify and recognize the words that were part of List A. After all of these procedural steps of data collection, different indicators were extracted for analysis and interpretation, in which the score for each subtest corresponded to the number of correct words recalled by the participants. During the 20 minute interval, the following instruments were applied: the Trail Making Test (version A and B; to evaluate executive function) and the Auditory Consonant Trigrams test (to evaluate working memory). The data from these instruments were not included in the present statistical analysis.

The first data collection occurred with the presentation and reading of the Volunteer Participation Consent and Confidentiality Terms, which was followed by the Neuropsychological Interview, MMSE, and (for participants older than 60 years of age) the GDS. After these screening procedures, the RAVLT was applied. During the 20 minute interval, which was necessary to collect data for the A7 trial, participants were tested using other neuropsychological tests that did not directly interfere with the RAVLT.

The testing sessions lasted approximately 50 minutes, and all activities were performed during only one session, with the exception of the elderly group which received permission to break the process into two sessions if they felt fatigued. The aim was not to overload them and to respect their willingness to contribute to the research. In such cases, the first session lasted until the application of the GDS, and the next session consisted of the remainder of the protocol.

The present study used exclusion criteria to select only healthy individuals. The exclusion criteria included the following: (i) a history of psychiatric disturbances or presently being in a psychiatrically disturbed state, (ii) diabetes, (iii) heart problems or any other related pathology, (iv) use of psychoactive drugs or drugs with well-known side-effects that impact mnemonic function or use of a psychoactive drug within the past 12 months. These criteria were applied during the structured interview using self-reports from the participants and using the MMSE, which was used to map any possible pathological cognitive decline. For subjects over 60 years of age, the Portuguese version of the GDS was used. The inclusion criteria for the MMSE and GDS were the following: (i) a score above the threshold on the MMSE (i.e., > 24), (ii) a score < 5 on the GDS. The results showed that the lowest score on the MMSE was 24, and the highest score was 30 (mean = 28.9; SD = 1.5). The GDS indices ranged from 0 to 5, and the mean of the sample was 0.6 (SD = 1.2; Table 1).

Statistics

The independent variable was the age group (17-34 years old, 35-49 years old, 50-64 years old, and 65-85 years old). The dependent variable was RAVLT performance (A1, A2, A3, A4, A5, B1, A6, A7, Sum of A1 to A5, and Recognition). In the descriptive analysis of the data, medium central tendency (mean) and dispersion measurement were used. Pearson's Correlation test was used to assess associations between variables. For the inferential analysis, two-way analysis of variance (ANOVA) was used (for age and years of education), followed by the Tukey Honestly Significant Difference (HSD) post hoc test to determine significant differences between group means. The p value was established for each test, and the null hypothesis was rejected with α = 5%.

 

Results

The demographic characterization of the sample according to the inclusion criteria is shown in Table 1. The 17-34 year old group was composed of 55 participants, with 22 males and 33 females, a mean age of 24.3 years (SD = 5.3 years), a mean education level of 11.7 years (SD = 2.5), and a mean MMSE score of 29.4 (SD = 1.2). The GDS was not applied in this group. The 35-49 year old group was composed of 70 participants, with 22 males and 48 females, a mean age of 43.5 years (SD = 4.0 years), a mean education level of 11.7 years (SD = 3.3 years), and a mean MMSE score of 28.9 (SD = 1.4). The GDS was not applied in this group. The 50-64 year old group was composed of 120 participants (the largest number of participants among the groups), with 47 males and 73 females, a mean age of 57.2 years (SD = 3.8 years), a mean education level of 12.1 years (SD = 3.5 years; the highest education level among the groups, indicating that these participants had access to a college or university education), and a mean MMSE score of 28.9 (SD = 1.4). Participants older than 60 years of age in the 50-64 year old group were administered the GDS, with a mean score of 0.5 (SD = 1.0). The 65-81 year old group was composed of 57 participants, with 22 males and 35 females, a mean age of 70.7 years (SD = 5.6 years), a mean education level of 9.0 years (SD = 4.5 years), and a mean MMSE score of 28.0 (SD = 1.6). All participants in the 65-81 year old group were administered the GDS, with a mean score of 2.0 (SD = 1.4).

Table 2 presents the results of the Pearson Correlation test with regard to the demographic variables and RAVLT performance. The gender factor did not show any significant correlation with RAVLT performance. Among the other demographic variables, the age factor demonstrated a significant correlation with all RAVLT measures. The correlation was negative on all measures and strongest, in descending order, on trials A4 and A7 (delay recall), the Sum from A1 to A5, and A5. A significant positive correlation was found between RAVLT performance and education level (i.e., the higher the education level, the better RAVLT performance).

Two-way ANOVA revealed a significant effect of age on all RAVLT measures: A1 (F3,294 = 6.173, p < .001), A2 (F3,294 = 13.569, p < .001), A3 (F3,294 = 15.671, p < .001), A4 (F3,294 = 16.621, p < .001), A5 (F3,294 = 17.405, p < .001), B1 (F3,294 = 5.308, p < .01), A6 (F3,294 = 18.569, p < .001), A7 (F3,294 = 17.831, p < 0.001), Sum (F3,294 = 19.163, p < .001), and Recognition (F3,294 = 9.273, p = .01). The Tukey HSD test indicated the following: (i) 17-34 year olds were significantly different from 65-85 year olds on all measures and from 50-64 year olds on A3, A4, A6, A7, and Sum, (ii) 35-49 year olds only differed from 65-85 year olds, (iii) 50-64 year olds did not differ from 35-49 year olds but differed from 17-34 year olds on A3, A4, A6, A7, and Sum, and (iv) 65-85 year olds exhibited significantly different performance on all measures of the RAVLT compared with the other groups (p < .001).

Two-way ANOVA showed the following effects of education on RAVLT performance: A1 (F1,294 = 9.611, p < .01), A2 (F1,294 = 5.595, p < .01), A3 (F1,294 = 10.289, p < .01), A4 (F1,294 = 9.837, p < .01), A5 (F1,294 = 17.297, p < .001), B1 (F1,294 = 5.308, p < .001), A6 (F1,294 = 11.156, p < .001), A7 (F1,294 = 15.431, p < .001), Sum (F1,294 = 14.491, p < .001), and Recognition (F1,294 = 1.177, p = .278). No significant age × education interactions were found: A1 (F3,294 = 1.106, p = .346), A2 (F3,294 = 1.137, p = .334), A3 (F3,294 = 0.983, p = .400), A4 (F3,294 = 0.316, p = .813), A5 (F3,294 = 1.632, p = .181), B1 (F3,294 = 1.937, p = .123), A6 (F3,294 = 1.350, p = .258), A7 (F3,294 = 2.053, p = .106), Sum (F3,294 = 0.785, p = .502), and Recognition (F3,294 = .569, p = .635).

Tables 3 and 4 show the means and standard deviations of the groups classified by age group and education level and show the compilation of normative data from our healthy Brazilian sample.

 

Discussion

The RAVLT is a very efficient neuropsychological instrument for assessing episodic declarative memory. The test allows the evaluation of the components of acquisition and recall of information and permits the investigation of separate memory processes (Mitrushina et al., 2005). The aim of the present study was to analyze the effects of demographic variables (i.e., age, education, and gender) on RAVLT performance in healthy adults and the elderly.

Consistent with previous studies (Malloy-Diniz et al., 2007; Messinis, Tsakona, Malefaki, & Papathanasopoulos, 2007; Mitrushina et al., 2005; Spreen & Strauss, 1998; Van der Elst et al., 2005), the data indicate that age was the main factor that influenced RAVLT performance, favoring younger healthy participants. Education level also affected RAVLT performance, although to a lesser extent. A decline in performance was observed with increasing age and decreasing formal education. The performance of adults in the 35-49 year old group was less distinguishable, however, from the other groups. The performance of participants in the 50-64 and 65-81 year old groups showed a more distinguishable pattern of performance decline, with the latter group exhibiting worse performance than all of the other groups, and the former group exhibiting worse performance than only the 17-34 year old group. This finding suggesting a less distinguishable pattern of performance in younger adults is consistent with a previous study (Messinis et al., 2007).

No effects of gender on RAVLT performance were found. This result contrasts with Malloy-Diniz et al. (2007) for an elderly Brazilian population and the results of Messinis et al. (2007). Some potential limitations to the generalization of the present findings need to be highlighted. One concern of the present study was that it included only healthy participants. Although we used self-reports, we could not definitively confirm the absence of some conditions using objective measures. Moreover, the possibility of a ceiling effect needs to be considered (Uttl, 2005). Another issue refers to the composition of the sample. The majority of the participants lived in Curitiba, Parana State, and we may not be able to generalize the data to the entire Brazilian population because of possible demographic, cultural, social, and economic differences. For example, Ardila (2005) reported the influence of cultural values on neuropsychological testing.

The literature suggests that age negatively correlates with RAVLT performance, whereas education level is positively correlated (Mitrushina et al., 2005; Spreen & Strauss, 1998). These studies suggest that the ability of memory storage and recall decreases with age. Some research has indicated that primarily the ability to recall newly obtained information reduces with age. However, higher education levels favor the ability to store information (Malloy-Diniz et al., 2007; Spreen & Strauss, 1998; Van der Elst et al., 2005).

The present results indicate a significant influence of age and education level, but not gender, on RAVLT performance. Nevertheless, a larger sample size will lend power to the generalization of the data and will also permit more in-depth investigations into the influence of gender on RAVLT performance. Additionally, future studies can be developed to compare performance of specific pathological groups and control groups of healthy subjects.

 

Acknowledgments

The authors are grateful to Leandro Fernandes Malloy-Diniz (Universidade Federal de Minas Gerais) who kindly gave the Brazilian Portuguese version of the Rey Auditory Verbal Learning Test.

 

References

Ardila, A. (2005). Cultural values underlying psychometric cognitive testing. Neuropsychology Review, 15, 185-195.         [ Links ]

Bowler, D.M., Limoges, E., & Mottron, L. (2009). Different verbal learning strategies in autism spectrum disorder: evidence from the Rey Auditory Verbal Learning Test. Journal of Autism and Developmental Disorders, 39, 910-915.         [ Links ]

Duff, K., Schoenberg, M.R., Scott, J.G., & Adams, R.L. (2005). The relationship between executive functioning and verbal and visual learning and memory. Archives of Clinical Neuropsychology, 20, 111-122.         [ Links ]

Fuentes, D., Malloy-Diniz, L.F., Camargo, C.H.P., & Cosenza, R.M. (2008). Neuropsicologia: teoria e prática. Porto Alegre: Artmed.         [ Links ]

Hamdan, A.C., & Bueno, O.F.A. (2005). Relações entre controle executivo e memória episódica verbal no comprometimento cognitivo leve e na demência tipo Alzheimer. Estudos de Psicologia, 10, 63-71.         [ Links ]

Loring, D.W., Strauss, E., Hermann, B.P., Barr, W.B., Perrine, K., Trenerry, M.R., Chelune, G., Westerveld, M., Lee, G.P., Meador, K.J., & Bowden, S.C. (2008). Differential neuropsychological test sensitivity to left temporal lobe epilepsy. Journal of the International Neuropsychological Society, 14, 394-400.         [ Links ]

Malloy-Diniz, L.F.M., Cruz, M.F, Torres, V.M., & Cosenza, R.M. (2000). O teste de Aprendizagem Auditivo-Verbal de Rey: normas para uma população Brasileira. Revista Brasileira de Neurologia, 36, 79-83.         [ Links ]

Malloy-Diniz, L.F.M., Lasmar, V.A.P., Gazinelli, L.S.R., Fuentes, D., & Salgado, J.V. (2007). The Rey Auditory-verbal Learning Test: applicability for the Brazilian elderly population. Revista Brasileira de Psiquiatria, 29, 324-329.         [ Links ]

Messinis, L., Tsakona, I., Malefaki, S., & Papathanasopoulos, P. (2007). Normative data and discriminant validity of Rey's Verbal Learning Test for the Greek adult population. Archives of Clinical Neuropsychology, 22, 739-752.         [ Links ]

Mitrushina, M.N., Boone, K.B., Razani, J., & D'Elia, L.F. (2005). Handbook of normative data for neuropsychological assessment. New York: Oxford University Press.         [ Links ]

Schoenberg, M.R., Dawson, K.A., Duff, K., Patton, D., Scott, J.G., & Adams, R.L. (2006). Test performance and classification statistics for the Rey Auditory Verbal Learning Test in selected clinical samples. Archives of Clinical Neuropsychology, 21, 693-703.         [ Links ]

Spreen, O., & Strauss, E. (1998). A compendium of neuropsychological tests: administration, norms, and commentary. New York: Oxford University Press.         [ Links ]

Teruya, L.C., Ortiz, K.Z., & Minett, T.S.C. (2009). Performance of normal adults on Rey Auditory Learning Test: a pilot study. Arquivos de Neuropsiquiatria, 67(2-A), 224-228.         [ Links ]

Tierney, M.C., Nores, A., Snow, W.G., Fisher, R.H., Zorzitto, M.L., & Reid, D.W. (1994). Use of the Rey Auditory Verbal Learning Test in differentiating normal aging from Alzheimer's and Parkinson's dementia. Psychological Assessment, 6, 129-134.         [ Links ]

Torres, I.J., Flashman, L.A., O'Leary, D.S., & Andreasen, N.C. (2001). Effects of retroactive and proactive interference on word list recall in schizophrenia. Journal of the International Neuropsychological Society, 7, 481-490.         [ Links ]

Uchiyama, C.L., D'Elia, L.F., Dellinger, A.M., Becker, J.T., Selnes, O.A., Wesch, J.E., Chen, B.B., Satz, P., van Gorp, W., & Miller, E.N. (1995). Alternate forms of the Auditory-Verbal Learning Test: issues of test comparability, longitudinal reliability, and moderating demographic variables. Archives of Clinical Neuropsychology, 10, 133-145.         [ Links ]

Uttl, B. (2005). Measurement of individual differences: lessons from memory assessment in research and clinical practice. Psychological Science, 16, 460-467.         [ Links ]

Van der Elst, W., Van Boxtel, M.P.J., Van Breukelen, G.J.P., & Jolles, J. (2005). Rey's verbal learning test: normative data for 1855 healthy participants aged 24-81 years and the influence of age, sex, education, and mode of presentation. Journal of the International Neuropsychological Society, 11, 290-302.         [ Links ]

 

 

Correspondence regarding this article should be directed to:
Amer Cavalheiro Hamdan
Universidade Federal do Paraná, Departamento de Psicologia
Praça Santos Andrade, 50
Curitiba, PR, 80020-240, Brazil
E-mail: achamdan@ufpr.br

Received 30 October 2009; received in revised form 19 April 2010; accepted 10 May 2010.
Available on line 25 June 2010

 

 

Sabrina S. Magalhães and Amer C. Hamdan, Departamento de Psicologia da Universidade Federal do Paraná (UFPR), Brazil.

Creative Commons License Todo o conteúdo deste periódico, exceto onde está identificado, está licenciado sob uma Licença Creative Commons