Acessibilidade / Reportar erro

PROMOTION OF LEARNING STRATEGIES FOR PSYCHOLOGY STUDENTS

ABSTRACT

Learning strategies are self-regulating processes of knowledge acquisition and use. They must be constantly stimulated so that students can be fully engaged in academic life. The aim of this study was to evaluate the effectiveness of a learning strategy program for university students. A quasi-experimental methodology was used with 20 Psychology students, distributed in experimental and control groups. There was a pretest, a first posttest, and a second posttest four months after the intervention. The program consisted of eight meetings, focusing on metacognition, internal and contextual resource management, and social self-regulation. The first posttest indicated a significant increase in the mean of the experimental group, which remained the same in the second posttest. The main increase was observed in the self-regulation of resources. The control group did not significantly change its total score over the posttests and still showed a significant decrease in social self-regulation. The results indicate the effectiveness of the intervention and reinforce the relevance of such programs.

Keywords:
learning; self-regulation; higher education

RESUMO

As estratégias de aprendizagem são processos autorregulatórios de aquisição e uso do conhecimento que devem ser constantemente estimuladas para um amplo aproveitamento da vida acadêmica. O objetivo do estudo foi avaliar a eficácia de um programa de promoção de estratégias de aprendizagem em universitários. Utilizou-se metodologia quasi-experimental com 20 alunos de Psicologia, distribuídos em grupo experimental e controle, sendo realizados pré-teste, pós-teste e um ndican pós-teste quatro meses após a intervenção. O programa consistiu de oito encontros, focalizando-se a metacognição, administração de recursos internos e contextuais e autorregulação social. O primeiro pós-teste indicou aumento significativo da ndic do grupo experimental, que se manteve no ndican pós-teste, sendo o principal aumento observado na autorregulação de recursos. O grupo controle não modificou significativamente seu escore total ao longo dos pós-testes e ainda apresentou diminuição ndicant tive na autorregulação social. Os resultados indicam eficácia da intervenção e reforçam a relevância de programas dessa natureza.

Palavras-chave:
aprendizagem; autorregulação; ndica superior

RESUMEN

Las estrategias de aprendizaje son procesos autorregulatorios de adquisición y uso del conocimiento que deben ser constantemente estimuladas para un ndica aprovechamiento de la vida académica. El objetivo del ndica fue evaluar la eficacia de un programa de promoción de estrategias de aprendizaje en universitarios. Se ndican metodología cuasi-experimental con 20 alumnos de Psicología, distribuidos en grupo experimental y control, siendo realizados pre-test, post-test y un ndican post-test cuatro meses tras la intervención. El programa consistió de ocho ndicant t, focalizándose la metacognición, administración de recursos internos y contextuales y autorregulación social. El primer post-test indicó aumento significativo del promedio del grupo experimental, que se mantuvo en el ndican post-test, siendo el principal aumento observado en la autorregulación de recursos. El grupo control no modificó significativamente ndicant total a lo largo de los post-testes y aún ndican disminución ndicant tive en la autorregulación social. Los resultados ndicant eficacia de la intervención y refuerzan la relevancia de programas de esa naturaleza.

Palabras clave:
aprendizaje; autorregulación; enseñanza universitaria

INTRODUCTION

In Brazil, the number of students who get access to college education has been increasing. In 2015, over 8 million registrations in 2.364 Institutions of Higher Learning (IHL) were made, according to data by the “Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira” (INEP), or National Institute of Studies and Education Research. However, it is possible to observe at this level a considerable incidence of evasion and a growing number of students changing courses (Anisio Teixeira National Institute for Educational Studies and Research INEP 2018Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira [INEP] (2018). Resumo técnico do censo da educação superior 2015. Brasília: INEP, Ministério da Educação. Recuperado de http://portal.inep.gov.br/censo-da-educacao-superior.
http://portal.inep.gov.br/censo-da-educa...
).

An important process for a student’s ability to handle the difficulties of university life is the self-regulation of learning. According to Zimmerman and Schunk (2011Zimmerman, B. J.; Schunk, D. (2011). Handbook of self-regulation of learning and performance. New York: Taylor & Francis.), such capacity refers to a student’s self-managing skills by means of proactive activity that monitors cognition, motivation, behavior and environment, while enhancing performance and learning results. Thus, students acquire academic skills such as establishing goals, selecting and applying strategies, and monitoring their own efficacy.

As self-regulated learning demands that students take responsibility over their own education process, it becomes a desired type of learning. In addition, there is evidence that self-regulation is conducive to good learning and academic performance. The use of self-regulating strategies is essential for the proper accomplishment of tasks (Broadbent & Poon, 2015Broadbent, J.; Poon, W. L. (2015). Self-regulated learning strategies and academic achievement in online higher education learning environments: a systematic review. Internet & Higher Education, 27, 1-13. http://dx.doi.org/10.1016/j.iheduc.2015.04.007.; Kizilcec, Pérez-Sanagustín, & Maldonado, 2017Kizilcec, R. F.; Pérez-Sanagustín, M.; Maldonado, J. J. (2017). Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses. Computers & Education, 104, 18-33. https://doi.org/10.1016/j.compedu.2016.10.001
https://doi.org/10.1016/j.compedu.2016.1...
; Pascual & Rosillo, 2015Pascual, L. R.; Rosillo, V. M. (2015). Efectividad del coaching grupal sobre el desarrollo de la autorregulación del aprendizaje en estudiantes de ingeniería. Cuadernos de Investigación Educativa, 6(1), 71-88. https://revistas.ort.edu.uy/cuadernos-de-investigacion-educativa/article/view/8/6
https://revistas.ort.edu.uy/cuadernos-de...
).

Learning strategies are actually a set of devices that lead to self-regulation. These strategies can be defined as procedures of conscious and intentional nature that are chosen in order to promote the acquisition, maintenance and utilization of knowledge or information in an effective way in different contexts (Gargallo, Suárez-Rodríguez, & Pérez-Pérez, 2009Gargallo, B.; Suárez-Rodríguez, J.; Pérez-Pérez, C. (2009). The CEVEPEAU questionnaire. An instrument to assess the learning strategies of university students. Revista Electrónica de Investigación y Evaluación Educativa,15(2), 1-26. http://www.uv.es/RELIEVE/v15n2/RELIEVEv15n2_5eng.htm.
http://www.uv.es/RELIEVE/v15n2/RELIEVEv1...
; Santos & Boruchovitch, 2009Santos, O. J. X.; Boruchovitch, E. (2009). Estratégias de aprendizagem na formação dos professores: Uma análise da produção científica. Educação, 32(3), 346-354. http://revistaseletronicas.pucrs.br/ojs/index.php/faced/article/view/5349/4206.
http://revistaseletronicas.pucrs.br/ojs/...
).

Boruchovitch and Santos (2015Boruchovitch, E.; Santos, A. A. A. (2015). Psychometric studies of the learning strategies scale for university students. Paidéia, 25(60), 19-27. http://dx.doi.org/10.1590/1982-43272560201504
http://dx.doi.org/10.1590/1982-432725602...
) explain that, although there is no consensus regarding classification, learning strategies can be divided into groups: cognitive strategies, meta-cognitive strategies, and resource-management strategies. Cognitive strategies involve behaviors that make information maintenance more efficient and employ skills such as memorization, and content production and organization. Metacognitive strategies are the ones by means of which students plan, monitor, and regulate their own thoughts while coordinating their own learning process. Resource-management strategies, on the other hand, are introduced in order to organize environments, remove negative thoughts, increase motivation, reduce anxiety, and so on.

Concerning the assessment of learning strategies, there are international (Pintrich & de Groot, 1990Pintrich, P. R.; de Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance.Journal of Educational Psychology, 82(1), 33-40. http://dx.doi.org/10.1037/0022-0663.82.1.33.
https://doi.org/10.1037/0022-0663.82.1.3...
; Zimmermann & Martinez-Pons, 1986Zimmerman, B. J.; Martinez-Pons, M. (1986). Development of a structured interview for assessing student use of self-regulated learning strategies. American Educational Research Journal, 23(4), 614-628. http://dx.doi.org/ 10.3102/00028312023004614.
https://doi.org/10.3102/0002831202300461...
) as well as national (Boruchovitch et al., 2006Boruchovitch, E.; Santos, A. A. A.; Costa, E. R.; Neves, E. R. C.; Cruvinel, M.; Primi, R.; Guimarães, S. E. R. (2006). A construção de uma escala de estratégias de aprendizagem para alunos do ensino fundamental. Psicologia: Teoria e Pesquisa, 22(3), 297-304. http://dx.doi.org/10.1590/S0102-37722006000300006.
http://dx.doi.org/10.1590/S0102-37722006...
; Minervino et al., 2005Minervino, C. A. S. M.; Silveira, N. J. D.; Figueiredo, A. A. F.; Oliveira, K. A.; Silva, E. R.; Rodrigues, S. G. (2005). Estudo de validação da escala de estratégias de estudo. Avaliação Psicológica, 4(2), 115-123. http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712005000200004&lng=pt&tlng=pt.
http://pepsic.bvsalud.org/scielo.php?scr...
) instruments of reference. For measurements in the university scenario, there is the “Escala de Estratégias de Aprendizagem para Estudantes Universitários” (EEA-U), or Scale of Learning Strategies for University Students (SLSUS), created by Boruchovitch and Santos (2015)Santos, O. J. X.; Boruchovitch, E. (2009). Estratégias de aprendizagem na formação dos professores: Uma análise da produção científica. Educação, 32(3), 346-354. http://revistaseletronicas.pucrs.br/ojs/index.php/faced/article/view/5349/4206.
http://revistaseletronicas.pucrs.br/ojs/...
, which presents good psychometric parameters that render it adequate for use in studies that assess the profile of strategies and investigate the efficacy of intervention.

Nückles, Rübner and Renkl (2009Nückles, M.; Rübner, S.; Renkl, A. (2009). Enhancing self-regulated learning by writing learning protocols. Learning and Instruction, 19, 259-271. https://doi.org/10.1016/j.learninstruc.2008.05.002.
https://doi.org/10.1016/j.learninstruc.2...
) emphasize that students have little knowledge on the appropriate use of learning strategies and on how to self-regulate their own learning process. In this sense, a study realized by Marini and Boruchovitch (2014Marini, J.; Boruchovitch, E. (2014). Self-regulated learning in students of pedagogy. Paidéia, 24(59), 323-330. http://dx.doi.org/10.1590/1982-43272459201406.
https://doi.org/10.1590/1982-43272459201...
) revealed a decline in the employment of strategies as the students’ permanence at the university increases. This fact goes against what should be considered normal because there is evidence of correlation between learning strategies and better academic performance (Gargallo et al., 2009Gargallo, B.; Suárez-Rodríguez, J.; Pérez-Pérez, C. (2009). The CEVEPEAU questionnaire. An instrument to assess the learning strategies of university students. Revista Electrónica de Investigación y Evaluación Educativa,15(2), 1-26. http://www.uv.es/RELIEVE/v15n2/RELIEVEv15n2_5eng.htm.
http://www.uv.es/RELIEVE/v15n2/RELIEVEv1...
; Gargallo, Campos, & Almerich, 2016; Soares, Guisande, Almeida, & Páramo, 2009Soares, A. P.; Guisande, M. A.; Almeida, L. S.; Páramo, M. (2009). Academic achievement in first-year Portuguese college students: The role of preparation and learning strategies. International Journal of Psychology, 44, 204-212. https://doi.org/10.1080/00207590701700545
https://doi.org/10.1080/0020759070170054...
).

Mega, Ronconi and De Beni (2014Mega, C.; Ronconi, L.; De Beni, R. (2014). What makes a good student? How emotions, self-regulated learning, and motivation contribute to academic achievement. Journal od Educational Psychology, 106(1), 121-131. http://dx.doi.org/10.1037/a0033546.
https://doi.org/10.1037/a0033546...
), in a research on the relation between emotions, motivation and self-regulated learning, found evidence that corroborates the initial discoveries of Pintrich and of Groot (1990Pintrich, P. R.; de Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance.Journal of Educational Psychology, 82(1), 33-40. http://dx.doi.org/10.1037/0022-0663.82.1.33.
https://doi.org/10.1037/0022-0663.82.1.3...
). Such study, which is a reference on the theme, points out that students’ mere knowledge on learning strategies is not enough. They must be encouraged to use them and regulate their own cognition and effort. That is the only way students would manage to become more competent in their studies and improve their school performance. In this perspective, it is possible to observe the relevance of the implementation of programs that teach such strategies to university students. According to Costa and Boruchovitch (2010Costa, E. R.; Boruchovitch, E. (2010). As estratégias de aprendizagem de alunos repetentes do ensino fundamental. Psicologia em Pesquisa, 4(1), 31-39. http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1982-12472010000100005&lng=pt&tlng=pt.
http://pepsic.bvsalud.org/scielo.php?scr...
), despite the existence of productions on the theme, more attention has been paid to the investigation of learning strategies in elementary and high school students.

Concerning the effects of learning strategies in university students, Montero and Arizmendiarrieta (2017Montero, C. R.; Arizmendiarrieta, B. S. (2017). The effectiveness of a learning strategies program for university students. Psicothema, 29(4), 527-532. doi: 10.7334/psicothema2016.171
https://doi.org/10.7334/psicothema2016.1...
) conducted an intervention study with a control group of 57 students (with no training) and an experimental group with the participation of 60 students. The participants were enrolled in the Education and Teacher Formation course at the University of Oviedo in Spain and underwent pre and post tests on the use of learning strategies by means of the Questionnaire of Learning Strategies and Motivation (Cuestionario de Estrategias de Aprendizaje y Motivación - 2ª Revisão - CEAM - R2). The intervention consisted of 13 practical sessions in the classroom, two hours each, as well as individual assignments estimated to last something between 6 and 13 hours. Results demonstrated a positive effect of the program, that is, a statistically significant difference concerning the use of learning strategies by the participants when there was a comparison between the pre and post intervention stages, and between the control and experimental groups. Even though such change has not occurred for all tested strategies, data suggest that students tend to benefit from programs of this nature.

In another program realized by Rosário et al. (2007Rosário, P.; Mourão, R.; Núñez, J. C.; González-Pienda, J.; Solano, P.; Valle, A. (2007). Eficacia de un programa instruccional para la mejora de procesos y estrategias de aprendizaje en la enseñanza superior. Psicothema, 19(3), 422-427. http://www.psicothema.com/pdf/3380.pdf
http://www.psicothema.com/pdf/3380.pdf...
), 66 students of the first year of the University of Oviedo got training on learning strategies by means of six one-hour sessions and underwent pre and post tests by means of the questionnaire on Self-regulation Strategies Knowledge, or “Conhecimentos das Estratégias de Autorregulação” (CEA). There was statistically significant improvement in declarative knowledge on learning strategies. Another study that stands out is the work of Gargallo and various collaborators. Ever since 2000Gargallo, B.; Remensal, A. (2000). Estratégias de aprendizaje. Un programa de intervención para ESO y EPA. Madrid: Ministério da Educación, Cultura y Deporte. (Gargallo, 2001; Gargallo et al., 2017), these researchers have been investigating the efficacy of intervention programs with a focus on learning strategies among university students. Their intervention model includes vicarious learning, the teaching of memory and attention strategies, and the approach of emotional learning aspects such as motivation and self-concept. The efficacy of the model has become evident with time and, in many cases, it has made its way into the curricula of Spanish schools (Gargallo & Remensal, 2000).

Gargallo et al. (2016Gargallo, B.; Campos, C.; Almerich, G. (2016). Learning to learn at university. The effects of an instrumental subject on learning strategies and academic achievement. Cultura y Educacion, 28(4), 771-810. https://doi.org/10.1080/11356405.2016.1230293.
https://doi.org/10.1080/11356405.2016.12...
) elaborated an optional discipline, entitled “Learning strategies and study techniques for university students”, in order to promote learning strategies in 47 students in the university of Valencia. The discipline offered 14 two-hour sessions. Participants underwent pre-tests and post-tests concerning the use of learning strategies by means of the Assessment Questionnaire on Learning Strategies for University Students, or Cuestionario de Evaluación de las Estrategias de Aprendizaje de los Estudiantes Universitarios - CEVEAPEU. Results revealed significant differences in learning strategies as well as in the academic performance of those who took the offered discipline, which is something that did not happen in the control group.

In Brazil, some research works have attempted to investigate the strategies mostly used by university students, which shows that the self-regulation of resources and cognitive strategies are favorites (Góes, Pavesi, & Alliprandini, 2013Góes, N. M.; Pavesi, M. A.; Alliprandini, P. M. Z. (2013). Estratégias de aprendizagem utilizadas por alunos do curso de Pedagogia de uma IES pública do estado do Paraná ofertado à distância. RENOTE Revista Novas Tecnologias na Educação, 11. https://doi.org/10.22456/1679-1916.44737; Mello, 2017Mello, C. G. (2017). Estratégias de aprendizagem em ações educacionais à distância: relação com características da clientela e reações ao curso. Dissertação de mestrado, Programa de pós-graduação de Psicologia da Universidade de São Paulo, São Paulo, SP.; Silva, 2012Silva, L. L. V. (2012). Estilos e estratégias de aprendizagem de estudantes universitários. Dissertação de mestrado, Programa de pós-graduação em Psicologia da Universidade de São Paulo, São Paulo, SP.). When it comes to interventions, the investigation by Polydoro, Pelissoni, Carmo, Emílio, Dantas and Rosário (2015Rosário, P.; Núñes, J. C.; Trigo, L.; Guimarães, C.; Fernandez, E.; Cerezo, R.; Fuentes, S.; Orellana, M.; Santibáñez, A.; Fulano, C.; Ferreira, A.; Figueiredo, M. (2015). Transcultural analysis of the effectiveness of a program to promote self-regulated learning in Mozambique, Chile, Portugal, and Spain. Higher Education Research & Development, 34(1), 173-187. https://doi.org/10.1080/07294360.2014.935932
https://doi.org/10.1080/07294360.2014.93...
) stands out. They detected the efficacy of an intervention program, in the discipline format, for 124 university students who answered an assessment questionnaire in the end of the course. The discipline approached time management, planning, memory, anxiety control, and other themes. As a result, students reported significant changes in their study routines and in the strategies they adopted in order learn.

The most important advantages of interventions in learning strategies are promotion, prevention, and treatment of educational difficulties. Scientific literature points at diverse successful experiences of interventions with learning strategies, no matter what theoretical approach is adopted. Rosário et al. (2015Rosário, P.; Núñes, J. C.; Trigo, L.; Guimarães, C.; Fernandez, E.; Cerezo, R.; Fuentes, S.; Orellana, M.; Santibáñez, A.; Fulano, C.; Ferreira, A.; Figueiredo, M. (2015). Transcultural analysis of the effectiveness of a program to promote self-regulated learning in Mozambique, Chile, Portugal, and Spain. Higher Education Research & Development, 34(1), 173-187. https://doi.org/10.1080/07294360.2014.935932
https://doi.org/10.1080/07294360.2014.93...
) emphasize that such intervention programs are efficacious for students with different cultural, linguistic, and educational origins.

The positive indicators in the mentioned studies allow us to consider how desirable it is that the university provide students with programs in order to promote the teaching of learning strategies. Therefore, this investigation intends to assess the effect of a program to promote learning strategies in a convenience sample made up of psychology students from a public university in the state of Minas Gerais. Intervention in such a context becomes useful because it embraces the country’s crescent population of university students with educational necessities to satisfy.

METHOD

Study design

In order to test the intervention effect, the methodology was quasi-experimental, by means of an experimental group (EG) and a non-equivalent control group (CG). Both groups took a pre-test, a post-test and a second post-test four months after the end of the intervention. The collected data were predominantly quantitative with qualitative records during the sessions. Sampling was done in an intentional, non-probabilistic, convenience-oriented way. In other words, all participants were psychology undergraduates and there was no randomization of the components of the groups. Most importantly, students demonstrated interest in participating in the intervention. This manner of selecting participants was necessary in order to make sure the group would remain in the promotion program from beginning to end.

Participants

The EG (N=10) was made up of seven women and three men and their average age was 20.50 years (DP=0,972). The CG (N=10) was made up of eight women and two men whose average age was 19.80 years (DP=1,22). All participants were students in the psychology course at a public university in the “Zona da Mata”, or Forest Zone in the state of Minas Gerais.

Instruments

Scale of Learning Strategies for University Students (SLSUS)

The SLSUS was created by Boruchovitch and Santos (2015Boruchovitch, E.; Santos, A. A. A. (2015). Psychometric studies of the learning strategies scale for university students. Paidéia, 25(60), 19-27. http://dx.doi.org/10.1590/1982-43272560201504
http://dx.doi.org/10.1590/1982-432725602...
) and consists of a self-report with 35 items for an assessment of the way university students study and learn. Responses are provided to the statements by means of a Likert scale of four points that range from Never = 1 and Always = 4. The authors used the Exploratory Factor Analysis applied on data from a broad university sample (N=1.490) and identified three learning strategies: 1) metacognitive and cognitive self-regulating strategies; 2) self-regulating strategies for internal and contextual resources and; 3) social self-regulating strategies. The precision statistics by Cronbach’s Alpha indicated a rather satisfactory rate for the total amount of the scale (α=0,87). Field Diary.

At every meeting, there were records of observations on the group’s development based on two observation axes: 1) the group manifestations demonstrated by means of reflections and comments on the trained contents and 2) reports on the use of strategies in academic routine during the implementation of the work.

Procedures

After approval of the project by the Research Ethics Committee of the university where the study was conducted (CAAE 67671817.5.0000.5147), the data-gathering phase was initiated. Extensive advertisement of the project was realized in order to reach all students in the psychology course. The affiliation of some volunteers was obtained and they signed a Term of Free Informed Consent (TFIC). Then, they answered the SLSUS. Procedures took 20 minutes in average. Participants were allotted into the EG and CG by means of criteria such as interest and convenience since there was no randomization of the sample. However, in order to keep an ethical policy in the provision of benefits for all research groups, a lecture on learning strategies was given to the CG. The lecture lasted 60 minutes.

The EG intervention took place in eight meetings, once a week, with a duration of approximately 90 minutes each. The sessions were conducted in a room at the institution at a time that was out of the regular schedules of the participating students. The promotion program was coordinated by scholarship holders who were members of the Tutorial Educational Program, or “Programa de Educação Tutorial” (PET), of the psychology course in the university where the research was conducted, and by a scholarship holder of scientific initiation. They were trained and supervised by the tutoring teacher of the PET and by a collaborating teacher. At every meeting, one of the scholarship holders recorded procedures in the field diary. In the end of the intervention, the first pre-test was realized with the SLSUS on the EG and on the CG. Four months later, a second post-test was applied on both groups in order to identify, by means of a follow-up assessment, the possibility of maintaining the acquired skills. The assessments took place collectively in the research room and in other parts of the institution.

Intervention Procedures

The meetings for the promotion of learning strategies were based on programs tested and published by international scientific literature. More specifically, the meetings were based on the program fully published by Gargallo and Remensal (2000Gargallo, B.; Remensal, A. (2000). Estratégias de aprendizaje. Un programa de intervención para ESO y EPA. Madrid: Ministério da Educación, Cultura y Deporte.) and updated in later publications (Gargallo et al., 2016; Gargallo, Esteban, Mateo, Peleato, & Rodríguez, 2015). The intention was to keep the intervention methodology in the said work while realizing few modifications such as the number of meetings, the adaptation to the themes for the psychology course, and the search for equivalences between the contents promoted in the meetings and the constructs assessed by the research scale. Each meeting was initiated with a lesson on the day’s theme, followed by a practical activity. In the conclusion, homework is assigned and checked out in the next meeting.

The first and the second meetings attempted to exercise metacognition by encouraging reflection by the students on their own learning process. There was a lecture and a discussion on learning strategies where the participants were requested to evaluate their motivation and self-concept, while relating these constructs with their academic goals. The third meeting approached study conditions, attention and concentration by means of the reading and the creation of stories in addition to an activity in order to stimulate attention to detail. In the fourth meeting, the coordinators of the group presented ways to set up a study schedule and applied relaxation techniques. In these two sessions, the focus was the self-regulation of internal and contextual resources. The fifth and the sixth meeting aimed to promote reading skills with the use of tactics such as summarizing, underlining and drawing conceptual maps. Psychology texts were used. The seventh meeting was dedicated to the training of memory resources by means of the promotion of awareness of significant memory construction. The researchers proposed exercises with lists of words, lists of images, and the recovery of details in narrated stories. Therefore, these sessions attempted at exercising cognitive and metacognitive capacities in learning. The eighth meeting dealt with social skills, more specifically, the resolution of conflicts and decision making, which were stimulated by means of conflict stories and the search for the means to solve such conflicts. Also, verbal and non-verbal communication skills were stimulated while promoting the social articulation requested for a series of academic tasks that do not involve group interaction.

DATA ANALYSIS

The descriptive and inferential analyses were realized by means of the Statistic Package of Social Sciences (SPSS) program by International Business Machines (IBM), version 21. Parametric statistics were chosen because the sample is small and displays little variability. The Wilcoxon test was applied in order to detect intragroup differences whereas the Mann-Whitney test was used for intergroup differences. Some precautions were taken concerning the level of significance. One of them was a calculation of the size of the effect (Cohen’s d), which determined the area of the difference between the group averages and the statistical power estimate, which considers the size of the sample, the size of the effect and the assumed level of significance (Hair, Black, Babin, Anderson, & Tatham, 2009Hair, J. F.; Black, W. C.; Babin, B. J.; Anderson, R.E.; Tatham, R.L. (2009). Análise multivariada de dados (6a ed.). Porto Alegre: Bookman.). The size of the effect and the statistical power were computed by the GPower software, version 3.1.9.2.

RESULTS AND DISCUSSION

Analysis within the groups

Data analysis was based on the performance of participants in each one of the three subscales and in the total score of the SLSUS at three test moments: pretest, first posttest, and second posttest. The way the items in each subscale were summed up followed the suggestion present in the work on factorial validity published by the authors of the instrument (Boruchovitch & Santos, 2015Boruchovitch, E.; Santos, A. A. A. (2015). Psychometric studies of the learning strategies scale for university students. Paidéia, 25(60), 19-27. http://dx.doi.org/10.1590/1982-43272560201504
http://dx.doi.org/10.1590/1982-432725602...
). Table 1 presents mean and standard deviation in these results for the EG (N=10) and the CG (N=10).

Table 1
Mean and standard deviation for EG and CG in the subscales and in the total score of the SLSUS

The Wilcoxon statistical test applied on the total score obtained by the EG on the SLSUS, in an analysis combined with the pre-test, found a significant increase in the first post-test (z=-1.836; p=0.036); d=0.43 and power of 35.2%; in the second post-test with p level close to statistical significance (z=-1.602; p=0.057); d=0.30 and power of 31.8%. Despite a slight fall of almost two points in the EG mean in the follow-up study, the comparison between the first and the second posttest did not find any significant difference (z=-0.153; p=0.453). Thus, even with the discovery of little effect and power, the significant result allows us to affirm that there was improvement in the performance of the EG after the intervention. The improvement was maintained in the end of the study.

Concerning the performance of the EG in the subscales of the SLSUS, the self-regulation of internal and contextual resources presented a significant rise when compared to the results of the pretest and with the first posttest (z=-1.781; p= 0.049) with d=0.65 and power of 82.2%; and with the second posttest (z=-1.794; p=0.043) with d=0.62 and power of 75.5%. The size of the effect demonstrated by Cohen’s d is considered medium and the power indicates a high probability in the acceptance of research hypothesis, that is, the efficacy of the intervention on these specific skills in the assessed individuals in this period.

The comparison between the first and the second posttest did not reveal any significant change (z=-0.316; p=0.434), which hints at the maintenance of the increase in such skills during the months of data gathering. The results lead us to the belief that the intervention presented relatively long-lasting effects, which corroborates the studies by Fernandes and Frison (2015Fernandes, V. R.; Frison, L. M. B. (2015). Estratégias de aprendizagem autorregulatória no ensino superior: escrita de um artigo científico. Psicologia da Educação, 41, 37-49. http://dx.doi.org/10.5935/2175-3520.20150013.
https://doi.org/10.5935/2175-3520.201500...
), and Lai and Hwang (2016Lai, C. L.; Hwang, G. J. (2016). A self-regulated flipped classroom approach to improving students’ learning performance in a mathematics course. Computers & Education, 100, 126-140. http://dx.doi.org/10.1016/j.compedu.2016.05.006.
https://doi.org/10.1016/j.compedu.2016.0...
), for example.

It is considered possible that the improvement obtained by the EG, concerning the total sore of the scale, is due to an improvement in the self-regulation of internal and contextual resources. The other subscales, despite the increase in means, did not present significant change in the p≤0,05 level. Therefore, the intervention seems to have adequately approached the mentioned strategies, which helped the participants deal with anxiety while attempting to accomplish academic tasks and better organize their workplace during the realization of the research. The cognitive and metacognitive self-regulation strategies as well as the social self-regulation strategies did not increase critically. The intervention dealt extensively with themes related to cognition and to metacognition. However, it might be necessary to increase the number of meetings in order to more deeply affect the skills of the spectrum. The difficulty to significantly increase the cognitive capacities is registered in literature (Melby-Lervag, Redick, & Hulme, 2016Melby-Lervag, M.; Redick, T. S.; Hulme, C. (2016). Working memory training does not improve performance on measures of intelligence or other measures of “far transfer”: evidence from a meta-analytic review. Perspectives on Psychological Science, 11(4), 512-534. doi: 10.1177/1745691616635612.
https://doi.org/10.1177/1745691616635612...
). On the other hand, social self-regulation was a theme only in the final session, which focused on aspects of the social skills that were slightly different from the ones assessed by the SLSUS. The adopted scale has four items that assess the social factor. They refer to strategies to ask the group for help. Thus, the increase in these characteristics by the EG might be due to a small disparity between what was produced in the training and what was measured by the instrument.

For the CG (N=10), the Wilcoxon analysis applied to the total score of the scale did not find any significant differences between the pretest, the first posttest (z=-1.424; p=0.091) and the second posttest (z=-1.020; p=0.163). Also, no significant difference was found between the first and the second posttest (z=-0.869; p=0.214). In other words, the group that did not undergo intervention did not significantly change their score on the SLSUS during the tests.

Observing the changes in the CG (N=10) in the subscales of the SLSUS, it is possible to observe a significant reduction in the score for social self-regulation resources. Thus, based on the pretest, the score significantly decreased in the first posttest (z=-1.980; p=0.035), with d=0.59 (medium effect) and power of 56.4%; in the second posttest (z=-1.761; p=0.056) with an almost significant p level; d=0.27 (small effect) and high power of 87.6%. There was no significant difference between the first and the second posttest (z=-0.689; p=0.282), despite the fall observed in the mean (see Table 1). The other subscales did not present significant alteration in the p≤0,05 level.

The CG did not evolve as much as the EG did. It actually lost points in the score of the SLSUS during the posttests. There is a discussion in the literature on the decline in the use of learning strategies among university students throughout the course. It seems to be due to a lack of motivation or to the fact that students have gotten used to the study routine of the university. Therefore, they do not put out much of an effort to reach academic results (Marini & Boruchovitch, 2014Marini, J.; Boruchovitch, E. (2014). Self-regulated learning in students of pedagogy. Paidéia, 24(59), 323-330. http://dx.doi.org/10.1590/1982-43272459201406.
https://doi.org/10.1590/1982-43272459201...
). Anyway, the results found here, due to the size of the sample of 20 participants, must be considered with caution.

Intergroup analysis

The analysis of inter-group differences, with the use of Mann Whitney’s statistics, shows that there was no significant difference in the pre-test between the CG (N=10) and the EG (N=10) with Mann Whitney’s U of 40.50 (z=-0.719; p=0.247), which demonstrates that the groups were at the same level concerning learning strategies before the intervention. In the first posttest after the intervention, the EG presented an average score higher than the one presented by the CG, according to Table 1. However, the p level found was merely marginal (z=-1.563; p=0.065) Mann Whitney’s U = 29.50; large-sized effect (d=0.77) and very high power 85.6%. The large effect found by Cohen’s d and the high statistical power indicate that the means in the groups are really different and that the research hypothesis must be accepted. Thus, the EG surpassed the CG in learning strategies in the first posttest. The same analysis applied to the second posttest did not reach the p≤0,05 level (z=-1.286; p=0.106), despite the fact that the EG (M=103.50; SD=12,43) presented average score higher than the one presented by the CG (M=94,20; SD=16,27) in almost ten points of the final assessment. The size of the effect for this analysis was medium (d=0.64) and the power of the test was acceptable with 78%. Therefore, it is possible to affirm that the means of the groups in the end of the study were significantly different and benefit the EG, that is, the advantage of the EG over the CG remained the same over time.

It is possible to conclude that the difference found between the groups was probably a consequence of the intervention. Results of efficacy by the intervention of learning strategies, with the use of a control group, has been reported by researchers in the area (Hattie, 2015Hattie, J. (2015). The applicability of Visible Learning to higher education.Scholarship of Teaching and Learning in Psychology, 1(1), 79-91. http://dx.doi.org/10.1037/stl0000021.
https://doi.org/10.1037/stl0000021...
). There was a greater number or participants though. Likewise, there are reports testifying that there were no significant differences for the control group. That has been attributed to the discontinuity and the scarce number of realized sessions (Gargallo & Remensal, 2000Gargallo, B.; Remensal, A. (2000). Estratégias de aprendizaje. Un programa de intervención para ESO y EPA. Madrid: Ministério da Educación, Cultura y Deporte.).

Concerning the subscales, the descriptive statistics showed that the EG obtained higher scores than the CG in the posttest subscales of the research. Nevertheless, the Mann Whitney’s analysis applied to the obtained scores in the three testing moments did not find any significant difference between the EG (N=10) and the CG (N=10), and the found p values were all greater than 0.05. Thus, in comparison with the control group, the gains obtained by the EG were on the total score of the SLSUS and not in the specific scores.

In a nutshell, the intra and inter statistic analysis showed the significant effect of the intervention during the data-gathering months. Such effect was demonstrated in the total score as well as in some subscales of the SLSUS. The result was also more significant, with greater power and size of effect in the intra-group level for specific scores, which shows an advancement for the EG and a regression for the CG. The differences in the total scores were stronger in the intergroup level, also in favor of the EG.

Field Diary

The psychological studies of intervention with small samples did not always reach the expected levels of significance but they can generate psychological significance (Dancey & Reidy, 2008Dancey, C. P.; Reidy, J. (2008). Estatística sem matemática para Psicologia. Usando SPSS para Windows. Porto Alegre: Artmed. Bookman.). For that reason, it is important to use qualitative data registers such as the field diary in order to complement the quantitative data. The field dairy register in this intervention study indicated that most participants in the group seemed curious, motivated and engaged. The participants asked questions, realized the proposed exercises in the research room and took notes during the interventions. It was also possible to observe, in the oral feedbacks of the students, that the program was a positive experience since some students reported that they had started using the strategies in order to accomplish tasks in their graduation courses.

FINAL CONSIDERATIONS

The present research attempted to promote learning strategies in psychology students. The results obtained showed that the investigated students benefitted from the intervention during the months in which the study was realized. The intervention sharpened their learning skills, in accordance with the measurements of the scale. They also had the opportunity to verbally report the benefits of the program in academic everyday life. On the other hand, the group that did not receive the intervention presented a significant decrease in self-regulation, which is a sign that skills can fade away with time when they are not continually practiced. Therefore, research and interventions in the area must be encouraged.

The study presents limitations. The most relevant one concerns the limited size of the sample, which was made up of volunteers and depended on the affiliation of a group of students. Results must be considered with caution. It is recommended that, for future studies of this nature, researchers amplify the sample along with the realization of more continuous intervention meetings. Besides, it is important to emphasize that the study was realized by graduation students. Although they were properly trained, they might not have had the necessary experience to stimulate a group made up of peers. It is necessary to engage teachers and professionals in the realization of this research. We suggest the insertion of interventions in learning strategies in the curricula of universities as well as the capacitation of teachers so that they can better stimulate the development of their students in the classrooms during the disciplines. We also point at the necessity for continuous, contextualized self-assessment on personal skills in order to better assess the impact of the interventions.

We believe that, by increasing the number of participants, making the sample more representative of the university population, and extending the duration of the program, it will be possible to reach more comprehensive conclusions on which learning strategies are more adequate for most of the students and which ones must be promoted by the teachers in the classroom. It is also fundamental to take into account that learning strategies are not skills that can be standardized or homogenized. It is necessary to respect the preferences of every student when it comes to studying and learning.

There is no doubt that the stimulation of learning strategies leads students to a better employment of the knowledge acquired in the graduation course, which maximizes academic performance and contributes to the insertion of these students into the job market. However, it is essential to overcome the excessively clinical and palliative nature of intervention programs and to make the promotion of strategies something intrinsic to the very curricula of institutions. We expect that the findings reported here can be useful to the fomentation of healthy discussions in other stages of school education

REFERÊNCIAS

  • Boruchovitch, E.; Santos, A. A. A. (2015). Psychometric studies of the learning strategies scale for university students. Paidéia, 25(60), 19-27. http://dx.doi.org/10.1590/1982-43272560201504
    » http://dx.doi.org/10.1590/1982-43272560201504
  • Boruchovitch, E.; Santos, A. A. A.; Costa, E. R.; Neves, E. R. C.; Cruvinel, M.; Primi, R.; Guimarães, S. E. R. (2006). A construção de uma escala de estratégias de aprendizagem para alunos do ensino fundamental. Psicologia: Teoria e Pesquisa, 22(3), 297-304. http://dx.doi.org/10.1590/S0102-37722006000300006
    » http://dx.doi.org/10.1590/S0102-37722006000300006
  • Broadbent, J.; Poon, W. L. (2015). Self-regulated learning strategies and academic achievement in online higher education learning environments: a systematic review. Internet & Higher Education, 27, 1-13. http://dx.doi.org/10.1016/j.iheduc.2015.04.007.
  • Costa, E. R.; Boruchovitch, E. (2010). As estratégias de aprendizagem de alunos repetentes do ensino fundamental. Psicologia em Pesquisa, 4(1), 31-39. http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1982-12472010000100005&lng=pt&tlng=pt
    » http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1982-12472010000100005&lng=pt&tlng=pt
  • Dancey, C. P.; Reidy, J. (2008). Estatística sem matemática para Psicologia. Usando SPSS para Windows Porto Alegre: Artmed. Bookman.
  • Fernandes, V. R.; Frison, L. M. B. (2015). Estratégias de aprendizagem autorregulatória no ensino superior: escrita de um artigo científico. Psicologia da Educação, 41, 37-49. http://dx.doi.org/10.5935/2175-3520.20150013.
    » https://doi.org/10.5935/2175-3520.20150013
  • Gargallo, B. (2001). Learning strategies. A training programme. European Journal of Psychology of Education, 16, 49-65. https://www.jstor.org/stable/23420414
    » https://www.jstor.org/stable/23420414
  • Gargallo, B.; Campos, C.; Almerich, G. (2016). Learning to learn at university. The effects of an instrumental subject on learning strategies and academic achievement. Cultura y Educacion, 28(4), 771-810. https://doi.org/10.1080/11356405.2016.1230293.
    » https://doi.org/10.1080/11356405.2016.1230293
  • Gargallo, B.; Esteban, P. R. G.; Mateo, P. M. S.; Peleato, I. V.; Rodríguez, M. A. J. (2015). Métodos centrados en el aprendizaje, estrategias y enfoques de aprendizaje en estudiantes universitarios. Revista de Educación, 370, 229-2540. doi: 10.4438/1988-592X-RE-2015-370-304.
    » https://doi.org/10.4438/1988-592X-RE-2015-370-304.
  • Gargallo, B.; Pérez, C. P.; Rodríguez, M. A. J.; Hervás, N. M.; Beut, J. A. G. (2017). Métodos centrados en el aprendizaje, implicación del alumno y percepción del contexto de aprendizaje em estudiantes universitarios. Educación XX1, 20, 161-187. http://revistas.uned.es/index.php/educacionXX1/article/view/19036
    » http://revistas.uned.es/index.php/educacionXX1/article/view/19036
  • Gargallo, B.; Remensal, A. (2000). Estratégias de aprendizaje. Un programa de intervención para ESO y EPA Madrid: Ministério da Educación, Cultura y Deporte.
  • Gargallo, B.; Suárez-Rodríguez, J.; Pérez-Pérez, C. (2009). The CEVEPEAU questionnaire. An instrument to assess the learning strategies of university students. Revista Electrónica de Investigación y Evaluación Educativa,15(2), 1-26. http://www.uv.es/RELIEVE/v15n2/RELIEVEv15n2_5eng.htm
    » http://www.uv.es/RELIEVE/v15n2/RELIEVEv15n2_5eng.htm
  • Góes, N. M.; Pavesi, M. A.; Alliprandini, P. M. Z. (2013). Estratégias de aprendizagem utilizadas por alunos do curso de Pedagogia de uma IES pública do estado do Paraná ofertado à distância. RENOTE Revista Novas Tecnologias na Educação, 11 https://doi.org/10.22456/1679-1916.44737
  • Hair, J. F.; Black, W. C.; Babin, B. J.; Anderson, R.E.; Tatham, R.L. (2009). Análise multivariada de dados (6a ed.). Porto Alegre: Bookman.
  • Hattie, J. (2015). The applicability of Visible Learning to higher education.Scholarship of Teaching and Learning in Psychology, 1(1), 79-91. http://dx.doi.org/10.1037/stl0000021.
    » https://doi.org/10.1037/stl0000021
  • Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira [INEP] (2018). Resumo técnico do censo da educação superior 2015 Brasília: INEP, Ministério da Educação. Recuperado de http://portal.inep.gov.br/censo-da-educacao-superior
    » http://portal.inep.gov.br/censo-da-educacao-superior
  • Kizilcec, R. F.; Pérez-Sanagustín, M.; Maldonado, J. J. (2017). Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses. Computers & Education, 104, 18-33. https://doi.org/10.1016/j.compedu.2016.10.001
    » https://doi.org/10.1016/j.compedu.2016.10.001
  • Lai, C. L.; Hwang, G. J. (2016). A self-regulated flipped classroom approach to improving students’ learning performance in a mathematics course. Computers & Education, 100, 126-140. http://dx.doi.org/10.1016/j.compedu.2016.05.006.
    » https://doi.org/10.1016/j.compedu.2016.05.006
  • Marini, J.; Boruchovitch, E. (2014). Self-regulated learning in students of pedagogy. Paidéia, 24(59), 323-330. http://dx.doi.org/10.1590/1982-43272459201406.
    » https://doi.org/10.1590/1982-43272459201406
  • Melby-Lervag, M.; Redick, T. S.; Hulme, C. (2016). Working memory training does not improve performance on measures of intelligence or other measures of “far transfer”: evidence from a meta-analytic review. Perspectives on Psychological Science, 11(4), 512-534. doi: 10.1177/1745691616635612.
    » https://doi.org/10.1177/1745691616635612
  • Mega, C.; Ronconi, L.; De Beni, R. (2014). What makes a good student? How emotions, self-regulated learning, and motivation contribute to academic achievement. Journal od Educational Psychology, 106(1), 121-131. http://dx.doi.org/10.1037/a0033546.
    » https://doi.org/10.1037/a0033546
  • Mello, C. G. (2017). Estratégias de aprendizagem em ações educacionais à distância: relação com características da clientela e reações ao curso Dissertação de mestrado, Programa de pós-graduação de Psicologia da Universidade de São Paulo, São Paulo, SP.
  • Minervino, C. A. S. M.; Silveira, N. J. D.; Figueiredo, A. A. F.; Oliveira, K. A.; Silva, E. R.; Rodrigues, S. G. (2005). Estudo de validação da escala de estratégias de estudo. Avaliação Psicológica, 4(2), 115-123. http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712005000200004&lng=pt&tlng=pt
    » http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712005000200004&lng=pt&tlng=pt
  • Montero, C. R.; Arizmendiarrieta, B. S. (2017). The effectiveness of a learning strategies program for university students. Psicothema, 29(4), 527-532. doi: 10.7334/psicothema2016.171
    » https://doi.org/10.7334/psicothema2016.171
  • Nückles, M.; Rübner, S.; Renkl, A. (2009). Enhancing self-regulated learning by writing learning protocols. Learning and Instruction, 19, 259-271. https://doi.org/10.1016/j.learninstruc.2008.05.002.
    » https://doi.org/10.1016/j.learninstruc.2008.05.002
  • Pascual, L. R.; Rosillo, V. M. (2015). Efectividad del coaching grupal sobre el desarrollo de la autorregulación del aprendizaje en estudiantes de ingeniería. Cuadernos de Investigación Educativa, 6(1), 71-88. https://revistas.ort.edu.uy/cuadernos-de-investigacion-educativa/article/view/8/6
    » https://revistas.ort.edu.uy/cuadernos-de-investigacion-educativa/article/view/8/6
  • Pintrich, P. R.; de Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance.Journal of Educational Psychology, 82(1), 33-40. http://dx.doi.org/10.1037/0022-0663.82.1.33.
    » https://doi.org/10.1037/0022-0663.82.1.33
  • Polydoro, S. A. J.; Pelissoni, A. M. S.; Carmo, M. C.; Emílio, E. R. V.; Dantas, M. A.; Rosário, P. (2015). Promoção da autorregulação da aprendizagem na universidade: percepção do impacto de uma disciplina eletiva. Revista de Educação PUC-Campinas, 20, 201-213. https://doi.org/10.24220/2318-0870v20n3a2877.
    » https://doi.org/10.24220/2318-0870v20n3a2877
  • Rosário, P.; Núñes, J. C.; Trigo, L.; Guimarães, C.; Fernandez, E.; Cerezo, R.; Fuentes, S.; Orellana, M.; Santibáñez, A.; Fulano, C.; Ferreira, A.; Figueiredo, M. (2015). Transcultural analysis of the effectiveness of a program to promote self-regulated learning in Mozambique, Chile, Portugal, and Spain. Higher Education Research & Development, 34(1), 173-187. https://doi.org/10.1080/07294360.2014.935932
    » https://doi.org/10.1080/07294360.2014.935932
  • Rosário, P.; Mourão, R.; Núñez, J. C.; González-Pienda, J.; Solano, P.; Valle, A. (2007). Eficacia de un programa instruccional para la mejora de procesos y estrategias de aprendizaje en la enseñanza superior. Psicothema, 19(3), 422-427. http://www.psicothema.com/pdf/3380.pdf
    » http://www.psicothema.com/pdf/3380.pdf
  • Santos, O. J. X.; Boruchovitch, E. (2009). Estratégias de aprendizagem na formação dos professores: Uma análise da produção científica. Educação, 32(3), 346-354. http://revistaseletronicas.pucrs.br/ojs/index.php/faced/article/view/5349/4206
    » http://revistaseletronicas.pucrs.br/ojs/index.php/faced/article/view/5349/4206
  • Silva, L. L. V. (2012). Estilos e estratégias de aprendizagem de estudantes universitários Dissertação de mestrado, Programa de pós-graduação em Psicologia da Universidade de São Paulo, São Paulo, SP.
  • Soares, A. P.; Guisande, M. A.; Almeida, L. S.; Páramo, M. (2009). Academic achievement in first-year Portuguese college students: The role of preparation and learning strategies. International Journal of Psychology, 44, 204-212. https://doi.org/10.1080/00207590701700545
    » https://doi.org/10.1080/00207590701700545
  • Zimmerman, B. J.; Martinez-Pons, M. (1986). Development of a structured interview for assessing student use of self-regulated learning strategies. American Educational Research Journal, 23(4), 614-628. http://dx.doi.org/ 10.3102/00028312023004614.
    » https://doi.org/10.3102/00028312023004614
  • Zimmerman, B. J.; Schunk, D. (2011). Handbook of self-regulation of learning and performance New York: Taylor & Francis.
  • The authors are grateful for the financial support provided by the Dean of Post-graduation and Research at the Federal University of Juiz de Fora (Propp - UFJF) and by the Tutorial Education Program of the Ministry of Education, or "Programa de Educação Tutorial do Ministério da Educação" (PET - MEC).
  • This paper was translated from Portuguese by Régis Lima

Publication Dates

  • Publication in this collection
    21 Aug 2020
  • Date of issue
    2020

History

  • Received
    23 Aug 2018
  • Accepted
    29 Oct 2019
Associação Brasileira de Psicologia Escolar e Educacional (ABRAPEE) Associação Brasileira de Psicologia Escolar e Educacional (ABRAPEE), Rua Mirassol, 46 - Vila Mariana , CEP 04044-010 São Paulo - SP - Brasil , Fone/Fax (11) 96900-6678 - São Paulo - SP - Brazil
E-mail: revista@abrapee.psc.br