Acessibilidade / Reportar erro

Satisfaction and self-confidence in the learning of nursing students: Randomized clinical trial

Satisfacción y autoconfianza en el aprendizaje de estudiantes de enfermeria: Ensayo clínico aleatorizado

ABSTRACT

Objective:

To identify and compare satisfaction and self-confidence in the learning of nursing students from the use of simulation and traditional teaching in adult immunization scenarios in the context of Primary Health Care

Methods:

A randomized controlled clinical trial. Thirty-four undergraduate nursing students from a Brazilian federal public university participated in the study. The students were allocated in two groups: dialogical exposition and training of skills (control) and dialogical exposition, training of skills and simulation (experimental). Student Satisfaction and Self-Confidence in Learning scale was applied after receiving the interventions. In the analysis of satisfaction and self-confidence in learning, the Mann-Whitney test was used for a level of significance of 5%.

Results:

Traditional teaching strategies and simulation promote satisfaction and self-confidence in students’ learning. The experimental group presented higher mean values in most of the variables of both subscales. However, there was no statistical significance in the learning satisfaction subscale (p-value ≥ 0.05) and self-confidence in learning (p-value ≥ 0.05) between the experimental and control groups.

Conclusion and implications for practice:

Because they generate satisfaction and self-confidence, simulation and traditional strategies can be mutually used in nursing training.

Keywords:
Simulation; Nursing education; Primary Health Care

RESUMEN

Objetivo:

Identificar y comparar la satisfacción y la confianza propia en el aprendizaje de estudiantes de enfermería a partir del uso de la simulación y de la enseñanza tradicional en escenarios de inmunización de adultos en el contexto de la Atención Primaria a la Salud.

Métodos:

Ensayo clínico controlado y aleatorizado. Participaron del estudio 34 estudiantes graduados en Enfermería de una universidad pública federal brasileña. Los estudiantes se asignaron a dos grupos: exposición dialógica y entrenamiento de habilidades (control) y exposición dialógica, entrenamiento de habilidades y simulación (experimental). Después de recibir las intervenciones, se aplicó la escala de Satisfacción y confianza propia de los estudiantes en el aprendizaje. En el análisis de la satisfacción y autoconfianza en el aprendizaje, se utilizó la prueba de Mann-Whitney, para un nivel de significancia del 5%.

Resultados:

Las estrategias tradicionales de enseñanza y la simulación promueven la satisfacción y la confianza propia en el aprendizaje de los estudiantes. El grupo experimental presentó valores medios superiores en la mayoría de las variables de ambas subescalas. Sin embargo, no se registró significancia estadística en la subescala de satisfacción en el aprendizaje (valor p ≥ 0,05) y la confianza propia en el aprendizaje (valor p ≥ 0,05) entre los grupos experimental y control.

Conclusión e implicaciones para la práctica:

Por generar satisfacción y confianza propia, la simulación y las estrategias tradicionales pueden utilizarse mutuamente en la enseñanza de Enfermería.

Palabras clave:
Simulación; Enseñanza de Enfermería; Atención Primaria a la Salud

RESUMO

Objetivo:

Identificar e comparar a satisfação e autoconfiança na aprendizagem de estudantes de enfermagem a partir do uso da simulação e do ensino tradicional, em cenários de imunização de adultos, no contexto da Atenção Primária à Saúde.

Método:

Ensaio clínico controlado e randomizado. Participaram do estudo 34 estudantes da graduação em enfermagem de uma universidade pública federal brasileira. Os estudantes foram alocados em dois grupos: exposição dialogada e treino de habilidades (controle), e exposição dialogada, treino de habilidades e simulação (experimental). Após receberem as intervenções, aplicou-se a escala Student Satisfaction and Self-Confidence in Learning. Na análise da satisfação e autoconfiança na aprendizagem, utilizou-se o teste de Mann - Whitney, considerando-se o nível de significância de 5%.

Resultados:

As estratégias tradicionais de ensino e a simulação promovem a satisfação e autoconfiança na aprendizagem dos estudantes. O grupo experimental apresentou médias superiores, na maioria das variáveis, de ambas as subescalas. Porém, não houve significância estatística na subescala de satisfação na aprendizagem (p valor ≥ 0,05) e a autoconfiança na aprendizagem (p valor ≥ 0,05) entre os grupos experimental e controle.

Conclusão e implicações para a prática:

Por serem geradoras de satisfação e autoconfiança, a simulação e as estratégias tradicionais podem ser utilizadas mutuamente na formação em enfermagem.

Keywords:
Simulação; Ensino de Enfermagem; Atenção Primária à Saúde

INTRODUCTION

Teaching-learning strategies are ways that will make it easier for the student to accomplish the technical-professional objectives on personal development and to become a transforming agent.11 Morin EA. Cabeça bem-feita. Repensar a reforma, reformar o pensamento. Rio de Janeiro: Bertrand Brasil; 2014. In the nursing training context, by thinking on these strategies, one must consider ethic issues, patient’s security, technological advances, the nursing science, the complexity of the care measurements, the current work world requirements, and the subjects’ emancipation.22 Costa RRO, Medeiros SM, Martins JCA, Coutinho VRD. A simulação no ensino de enfermagem: reflexões e justificativas à luz da bioética e dos direitos humanos. Acta Bioeth [Internet]. 2018 jun; [cited 2018 nov 10]; 24(1):31-8. Available from: http://dx.doi.org/10.4067/S1726-569X2018000100031
http://dx.doi.org/10.4067/S1726-569X2018...

Besides, during the teacher activities’ plan, the teacher must reflect on the objectives of learning and its contribution to facilitating knowledge. For this, it is necessary to shed some light on the role of the diverse actors involved in the teaching-learning process and to dispose of different ways of contemplating what was planned.

Nowadays, it is indispensable that the teachers act as researches and that they are capable of evaluating the strategies they choose to adopt. This requires them to be open to getting out of their comfort zone.33 Foronda C, Liu S, Bauman EB. Evaluation of simulation in undergraduate nurse education: An integrative review. Clin Simul Nurs [Internet]. 2013 oct; [cited 2018 nov 10]; 9(10):e409-e16. Available from: https://doi.org/10.1016/j.ecns.2012.11.003
https://doi.org/10.1016/j.ecns.2012.11.0...

In the nursing training context, three strategies of teaching and learning are commonly present: lecture and dialogical classes, skills practicing and the development of simulated clinical scenarios. From this perspective we add the need for a closer look at the identification of satisfaction with teaching-learning and the development in the self-confidence of the students who experiences these strategies throughout their training. That is, because the learner’s satisfaction and self-confidence are variables that allow us to identify and evaluate the effectiveness of the teaching and learning strategies used during the training.44 Almeida RGS, Mazzo A, Martins JCA, Baptista RCN, Girão FB, Mendes IAC. Validation to Portuguese of the Scale of Student Satisfaction and Self-Confidence in Learning. Rev Latino-Am Enfermagem [Internet]. 2015 dec; [cited 2018 nov 10]; 23(6):1007-13. Available from: http://dx.doi.org/10.1590/0104-1169.0472.2643
http://dx.doi.org/10.1590/0104-1169.0472...

Satisfaction is a wish of pleasure or disappointment, which stems from the outcome of an event and from the individual’s prior expectancies of themselves. It is an affective reaction, reached or not regarding a service. In the teaching-learning activities, it is usually guided by the learning experience and its relationship with the learners’ expectations,55 Kootler P. Administração de marketing: análise, planejamento, implementação e controle. 5ª ed. São Paulo: Atlas; 1998.-66 Kootler P, Clarke RN. Marketing for health care organizations. Englewood Cliffs, NJ: Prentice-Hall; 1987. and it can be manifested globally within the process as a whole, or for each stage of the lived experience.77 Lemos SIM. Análise da satisfação de estudantes num curso em e-learning no ensino superior [dissertação]. Lisboa: Universidade de Lisboa; 2011. In simulated teaching, satisfaction can be considered an important component, not only for the success of reaching the craved item but especially for the positive reinforcement in self-confidence and in the experiences that will build the profile of the future professional.

Satisfaction is contentment, a feeling of pleasure coming from what one hopes. In the context of simulation, it refers to contentment with learning through the scenarios.88 DeYoung S. Teaching strategies for nurse educators. Upper Saddle River: Prentice Hall Health; 2003. Moreover, it is associated with greater involvement and motivation in the teaching and learning process, as the motivated student learns more and better.99 Baptista RCN, Martins JCA, Pereira MFCR, Mazzo A. Students’ satisfaction with simulated clinical experiences: validation of an assessment scale. Rev Latino-Am Enfermagem [Internet]. 2014 oct; [cited 2018 nov 10]; 22(5):709-15. Available from: http://dx.doi.org/10.1590/0104-1169.3295.2471
http://dx.doi.org/10.1590/0104-1169.3295...
Self-confidence is trusting in the solidity of one’s own judgment and performance.1010 Jeffries PR. A framework for designing, implementing, and evaluating: Simulations used as teaching strategies in nursing. Nurs Educ Perspect [Internet]. 2005 mar/apr; [cited 2018 nov 10]; 26(2):96-103. Available from: https://www.ncbi.nlm.nih.gov/pubmed/15921126
https://www.ncbi.nlm.nih.gov/pubmed/1592...
Relying on performance can be an influential factor synonymous with a better transition from theoretical knowledge to the clinical practice.

For the National League for Nursing, simulation is defined as an attempt to imitate essential aspects of a clinical situation, aiming to understand and manage the best situation when it occurs in the actual clinical practice. It is about a strategy that uses a situation or an environment created to allow the individuals to experience a real event representation for the purpose of practice, learning, evaluation, testing or to get an understanding of human systems or actions.1111 National League for Nursing (NLN). Simulation Innovation Resource Center. SIRC Glossary; 2013; [cited 2018 nov 10]. Available from: http://sirc.nln.org/mod/glossary/view.php?id=183
http://sirc.nln.org/mod/glossary/view.ph...

For being such a valuable strategy, it has been gaining acceptancy from students, teachers and health accreditation bodies. However, research studies on simulation are still late regarding the great number of available technology.33 Foronda C, Liu S, Bauman EB. Evaluation of simulation in undergraduate nurse education: An integrative review. Clin Simul Nurs [Internet]. 2013 oct; [cited 2018 nov 10]; 9(10):e409-e16. Available from: https://doi.org/10.1016/j.ecns.2012.11.003
https://doi.org/10.1016/j.ecns.2012.11.0...
,1212 Costa RRO, Medeiros SM, Martins JCA, Cossi MS, Araújo MS. Percepção de estudantes da graduação em enfermagem sobre a simulação realística. Rev Cuid [Internet]. 2017 sep/dec; [cited 2018 nov 10]; 8(3):1799-1808. Available from: http://dx.doi.org/10.15649/cuidarte.v8i3.425
http://dx.doi.org/10.15649/cuidarte.v8i3...
The literature points out several contributions from the simulation in the nursing teaching context. Nevertheless, there is a gap regarding the results of perceptions on satisfaction and self-confidence.1313 Norman J. Systematic Review of the Literature on Simulation in Nursing Education. ABNF Journal [Internet]. 2012; [cited 2018 nov 10]; 23(2):24-28. Available from: https://www.ncbi.nlm.nih.gov/pubmed/22774355
https://www.ncbi.nlm.nih.gov/pubmed/2277...
,1414 Zapko KA, Ferranto MLG, Blasiman R, Shelestak D. Evaluating best educational practices, student satisfaction, and self-confidence in simulation: A descriptive study. Nurse Educ Today [Internet]. 2018 jan; [cited 2018 nov 10]; 60:28-34. Available from: https://www.ncbi.nlm.nih.gov/pubmed/28987895
https://www.ncbi.nlm.nih.gov/pubmed/2898...

This gap becomes even more prominent when looking for evidence of comparisons of these perceptions between different teaching and learning strategies. Recognizing this lack, the study aimed to identify and compare the satisfaction and self-confidence in the learning of nursing students from the use of simulation and traditional teaching in adult immunization scenarios, within the context of Primary Health Care.

METHOD

This is a controlled and randomized clinical trial, applied in random experimental and control groups.1515 Campbell DT, Stanley JC. Delineamentos experimentais e quase-experimentais de pesquisa. São Paulo: USP/EPU; 1979. After approval by the Research Ethics Committee with No. 1,958,827 and CAAE No. 64874817.3.0000.5537, and obtaining the Brazilian Clinical Trial Registry (RBR-9sqr6b and UTN: u1111-1195-2580), the study was conducted in a Brazilian federal public university, in the period from May to June 2017.

All the ethical issues were respected. Participation in the study was voluntary. Besides, anonymity, confidentiality, freedom of withdrawal at any time from the training and the character of complementary training were guaranteed, without interference in the evaluation of the nursing course curricular units with which they were linked.

The study included students regularly enrolled between the 5(th) and 9(th) semesters of the undergraduate nursing course who obtained a frequency equal to or greater than 75% in a 40-hour training course. The following were excluded: participants who partially answered the research instruments; scholarship students and collaborators who contributed to the execution of the study. The initial non-probabilistic convenience sample consisted of 58 students. After applying the inclusion criteria, the final sample consisted of 34 students, as detailed in Figure 1.

Figure 1
Follow-up flowchart. Natal, RN, Brazil, 2019.

The students were allocated in two groups, sample composition made by an independent statistician and not linked to the research project. After randomization, students participated in a 40-hour course on adult immunization. The workload was distributed as follows: 20 hours for teaching activities, 16 hours for evaluative activities, and 4 hours for conclusion and training feedback activities.

The control group participated in the training with traditionally used strategies: lecture and dialogical classes and simulated skills training. The experimental group was directed to the simulation group, using clinical scenarios: lecture classes, skills training, and simulated scenarios.

The lecture classes were built from the contents available in official documents of the Ministry of Health, regarding immunization. Three face-to-face meetings of 4 hours/class each were offered. The classes were taught by the researchers.

For skills training, checklist guides were provided. Four skill stations were set up at the university’s nursing laboratory where the study was conducted. A face-to-face meeting of 4 hours/class was offered. At the stations, we worked with the following skills: a) administration of immunobiological doses, b) routines and organization of the vaccine room, c) scheduling and d) vaccination portfolio analysis. The students were divided into four subgroups. After performing the proposed tasks in each station, they received feedback from the research team.

For the experimental group three clinical situation scenarios were created. At this stage, all simulation design guidelines for the teaching-learning objectives, fidelity, problem-solving, student support, and debriefing were followed.77 Lemos SIM. Análise da satisfação de estudantes num curso em e-learning no ensino superior [dissertação]. Lisboa: Universidade de Lisboa; 2011. The scenario construction process was based on a literature review and on the collaboration of experts in the clinical area. Three professors (nurses, Ph.D. in nursing with professional activity in Primary Care) were consulted.

Scenarios of high complexity and fidelity were built, and validated in appearance and content by a group of five experts. These specialists were professors at the Federal University of Rio Grande do Norte and the Coimbra College of Nursing, with a doctorate and experience in the area of simulation. There was 100% agreement among them.

In line with the scenarios, three learning objectives were established, namely: in scenario 1, the objective was for the student to manage care measurements - from the point of view of immunization - of a patient who suffered a traumatic accident; in scenario 2, the objective was for the student to perform the scheduling and administration of immunobiologicals as needed by the patient; and in scenario 3, the objective was for the student to recognize an immunobiological contraindication situation.

For the simulations, the students received support materials, previously discussed before each simulation aiming at scenarios training and development. This material consisted of a document containing the general information - such as the scenario resources - and the context of the simulated cases: basic descriptive elements before performing a scenario.

The scenarios were performed in a simulated practices laboratory, located at the Federal University of Rio Grande do Norte, Brazil. In each scenario, students were provided with a description of the semi-structured clinical case, containing information such as clinical examinations, health conditions, and the reason for seeking service, among others.

To ensure the quality of the information while developing the scenario, a script was written and used. In the simulations, we used the standard-patient tool, which are actors trained to act and reproduce user behaviors in several situations and health care establishments.1111 National League for Nursing (NLN). Simulation Innovation Resource Center. SIRC Glossary; 2013; [cited 2018 nov 10]. Available from: http://sirc.nln.org/mod/glossary/view.php?id=183
http://sirc.nln.org/mod/glossary/view.ph...
The actors were prepared in meetings, involving them together with the researchers, clarifying and defining the ways of acting and the speeches. After this procedure, the scenarios were subjected to appearance and content testing and to validation by experts.

The intervention time was 50 minutes, with five minutes for acknowledging the scenario , 15 for development, and 30 minutes for debriefing. For this activity, a classroom meeting of 4 hours/class was offered.

Debriefing is a structured way of leading students to reflect on the action. This technique helps to consolidate knowledge and to correct inappropriate behaviors. Thus, it is an essential stage in the simulation.1616 Coutinho VRD, Martins JCA, Pereira MFCR. Construção e Validação da Escala de Avaliação do Debriefing associado à Simulação (EADaS). Rev Enf Ref [Internet]. 2014; [cited 2018 nov 10]; 4(2):41-50. Available from: http://dx.doi.org/10.12707/RIII1392
http://dx.doi.org/10.12707/RIII1392...

There are several ways to perform debriefing.1717 Almeida RGS, Mazzo A, Martins JCA, Coutinho VRD, Jorge BM, Mendes IAC. Validação para a língua portuguesa da Debriefing Experience Scale. Rev Bras Enferm [Internet]. 2016 aug; [cited 2018 nov 10]; 69(4):705-11. Available from: http://dx.doi.org/10.1590/0034-7167.2016690413i
http://dx.doi.org/10.1590/0034-7167.2016...
In this research, after participating in the simulated clinical scenarios, the intervention group students were invited to: describe the sensations and activities performed during the simulation; analyze behaviors and decision making; and reflect on the behaviors and learning from the experience in the scenarios. Three students participated directly in the execution of the scenario. It is important to highlight that the students voluntarily applied to perform the scenario. The others participated actively, with the previous students, in the debriefing session.

After the students had participated in the simulated practices, the Learning Satisfaction and Self-Confidence Questionnaire was applied - to all students -, with the purpose of measuring their satisfaction and ability to perform nursing interventions after the experiments and the simulation conclusion.1111 National League for Nursing (NLN). Simulation Innovation Resource Center. SIRC Glossary; 2013; [cited 2018 nov 10]. Available from: http://sirc.nln.org/mod/glossary/view.php?id=183
http://sirc.nln.org/mod/glossary/view.ph...
In this study, the instrument was applied to both the CG (Control Group) and the EG (Experimental Group), understanding that skill training is also characterized as a low fidelity simulated practice. It is important to say that this questionnaire was developed by the National League for Nursing. In this study, the Student Satisfaction and Self-Confidence in Learning scale validated for the Portuguese language was used.1717 Almeida RGS, Mazzo A, Martins JCA, Coutinho VRD, Jorge BM, Mendes IAC. Validação para a língua portuguesa da Debriefing Experience Scale. Rev Bras Enferm [Internet]. 2016 aug; [cited 2018 nov 10]; 69(4):705-11. Available from: http://dx.doi.org/10.1590/0034-7167.2016690413i
http://dx.doi.org/10.1590/0034-7167.2016...

The applied instrument is composed of two subscales: satisfaction and self-confidence, with a total of 13 items. The satisfaction subscale contains five assessment items. In the self-confidence scale, eight items are included. Both are composed of a five-item Likert scale, namely: 1 = I strongly disagree with the statement; 2 = I disagree with the statement; 3 = undecided - I neither agree nor disagree with the statement; 4 = I agree with the statement; 5 = I strongly agree with the statement.1010 Jeffries PR. A framework for designing, implementing, and evaluating: Simulations used as teaching strategies in nursing. Nurs Educ Perspect [Internet]. 2005 mar/apr; [cited 2018 nov 10]; 26(2):96-103. Available from: https://www.ncbi.nlm.nih.gov/pubmed/15921126
https://www.ncbi.nlm.nih.gov/pubmed/1592...

Data was analyzed by the SPSS (Statistical Package for Social Sciences), version 24. To characterize the sociodemographic profile, descriptive statistics were used. In the analysis of learning satisfaction and self-confidence, the Mann - Whitney test was applied for a significance level of 5%. For the intragroup description, in the subscale variables of the instrument used, the mean value, the Standard Deviation (SD) and the mode were traced. The results were presented in tables.

RESULTS

Regarding the profile of the study participants, most students were female (79.4%). The most frequent age group was from 21 to 23 years old (61.8%), with a minimum age of 18 and a maximum of 34.

The internal consistency of the instrument used, as well as in the original versions and the validation process for Brazil, showed high levels of internal consistency;1414 Zapko KA, Ferranto MLG, Blasiman R, Shelestak D. Evaluating best educational practices, student satisfaction, and self-confidence in simulation: A descriptive study. Nurse Educ Today [Internet]. 2018 jan; [cited 2018 nov 10]; 60:28-34. Available from: https://www.ncbi.nlm.nih.gov/pubmed/28987895
https://www.ncbi.nlm.nih.gov/pubmed/2898...
the learning satisfaction and self-confidence questionnaire presented a Cronbach’s Alpha of 0.848, and a subscale of satisfaction with learning presented a Cronbach’s Alpha of 0.851 and, in the learning self-confidence subscale, Cronbach’s Alpha was 0.795.

Table 1 shows the distribution of the mean value, standard deviation, mode, maximum, minimum and percentiles of the satisfaction subscales with the current learning and learning self-confidence.

Table 1
Students' satisfaction and learning self-confidence (n=34) - Natal, 2017

Both subscales presented a mean value higher than 4.00. However, the subscale of satisfaction with current learning presented a higher mean value and a lower standard deviation regarding learning self-confidence (Mean = 3.65 and SD 0.38).

Table 2 presents the subscales and the statistical significance for both. From the Mann-Whitney’s U test, for a significance level of 5%, no statistical significance was found between the Control Groups (CGs) and Experimental Group (EG).

Table 2
Satisfaction with current learning and learning self-confidence of students in the control and experimental groups, and statistical significance - Natal, 2017

Table 3 shows the mean value, the standard deviation, and the variables’ mode concerning the subscales applied to the CG and the EG. It was possible to identify that, in the subscale of satisfaction with the current learning, the EG presented higher mean values in three of the five items versus the CG. Regarding the self-confidence learning subscale, the EG presented higher mean values in six of eight items compared to the CG.

Table 3
Students' satisfaction and learning self-confidence - Natal, 2017

DISCUSSION

Identifying how students learn more and better from what motivates them and makes them confident can be a piece of relevant evidence for improving the quality of the nurse professional training and preparation for the current world of work in the health area.

In this study, it was evidenced that both traditional teaching methods and simulation promote learning satisfaction and self-confidence among nursing students. Although such feelings are present in both strategies, it is relevant to highlight that the students of the EG, compared to the CG, had higher mean values in most variables of both subscales of the instrument used.

The results of this study are compatible with other published researches on these phenomena. A study involving 79 graduate nursing students revealed that they were satisfied with the simulation experiences, and self-confidence increased in all of them.1818 Swenty CF, Eggleston BM. The Evaluation of Simulation in a Baccalaureate Nursing Program. Clin Simul Nurs [Internet]. 2011; [cited 2018 nov 10]; 7(5):e181-e7. Available from: https://doi.org/10.1016/j.ecns.2010.02.006
https://doi.org/10.1016/j.ecns.2010.02.0...

Another study, developed with 117 nursing students in Saudi Arabia, showed that simulation as a strategy for clinical education promotes student satisfaction with learning and improves their self-confidence.1919 Omer T. Nursing students’ perceptions of satisfaction and self-confidence with clinical simulation experience. J Educ Pract [Internet]. 2016; [cited 2018 nov 10]; 7(5):131-8 Available from: https://eric.ed.gov/?id=EJ1092418
https://eric.ed.gov/?id=EJ1092418...
A study conducted with 50 students in a nursing leadership and management course showed that they were more satisfied after experiencing the simulation, and that they felt more confident about the situations lived after the simulated experience. 2020 Xiaoying MA. BSN Students’ Perception of Satisfaction and Self-confidence After a Simulated Mock Code Experience: A Descriptive Study [these]. Cedarville: Centennial Library - School of Nursing, Cedarville University; 2013; [cited 2018 nov 10]. Available from: https://pdfs.semanticscholar.org/0be3/2a17e5ef22017865d4f80a9715e1dd4e4a72.pdf
https://pdfs.semanticscholar.org/0be3/2a...

As regards the study with 199 undergraduate nursing students - which examined the perception of their satisfaction and self-confidence in the simulation, using high and medium fidelity simulators and actors - concluded that the respondents were satisfied with the experience and felt confident in their performance.1414 Zapko KA, Ferranto MLG, Blasiman R, Shelestak D. Evaluating best educational practices, student satisfaction, and self-confidence in simulation: A descriptive study. Nurse Educ Today [Internet]. 2018 jan; [cited 2018 nov 10]; 60:28-34. Available from: https://www.ncbi.nlm.nih.gov/pubmed/28987895
https://www.ncbi.nlm.nih.gov/pubmed/2898...

A satisfied student gets more motivated to learn. The satisfaction that the simulated practices generate in the student revolves around experiencing different realities in a controlled and protected environment. From the experimentation, they feel more proactive and what has been experienced takes on meaning. It is also possible to reflect on practices, contexts, scenarios, attitudes, and to minimize edges from a theoretical and clinical point of view. Therefore, the student feels more self-confident. Self-confidence is the ability to demonstrate, in a given situation, a belief in success, the powers, and the skills. Self-confidence leads to good results, thought organization, actions, and self-effectiveness.2121 Bandura A. Perceived self-efficacy in cognitive development and functioning. Educ Psych [Internet]. 1993; [cited 2018 nov 10]; 28(2):117-48. Available from: http://dx.doi.org/10.1207/s15326985ep2802_3
http://dx.doi.org/10.1207/s15326985ep280...

22 Bandura A. Self-efficacy in changing societies. New York: Cambridge University Press; 1995.
-2323 Perry P. Concept analysis: confidence/self-confidence. Nurs Forum [Internet]. 2011 oct/dec; [cited 2018 nov 10]; 46(4):218-30. Available from: https://www.ncbi.nlm.nih.gov/pubmed/22029765
https://www.ncbi.nlm.nih.gov/pubmed/2202...

In the satisfaction with current learning subscale in both groups, variables 3 “I liked how my professor taught through the simulation” and 4 “The teaching materials used in this simulation were motivating and helped me to learn” presented similar mean values, standard deviations, and modes.44 Almeida RGS, Mazzo A, Martins JCA, Baptista RCN, Girão FB, Mendes IAC. Validation to Portuguese of the Scale of Student Satisfaction and Self-Confidence in Learning. Rev Latino-Am Enfermagem [Internet]. 2015 dec; [cited 2018 nov 10]; 23(6):1007-13. Available from: http://dx.doi.org/10.1590/0104-1169.0472.2643
http://dx.doi.org/10.1590/0104-1169.0472...
These results show the satisfaction of the students regarding the conduction of the training and the researchers’ choice of resources. Besides, it may be related to the training posture and conduction in both groups by the intervention researchers, in the sense of facilitating non-biased learning.

Regarding self-confidence, the EG presented higher mean values in variables 6, 7, 8, 9, 10 and 11. These variables are related to the pursuit of knowledge, skill development, responsibility, help, and other factors that favor the development of more self-confident students.

The literature also points out that simulation generates high rates of self-effectiveness when compared to the traditional teaching strategies. An almost-experimental study, conducted with 110 students, aiming at basic life support training and which sought to assess students’ knowledge and self-effectiveness before and after educational interventions between traditional teaching methods (PowerPoint presentation and demonstration) and high fidelity simulation, identified the statistical significance and the high self-effectiveness rates in the simulation group in relation to the group with traditional methods.2424 Akhu-zaheya LM, Gharaibeh MK, Alostaz ZM. Effectiveness of simulation on knowledge acquisition, knowledge retention, and self-efficacy of nursing students in Jordan. Clin Simul Nurs [Internet]. 2013; [cited 2018 nov 10]; 9(9):e335-e42. Available from: https://www.sciencedirect.com/science/article/pii/S1876139912000710
https://www.sciencedirect.com/science/ar...

High fidelity simulation-based education can be an important component in preparing students for a successful transition to the clinical practice. Also, it contributes to student satisfaction, knowledge, confidence, competence, and critical thinking. For these reasons, simulated experiences may be an option to complement traditional nursing education methods.2525 Warren JN, Luctkar-flude M, Godfrey C, Lukewich J. A systematic review of the effectiveness of simulation-based education on satisfaction and learning outcomes in nurse practitioner programs. Nurse Educ Today [Internet]. 2016 nov; [cited 2018 nov 10]; 46(1):99-108. Available from: https://www.ncbi.nlm.nih.gov/pubmed/27621199
https://www.ncbi.nlm.nih.gov/pubmed/2762...

Corroborating with this, an almost-experimental pre-test and post-test study, involving 91 nursing students from a Singapore university, showed that after attending a 15-hour professional practice preparation course and the use of simulation, they demonstrated a significant improvement in post-test evaluations in relation to the clinical internship of transition to the practice. Also, the participants had high levels of satisfaction by learning from the simulation.2626 Liaw SY, Koh Y, Dawood R, Kowitlawakul Y, Zhou W, Lau ST. Easing student transition to graduate nurse: A simulated professional learning environment (SIMPLE) for final year student nurses. Nurse Educ Today [Internet]. 2014 mar; [cited 2018 nov 10]; 34(3):349-55. Available from: https://www.ncbi.nlm.nih.gov/pubmed/23706963
https://www.ncbi.nlm.nih.gov/pubmed/2370...

There are several evidences that, from the simulation, the students increased their satisfaction, confidence, and self-effectiveness levels in the context of the nursing undergraduate course.33 Foronda C, Liu S, Bauman EB. Evaluation of simulation in undergraduate nurse education: An integrative review. Clin Simul Nurs [Internet]. 2013 oct; [cited 2018 nov 10]; 9(10):e409-e16. Available from: https://doi.org/10.1016/j.ecns.2012.11.003
https://doi.org/10.1016/j.ecns.2012.11.0...
Furthermore, there is also evidence that satisfaction and self-confidence can improve as they move from year to year from simulation experiences.2727 Cummings C, Connelly LK. Improvement in Student Satisfaction and Confidence Levels through Simulation Activities [dissertação]. Florida: University of North Florida; 2017; [cited 2018 feb 22]. Available from: http://hdl.handle.net/10755/601810
http://hdl.handle.net/10755/601810...
The strategies, processes and/or activities that increase the students’ confidence culminate in benefits to the training of better-prepared nurses.2828 Cummings C, Connelly LK. Can nursing students’ confidence levels increase with repeated simulation activities?. Nurse Educ Today [Internet]. 2016 jan; [cited 2018 feb 22]; 36:419-21. Available from: https://www.ncbi.nlm.nih.gov/pubmed/26599594
https://www.ncbi.nlm.nih.gov/pubmed/2659...

It is herein suggested that this accumulation of knowledge and the maturity of the students, through the teaching with simulated situations, may result in a concentration of experiences in this teaching-learning process. With this, it is possible to develop in this student a researcher look, by the ability to problematize these experiences and to elaborate proposals for care improvements in the practice, which can result in satisfaction and self-confidence.

Regarding the mean values concerning variables 12 and 13 - that is, “I know how to use simulation activities to learn skills” and “It is the professor’s responsibility to tell me what I need to learn in the topic developed in the simulation during the class”, respectively -, the CG students had higher mean values (3.82 and 4.64) compared to the EG (3.70 and 4.58).44 Almeida RGS, Mazzo A, Martins JCA, Baptista RCN, Girão FB, Mendes IAC. Validation to Portuguese of the Scale of Student Satisfaction and Self-Confidence in Learning. Rev Latino-Am Enfermagem [Internet]. 2015 dec; [cited 2018 nov 10]; 23(6):1007-13. Available from: http://dx.doi.org/10.1590/0104-1169.0472.2643
http://dx.doi.org/10.1590/0104-1169.0472...

Regarding these variables, in both groups, the mode was 5, which represents a high level of agreement with the statements and emphasizes the minimum autonomy of students in the teaching and learning process. At the same time, it suggests the relevance of the professor’s facilitation to lead them in the learning objectives. This autonomy can be translated as a form of self-confidence.

It is also worth noting that simulation is as effective as the traditional methods for cognitive gain, skills development, satisfaction, and self-confidence. Thus, it can be accepted as an effective method of laboratory teaching.2929 Stroup C. Simulation usage in nursing fundamentals: Integrative literature review. Clin Simul Nurs [Internet]. 2014 mar; [cited 2018 feb 22]; 10(3):e155-e64. Available from: https://www.sciencedirect.com/science/article/pii/S1876139913002351
https://www.sciencedirect.com/science/ar...

In addition, the simulation favors the development of psychomotor skills, social skills, creativity, information seeking, reasoning, prediction, problem-solving, teamwork, assessment, decision making, medication administration, prioritization, cognitive knowledge related to nursing content, critical thinking, collaboration, clinical learning, high score test performance, and communication skills.33 Foronda C, Liu S, Bauman EB. Evaluation of simulation in undergraduate nurse education: An integrative review. Clin Simul Nurs [Internet]. 2013 oct; [cited 2018 nov 10]; 9(10):e409-e16. Available from: https://doi.org/10.1016/j.ecns.2012.11.003
https://doi.org/10.1016/j.ecns.2012.11.0...

The simulation is also a strategy often mentioned in the development of interdisciplinary experiences to promote teamwork, collaboration, understanding, and communication. The research studies suggest that the simulation improves interdisciplinary communication. 33 Foronda C, Liu S, Bauman EB. Evaluation of simulation in undergraduate nurse education: An integrative review. Clin Simul Nurs [Internet]. 2013 oct; [cited 2018 nov 10]; 9(10):e409-e16. Available from: https://doi.org/10.1016/j.ecns.2012.11.003
https://doi.org/10.1016/j.ecns.2012.11.0...

It is valid to consider that, in the context of nursing education, the use of simulation is not recent. However, only in recent years, there have been references to the insertion of high fidelity simulation.1515 Campbell DT, Stanley JC. Delineamentos experimentais e quase-experimentais de pesquisa. São Paulo: USP/EPU; 1979. Also, several factors interfere with the simulation realization in the nursing curriculum such as, for instance, excessive working hours, commitment to technology-based learning, professor expertise and isolation in specific areas, and lack of experience in research development.33 Foronda C, Liu S, Bauman EB. Evaluation of simulation in undergraduate nurse education: An integrative review. Clin Simul Nurs [Internet]. 2013 oct; [cited 2018 nov 10]; 9(10):e409-e16. Available from: https://doi.org/10.1016/j.ecns.2012.11.003
https://doi.org/10.1016/j.ecns.2012.11.0...

In this study, students strongly agreed with the statement that signals their responsibility to what they need to know through the simulation activity. However, while stating this, they were indecisive about their orientation in learning skills from simulation activities. Besides this, they pointed out the professor’s responsibility in guiding them to identify the learning objectives.

In the self-confidence learning subscale, regarding variable 1344 Almeida RGS, Mazzo A, Martins JCA, Baptista RCN, Girão FB, Mendes IAC. Validation to Portuguese of the Scale of Student Satisfaction and Self-Confidence in Learning. Rev Latino-Am Enfermagem [Internet]. 2015 dec; [cited 2018 nov 10]; 23(6):1007-13. Available from: http://dx.doi.org/10.1590/0104-1169.0472.2643
http://dx.doi.org/10.1590/0104-1169.0472...
“It is the professor’s responsibility to tell me what I need to learn in the topic developed in the simulation during the class.”, the EG presented a lower mean value than the CG (4.58 and 4.64, respectively). These data can be related to the debriefing strategy in the simulation, since, unlike the traditional teaching strategies, the simulation suggests the need for student reflection on the scenario knowledge and its actions.

For this reason, it is important to highlight that the insertion of simulation in nursing education requires the creation of a cooperation culture to strengthen technical capacities among educators.2424 Akhu-zaheya LM, Gharaibeh MK, Alostaz ZM. Effectiveness of simulation on knowledge acquisition, knowledge retention, and self-efficacy of nursing students in Jordan. Clin Simul Nurs [Internet]. 2013; [cited 2018 nov 10]; 9(9):e335-e42. Available from: https://www.sciencedirect.com/science/article/pii/S1876139912000710
https://www.sciencedirect.com/science/ar...
Partnerships and cooperation between educational and research institutions seem to be indispensable for strengthening these capacities.

It is worth noting that satisfaction with learning can be influenced by the teaching methods used, the way the professor teaches and conducts the students’ learning and motivation. To do so, the professor needs to know different ways of teaching and learning, share the knowledge they have according to learning objectives, support students and encourage them to go beyond.

The struggle for integrating simulation into the nursing curriculum, maximizing its potential, student evaluation and the effectiveness of this strategy has been a frequent activity for many nursing educators.33 Foronda C, Liu S, Bauman EB. Evaluation of simulation in undergraduate nurse education: An integrative review. Clin Simul Nurs [Internet]. 2013 oct; [cited 2018 nov 10]; 9(10):e409-e16. Available from: https://doi.org/10.1016/j.ecns.2012.11.003
https://doi.org/10.1016/j.ecns.2012.11.0...
For more support, performing research is needed to elucidate the potential of simulation in different contexts and settings in nursing education. This investment is expected to improve teaching, research in this area, and student outcomes.33 Foronda C, Liu S, Bauman EB. Evaluation of simulation in undergraduate nurse education: An integrative review. Clin Simul Nurs [Internet]. 2013 oct; [cited 2018 nov 10]; 9(10):e409-e16. Available from: https://doi.org/10.1016/j.ecns.2012.11.003
https://doi.org/10.1016/j.ecns.2012.11.0...

From the teaching perspective, we hope that the approaches used during the training may contemplate and allow the nurses’ performance in the current demands of the health work world, the complexity of life and nursing care.

From the research point of view, we expect that it provides solid evidence and that it contributes to the evolution of the nursing science, since research in different contexts of nursing contributes to the consolidation of this science.3030 Costa RRO, Medeiros SM, Martins JCA, Dias VR. Perceptions of nursing students on the structural dimensions of clinical simulation. Sci Med [Internet]. 2019; [cited 2018 feb 22]; 29(1):e32972. Available from: https://doi.org/10.15448/1980-6108.2019.1.32972
https://doi.org/10.15448/1980-6108.2019....

Thus, satisfied and self-confident students are expected to be able to master the content presented to them, develop competencies and skills, be able to perform what has been learned in the clinical setting and change the realities into which they will be inserted.

CONCLUSION

The study concluded that both the traditional teaching strategies (lecture classes and skill training) and the simulation promote student satisfaction and self-confidence in learning. There was no statistical significance in the learning satisfaction subscale (p value ≥ 0.05) and learning self-confidence (p value ≥ 0.05). However, in the study, the experimental group presented higher mean values in most of the variables of both subscales. Because they generate satisfaction and self-confidence, simulation and traditional strategies can be mutually and cumulatively used in nursing education.

The sample (n=34) is one of the study’s limitations. Due to the long follow-up of the participants, losses were inevitable. Neither was possible to compare the study with a similar design and sample. Although there are limitations, this study contributes to filling the gap in the literature regarding the identification of satisfaction and student learning self-confidence, from the comparison among different teaching and learning strategies.

The researchers suggest that the simulation be added to the undergraduate nursing curriculum as a complementary strategy to the existing ones, especially in teaching topics related to Primary Health Care, in which simulation studies and experiments are still incipient. Besides, institutions are encouraged to promote improvements in the laboratory infrastructure and in the faculty capacity to work with the simulation strategy.

  • FINANCING
    The present work was carried out with the support of the Coordination of Improvement of Higher Level Personnel - Brazil (CAPES) - Financing Code 001.

ACKNOWLEDGMENT

To professor José Carlos Amado Martins, for supporting the development of this study.

REFERENCES

  • 1
    Morin EA. Cabeça bem-feita. Repensar a reforma, reformar o pensamento. Rio de Janeiro: Bertrand Brasil; 2014.
  • 2
    Costa RRO, Medeiros SM, Martins JCA, Coutinho VRD. A simulação no ensino de enfermagem: reflexões e justificativas à luz da bioética e dos direitos humanos. Acta Bioeth [Internet]. 2018 jun; [cited 2018 nov 10]; 24(1):31-8. Available from: http://dx.doi.org/10.4067/S1726-569X2018000100031
    » http://dx.doi.org/10.4067/S1726-569X2018000100031
  • 3
    Foronda C, Liu S, Bauman EB. Evaluation of simulation in undergraduate nurse education: An integrative review. Clin Simul Nurs [Internet]. 2013 oct; [cited 2018 nov 10]; 9(10):e409-e16. Available from: https://doi.org/10.1016/j.ecns.2012.11.003
    » https://doi.org/10.1016/j.ecns.2012.11.003
  • 4
    Almeida RGS, Mazzo A, Martins JCA, Baptista RCN, Girão FB, Mendes IAC. Validation to Portuguese of the Scale of Student Satisfaction and Self-Confidence in Learning. Rev Latino-Am Enfermagem [Internet]. 2015 dec; [cited 2018 nov 10]; 23(6):1007-13. Available from: http://dx.doi.org/10.1590/0104-1169.0472.2643
    » http://dx.doi.org/10.1590/0104-1169.0472.2643
  • 5
    Kootler P. Administração de marketing: análise, planejamento, implementação e controle. 5ª ed. São Paulo: Atlas; 1998.
  • 6
    Kootler P, Clarke RN. Marketing for health care organizations. Englewood Cliffs, NJ: Prentice-Hall; 1987.
  • 7
    Lemos SIM. Análise da satisfação de estudantes num curso em e-learning no ensino superior [dissertação]. Lisboa: Universidade de Lisboa; 2011.
  • 8
    DeYoung S. Teaching strategies for nurse educators. Upper Saddle River: Prentice Hall Health; 2003.
  • 9
    Baptista RCN, Martins JCA, Pereira MFCR, Mazzo A. Students’ satisfaction with simulated clinical experiences: validation of an assessment scale. Rev Latino-Am Enfermagem [Internet]. 2014 oct; [cited 2018 nov 10]; 22(5):709-15. Available from: http://dx.doi.org/10.1590/0104-1169.3295.2471
    » http://dx.doi.org/10.1590/0104-1169.3295.2471
  • 10
    Jeffries PR. A framework for designing, implementing, and evaluating: Simulations used as teaching strategies in nursing. Nurs Educ Perspect [Internet]. 2005 mar/apr; [cited 2018 nov 10]; 26(2):96-103. Available from: https://www.ncbi.nlm.nih.gov/pubmed/15921126
    » https://www.ncbi.nlm.nih.gov/pubmed/15921126
  • 11
    National League for Nursing (NLN). Simulation Innovation Resource Center. SIRC Glossary; 2013; [cited 2018 nov 10]. Available from: http://sirc.nln.org/mod/glossary/view.php?id=183
    » http://sirc.nln.org/mod/glossary/view.php?id=183
  • 12
    Costa RRO, Medeiros SM, Martins JCA, Cossi MS, Araújo MS. Percepção de estudantes da graduação em enfermagem sobre a simulação realística. Rev Cuid [Internet]. 2017 sep/dec; [cited 2018 nov 10]; 8(3):1799-1808. Available from: http://dx.doi.org/10.15649/cuidarte.v8i3.425
    » http://dx.doi.org/10.15649/cuidarte.v8i3.425
  • 13
    Norman J. Systematic Review of the Literature on Simulation in Nursing Education. ABNF Journal [Internet]. 2012; [cited 2018 nov 10]; 23(2):24-28. Available from: https://www.ncbi.nlm.nih.gov/pubmed/22774355
    » https://www.ncbi.nlm.nih.gov/pubmed/22774355
  • 14
    Zapko KA, Ferranto MLG, Blasiman R, Shelestak D. Evaluating best educational practices, student satisfaction, and self-confidence in simulation: A descriptive study. Nurse Educ Today [Internet]. 2018 jan; [cited 2018 nov 10]; 60:28-34. Available from: https://www.ncbi.nlm.nih.gov/pubmed/28987895
    » https://www.ncbi.nlm.nih.gov/pubmed/28987895
  • 15
    Campbell DT, Stanley JC. Delineamentos experimentais e quase-experimentais de pesquisa. São Paulo: USP/EPU; 1979.
  • 16
    Coutinho VRD, Martins JCA, Pereira MFCR. Construção e Validação da Escala de Avaliação do Debriefing associado à Simulação (EADaS). Rev Enf Ref [Internet]. 2014; [cited 2018 nov 10]; 4(2):41-50. Available from: http://dx.doi.org/10.12707/RIII1392
    » http://dx.doi.org/10.12707/RIII1392
  • 17
    Almeida RGS, Mazzo A, Martins JCA, Coutinho VRD, Jorge BM, Mendes IAC. Validação para a língua portuguesa da Debriefing Experience Scale. Rev Bras Enferm [Internet]. 2016 aug; [cited 2018 nov 10]; 69(4):705-11. Available from: http://dx.doi.org/10.1590/0034-7167.2016690413i
    » http://dx.doi.org/10.1590/0034-7167.2016690413i
  • 18
    Swenty CF, Eggleston BM. The Evaluation of Simulation in a Baccalaureate Nursing Program. Clin Simul Nurs [Internet]. 2011; [cited 2018 nov 10]; 7(5):e181-e7. Available from: https://doi.org/10.1016/j.ecns.2010.02.006
    » https://doi.org/10.1016/j.ecns.2010.02.006
  • 19
    Omer T. Nursing students’ perceptions of satisfaction and self-confidence with clinical simulation experience. J Educ Pract [Internet]. 2016; [cited 2018 nov 10]; 7(5):131-8 Available from: https://eric.ed.gov/?id=EJ1092418
    » https://eric.ed.gov/?id=EJ1092418
  • 20
    Xiaoying MA. BSN Students’ Perception of Satisfaction and Self-confidence After a Simulated Mock Code Experience: A Descriptive Study [these]. Cedarville: Centennial Library - School of Nursing, Cedarville University; 2013; [cited 2018 nov 10]. Available from: https://pdfs.semanticscholar.org/0be3/2a17e5ef22017865d4f80a9715e1dd4e4a72.pdf
    » https://pdfs.semanticscholar.org/0be3/2a17e5ef22017865d4f80a9715e1dd4e4a72.pdf
  • 21
    Bandura A. Perceived self-efficacy in cognitive development and functioning. Educ Psych [Internet]. 1993; [cited 2018 nov 10]; 28(2):117-48. Available from: http://dx.doi.org/10.1207/s15326985ep2802_3
    » http://dx.doi.org/10.1207/s15326985ep2802_3
  • 22
    Bandura A. Self-efficacy in changing societies. New York: Cambridge University Press; 1995.
  • 23
    Perry P. Concept analysis: confidence/self-confidence. Nurs Forum [Internet]. 2011 oct/dec; [cited 2018 nov 10]; 46(4):218-30. Available from: https://www.ncbi.nlm.nih.gov/pubmed/22029765
    » https://www.ncbi.nlm.nih.gov/pubmed/22029765
  • 24
    Akhu-zaheya LM, Gharaibeh MK, Alostaz ZM. Effectiveness of simulation on knowledge acquisition, knowledge retention, and self-efficacy of nursing students in Jordan. Clin Simul Nurs [Internet]. 2013; [cited 2018 nov 10]; 9(9):e335-e42. Available from: https://www.sciencedirect.com/science/article/pii/S1876139912000710
    » https://www.sciencedirect.com/science/article/pii/S1876139912000710
  • 25
    Warren JN, Luctkar-flude M, Godfrey C, Lukewich J. A systematic review of the effectiveness of simulation-based education on satisfaction and learning outcomes in nurse practitioner programs. Nurse Educ Today [Internet]. 2016 nov; [cited 2018 nov 10]; 46(1):99-108. Available from: https://www.ncbi.nlm.nih.gov/pubmed/27621199
    » https://www.ncbi.nlm.nih.gov/pubmed/27621199
  • 26
    Liaw SY, Koh Y, Dawood R, Kowitlawakul Y, Zhou W, Lau ST. Easing student transition to graduate nurse: A simulated professional learning environment (SIMPLE) for final year student nurses. Nurse Educ Today [Internet]. 2014 mar; [cited 2018 nov 10]; 34(3):349-55. Available from: https://www.ncbi.nlm.nih.gov/pubmed/23706963
    » https://www.ncbi.nlm.nih.gov/pubmed/23706963
  • 27
    Cummings C, Connelly LK. Improvement in Student Satisfaction and Confidence Levels through Simulation Activities [dissertação]. Florida: University of North Florida; 2017; [cited 2018 feb 22]. Available from: http://hdl.handle.net/10755/601810
    » http://hdl.handle.net/10755/601810
  • 28
    Cummings C, Connelly LK. Can nursing students’ confidence levels increase with repeated simulation activities?. Nurse Educ Today [Internet]. 2016 jan; [cited 2018 feb 22]; 36:419-21. Available from: https://www.ncbi.nlm.nih.gov/pubmed/26599594
    » https://www.ncbi.nlm.nih.gov/pubmed/26599594
  • 29
    Stroup C. Simulation usage in nursing fundamentals: Integrative literature review. Clin Simul Nurs [Internet]. 2014 mar; [cited 2018 feb 22]; 10(3):e155-e64. Available from: https://www.sciencedirect.com/science/article/pii/S1876139913002351
    » https://www.sciencedirect.com/science/article/pii/S1876139913002351
  • 30
    Costa RRO, Medeiros SM, Martins JCA, Dias VR. Perceptions of nursing students on the structural dimensions of clinical simulation. Sci Med [Internet]. 2019; [cited 2018 feb 22]; 29(1):e32972. Available from: https://doi.org/10.15448/1980-6108.2019.1.32972
    » https://doi.org/10.15448/1980-6108.2019.1.32972

Publication Dates

  • Publication in this collection
    13 Dec 2019
  • Date of issue
    2020

History

  • Received
    31 Mar 2019
  • Accepted
    16 Sept 2019
Universidade Federal do Rio de Janeiro Rua Afonso Cavalcanti, 275, Cidade Nova, 20211-110 - Rio de Janeiro - RJ - Brasil, Tel: +55 21 3398-0952 e 3398-0941 - Rio de Janeiro - RJ - Brazil
E-mail: annaneryrevista@gmail.com