Randomized study: lecture versus gamification in knowledge retention in Medical studentsABSTRACT
Introduction: Even in the presence of a generation of students created in the gaming and interactivity environment, the Gamification methodology associated with interactive stories has not yet been sufficiently tested through studies with greater statistical power.
Objective: This study aims to compare knowledge retention in medical students exposed to lectures or classes through games and interactive stories and to compare levels of knowledge acquisition and satisfaction after exposure to the two mentioned different methodologies.
Methods: It is characterized as a randomized, controlled, crossover study comparing the use of Gamification classes and Interactive Stories with the use of Traditional Classroom method. The main outcomes assessed were the level of knowledge retention one month after exposure to the last class, the increase in knowledge before and after each class, and the level of satisfaction at the end of each class. Statistical analysis was performed using the Wilcoxon signed rank test with a significance level set at 5%.
Results: Eighty-three students attending the 5th and 6th years of medical school were enrolled in this study. Knowledge fixation in the Gamification and Stories group was 3.6 times higher (p <0.001) than in the Lecture group. Knowledge acquisition and satisfaction levels were also higher in the Gamification group.
Conclusion: The present study indicates that the Gamification methodology with Interactive Stories methodology is superior to the Traditional Class regarding knowledge retention in medical students. Moreover, the knowledge acquisition and satisfaction levels were also superior.
Keywords:
Medical education; Gamification; Educational Models
RESUMO
Introdução: Mesmo diante de uma geração de estudantes criada no ambiente dos jogos e da interatividade, a metodologia da gamificação associada a histórias interativas ainda não foi suficientemente testada por meio de estudos com maior poder estatístico.
Objetivo: Este estudo visa comparar a fixação de conhecimento de estudantes de Medicina expostos à aula expositiva ou aula por meio de jogos e histórias interativas, e comparar os níveis de aquisição de conhecimento e de satisfação após a exposição às duas diferentes metodologias referidas.
Método: Trata-se de um estudo randomizado, controlado e cruzado que compara o uso de aulas com gamificação e histórias interativas com o uso de aula tradicional. Os principais desfechos avaliados foram o nível de fixação de conhecimento um mês depois da exposição à última aula, o incremento de conhecimento antes e depois de cada aula, e o nível de satisfação ao final de cada aula. Realizou-se análise estatística usando-se o Teste dos Sinais de Wilcoxon, ao nível de significância de 5%.
Resultado: Incluíram-se no estudo 83 estudantes do quinto e do sexto ano. A fixação de conhecimento no grupo de gamificação e histórias foi 3,6 vezes maior (p < 0,001) do que no grupo de aula expositiva. Os níveis de aquisição de conhecimento e a satisfação também foram superiores no grupo de gamificação.
Conclusão: O presente estudo indica que a aula com metodologia de gamificação e histórias interativas é superior à aula tradicional na fixação do conhecimento em estudantes de Medicina. Além disso, observou-se também superioridade nos níveis de aquisição de conhecimento e de satisfação.
Palavras-chave:
Educação Médica; Gamificação; Modelos Educacionais
INTRODUCTION
Even currently, many curricula of medical courses are based on the mechanistic model of the health-disease process1. It is known that in the fields of practice of medical schools, the return of basic scientific knowledge from the classroom is often slow and inappropriate. Only after making new connections with the clinical context does the final learning take place1),(2. Innovative pedagogical practices, in the context of the active methodology, seek to stimulate critical and reflective teaching processes, in which the student interacts and commits to their learning3),(4. Studies in different countries have suggested the strength of the formal use of interactive games in the apprehension of knowledge in medicine5)-(7. Additionally, there are data suggesting greater knowledge retention and greater satisfaction, including implications for medical practice, with better results in the application of the contents to care7),(8. Even in the presence of a generation of students raised and accustomed with games and the digital world, these aspects had not yet been sufficiently tested. In the context of innovative pedagogical practices, the existing data are not sufficient to confirm or refute the usefulness of competitive games as an effective teaching strategy for medical students. In this sense, this project seeks to evaluate, on scientific bases, a combination of active methodologies consisting of Gamification and Interactive Stories, compared to the traditional classroom model, regarding content retention. Regardless of the gains in knowledge arising from any of the teaching models, we are willing to measure and compare the retention of information in the context of an environment centered not on the content or on the teacher, but on the individual and their social performance integrated in a playful activity.
METHODS
Study design and setting
This is a randomized trial, carried out in a care and teaching hospital that offers Internal Medicine services through the Brazilian Unified Health System in the city of Salvador, state of Bahia, Brazil. The study target population comprised students attending the fifth and sixth years of medical school from four public and private institutions. However, the accessible population of the trial comprised students attending the Internal Medicine internship period of these institutions who were randomly assigned to the abovementioned hospital during the study months. As there were no exclusion criteria and as the inclusion criteria were met, medical students from the aforementioned service who agreed to participate in the study were selected. All students signed the Free and Informed Consent Form (TCLE, Termo de Consentimento Livre e Informado). The project was approved by the local Research Ethics Committee. under CAAE number 97784918.2.0000.0047.
Procedure
The study protocol consists of six stages. The first stage consisted of preparation for the research and teacher training. At that time, 36 classes were prepared, over a two-year period, which addressed syndromic topics of Internal Medicine, and were included in the didactic program of the medical internship. Thus, the resident physicians of the service were trained by the investigator to teach these classes, using two different educational methodologies. Then, in the second stage, the participants who came for the internship period at the hospital were invited to participate, being randomly divided into two groups of similar size (Gamification group and Lecture Group) aiming to follow the stratification by university and by year of the course, fifth or sixth year. Moreover, the topics “Chest Pain”, “Weaknesses (myopathy)”, “Hepatosplenomegaly” and “Pruritus” were randomly selected among the 36 previously prepared topics, aiming to eliminate any bias arising from the topic selection and minimize the influence of previous knowledge of the subject by the students or residents selected to administer the classes using the two methodologies. Sample size calculation was made based on the average grades of the students measured in the education studies cited in the literature review, and it was postulated that if the variation between the grades of the groups was 2 points, a number (N) equal to 36 individuals per group would be necessary to demonstrate this difference.
In the third stage, the first topics were presented and the pre- and post-tests were applied. The only difference between the groups was the use of the methodology of games and interactive stories for one of them (Gamification) and traditional class (Lecture) for the other group. Thus, both groups took the same tests, on the same days, and at the same time, before and after attending the same class in different auditoriums of the hospital.
Following the study protocol, the students were exposed to the remaining topics for two days. The classes were taught by eight resident physicians drawn from a total of 38 physicians at the hospital, all previously trained by the investigator to apply the two types of methodologies. The reason why they were resident physicians was to minimize some bias that could arise if the classes were taught by specialists, with different didactic backgrounds.
The students were evaluated through a pre- and post-system with printed tests applied at the beginning and end of each of the eight classes. These tests consisted of five multiple-choice questions for each of the topics, previously prepared by specialist teachers, different from those who taught the classes: Myopathy by a neurologist; chest pain by a cardiologist; hepatomegaly by a gastroenterologist and pruritus by a dermatologist. The test questions were not known to the researcher or the physicians who taught the classes, nor should they coincide with the content of the cases used in the gamification class. The same test was applied in the classes of the two groups and repeated, aiming to investigate the level of decrease in correct answers after one month of contact with the contents. The tests were applied by teachers who did not teach the class.
In the application of the Gamification Methodology, mini clinical cases were presented during classes and read by a narrator, chosen by the audience, with questions being asked at the end. The cases always had as backdrop a fictional, playful and interactive script, lulled with music and sounds related to the topics. The interns were divided into two groups of five to six people, who had to compete through the number of correct diagnoses of the mini-cases, with a defined time of four minutes for discussion and debate among the members before sending the answers. At the end of the class, the winning group with the highest number of correct answers was revealed and then, after two consecutive classes, the big winner among the teams was announced.
When applying the Traditional Methodology, the same topics were presented, only this time through lectures, containing the same slides and with the same content as the gamification class. The differences between the application of the methodologies were the addition of gamification and the teachers, who however were resident physicians, drawn from the same group, and who had the same training.
In the fourth stage, the study participants were submitted to a crossover strategy, with the members of the Gamification Group being transferred to the Lecture Group in the third week and vice versa, with the aim of minimizing any potential bias. In the fifth stage, the last two topics were presented, and repetition of the testing process occurred.
The sixth stage consisted of evaluating knowledge retention, so one month after the last class, all students were submitted to the same tests as in stage III. The researcher did not participate in classes or tests at any time during the study.
Student satisfaction was assessed at the end of each class, using a Likert-type scale9)-(12 of five graded levels, from the extreme of “very dissatisfied” to the “very satisfied” level. This scale was previously validated in the preparation phase of the study, with good understanding by the tested students and good capacity for discernment and differentiation between the five levels of satisfaction.13
The data collected from the knowledge retention assessment, carried out one month after the last class, and the students’ satisfaction, assessed at the end of each class, were submitted to an analysis and these data were treated by analyzing their respective frequencies and proportions in relation to the total.
The variables evaluated in the study were the increase in test scores and the level of Student Satisfaction after contact with the Gamification and Traditional methods. In the first variable, the difference between the scores obtained after applying the Gamification and Traditional Methodology and before the methodology was considered, as well as the difference between the scores obtained after applying the Gamification and Traditional Methodology and in the retention evaluation. Statistical significance was assessed using the Wilcoxon Signed Rank Test.
The data did not follow a normal distribution, either in the graphical analysis of the QQ - Plot and BoxPlot or in the Shapiro-Wilk Normality test. All analyses were obtained with the aid of the R-Project Statistical Software.
RESULTS
The study population consisted of 83 medical students during their medical internship. The mean age found was 24 years, with 43 female students and 34 male students. Of the total, 60 students were attending the sixth year of medical school and 23 were attending the fifth year.
The population came from four different medical teaching institutions in the state of Bahia, which were not mentioned for ethical reasons. The randomization of all the abovementioned characteristics followed a pattern of paired randomization and is explained in the table below. Only two students did not finish the study, which did not compromise the statistical analysis.
With the analysis by the non-parametric test of the Wilcoxon Signed Rank Test (Table 2), the results statistically showed the rejection of H0, that is, the Median Increment with the Gamification methodology was higher than the Median Increment with the traditional methodology (all p-values show values much lower than 0.05).
The retention of knowledge one month later, also analyzed by the Wilcoxon Signed Rank Test, showed a lower decrease in the median of scores in the Gamification Group.
Satisfaction with the Gamification methodology associated with Interactive Stories was also higher, with 89.6% of students comprising levels 4 and 5 of satisfaction (satisfied or very satisfied), in the Gamification Group. In the Traditional Class Group, only 34.6% represented the number of students who were at these same levels. This comparison showed a p value <0.000.
DISCUSSION
Compared to the traditional class, the study results showed that in the Gamification with Interactive Stories group, the gain in knowledge was 32% greater between the pre- and post-test and, mainly, the learning decrease was 3.7 times lower, in the evaluation carried out one month later.
The studies by Mayel et al., Imran et al. and Gong et al. obtained similar results by finding statistically significant differences indicating better post-test performance for students who participated in team-based learning (TBL) methodologies compared to traditional teaching methodologies. 14)-(16)
In relation to the participants’ satisfaction and overall perception in the experiments, the work developed by Meyel et al. does not apply any methodology that seeks to evaluate more subjective concepts of the teaching model. Nevertheless, it shows that the TBL method increased interpersonal skills in female students; however, this statement was derived only from the comparison of scores between groups with TBL methodology and traditional classes.
Similar to the present study, the investigations by Meyel et al. and Gong et al.14),(16 applied satisfaction questionnaires to the groups, indicating that the students obtained greater satisfaction with the TBL method, considering it a better learning strategy. Moreover, the participants also had the impression that in addition to a better form of content retention, the method can promote the improvement and gain of interpersonal skills such as teamwork, effective communication, teaching and collaboration in academic activities and greater development of clinical reasoning through discussions.
As far as the current research demonstrates, consistent with the results of the present study, only the work of Delgado et al.17 aimed to evaluate knowledge retention through an evaluation one month after the content exposure sessions.
However, similarly to the study by Meyel et al.16 and differently from the present study, it evaluates the students’ perceptions only by evaluating the high scores and associating them with positive perceptions of the methodology and the acquisition of knowledge.
As already mentioned, knowledge retention in the present study was much higher in the Gamification group, with no significant decrease in the median of grades one month later (0.0), compared to the traditional class group (-2.0). A crucial point to be discussed and sought are the possible reasons that may participate in the explanation for such a significant result.
The first group of hypotheses, to be evaluated in new specific studies, come from deep and interesting research on the mechanisms that influence memory, especially information retention in long-term memory.
According to the studies by Atkinson and Shiffrin18, there is a direct relationship between the amount of recitation in short-term storage and the power of the memory trace stored in long-term memory. Mechanisms such as having to repeat information to other people such as members of the same team or having to defend a point of view to find a solution, as occurred in the study, could decisively influence a higher quality of information retention.6),(19
These results of much greater retention in the gamification group lead us to raise possible explanations, based on classic studies of cognitive psychology on the modulating mechanisms of long-term memory. The action of defending a point of view to find a solution, as occurred in the study, could decisively influence a higher quality of information retention6),(19. The use of narration, music and storytelling could be another explanation for the change in the learning experience, as it is already known that visual (iconic) and sound (echoic) memory positively influence retention20),(21. Therefore, the greater the range of sensations in which meaning is processed, the deeper the level of processing (source). On the other hand, Morris et al.22 demonstrated in their research that memory depends on the specific demands that justify its use. Thus, when people know that they will answer a test, and with competitive value, they could record better.
There is also the influence of brain networks involved in the act of listening to stories and intervening in them, as occurs in the gamification group of the study. This type of teaching can allow learning that transcends the cognitive level, to also act on the affective and experiential levels, expanding the student’s perception13. A plot that correlates characters and captures the attention and arouses emotions in the listener could convey the message in a way that is easier to understand, due to the emotional and social aspects present in it.
Additionally, the level of satisfaction was also higher in the Gamification group. On a Likert-type scale9)-(12 of graded levels of satisfaction measured from 1 to 5, 56% of the students in the Gamification group gave the maximum score to this methodology, as opposed to only 6.8% in the Traditional Class group.
According to Kirkpatrick’s studies23, a student satisfied with the activity would better absorb the content and apply this knowledge in their environment. Thus, the high satisfaction measured in the gamification group could influence learning. In addition to the satisfaction evaluation, this author also suggests learning evaluations through pre- and post-test questionnaires, such as those employed in this study.
Finally, the warning of the authors Berbel and Behrens24),(25 about the inherent need for the exercise of rupture in the educational process returns. Thus, the challenge and encouragement to always seek, students and teachers, to go beyond the apparent limits are essential. From this viewpoint, the innovative hypothesis on which this study is based (an unusual blend of active methodologies) and the results found on retention and learning are reasons not only to add these methodologies to the repertoire of possible activities, but, in addition, to reflect on what we can do in practice to make education truly transformative.
As a future perspective, there is the need to replicate the project and incorporate it into other teaching places and for other topics and subjects in addition to internal medicine.
The study makes available to educators the association of gamification with storytelling, a blend of teaching methodologies that has proven to be inexpensive, engaging and effective. Using the game and narrative environment, without the need for technological sophistication, and with educational resources that include a room, a data show device and a stereo device.
The level of retention and satisfaction achieved with few resources and based only on the content and active participation of the student should be observed.
The limitations of the study include the fact that the data were collected only with internal medicine topics and with internship students, which may restrict the extrapolation of these results to other segments.
CONCLUSION
The association of gamification and interactive stories proved to be superior to the lecture in increasing knowledge between the pre- and the immediate post-test periods. Additionally, knowledge retention in the Gamification with Interactive Stories group did not attain a median grade decrease after 30 days, while the Traditional Classes group showed a median decrease of 2 points in the grades. Moreover, student satisfaction was also higher in the Gamification group. Therefore, this particular association of methodologies is available as a useful and easy-to-apply educational strategy for educators and medical education institutions.
References
- 1 Silva Souza C da, Iglesias AG, Pazin-Filho A. Estratégias inovadoras para métodos de ensino tradicionais - aspectos gerais. Rev Med (Ribeirão Preto). 2014;47(3):284-92.
- 2 Cyrino EG, Toralles-Pereira ML. Trabalhando com estratégias de ensino-aprendizado por descoberta na área da saúde: a problematização e a aprendizagem baseada em problemas. Cad Saude Publica. 2004;20(3):780-8.
- 3 Capra F. O ponto de mutação: a ciência, a sociedade e a cultura emergente. 30a ed. São Paulo: Cultrix; 2006.
- 4 Siqueira-Batista R, Siqueira-Batista R. Os anéis da serpente: a aprendizagem baseada em problemas e as sociedades de controle. Cien Saude Colet. 2009;14(4):1183-92.
- 5 Gentry SV, Gauthier A, Ehrstrom BLE, Wortley D, Lilienthal A, Car LT, et al. Serious gaming and gamification education in health professions: systematic review. J Med Internet Res. 2019;21(3):e12994.
- 6 McCoy L, Lewis JH, Dalton D. Gamification and multimedia for medical education: a landscape review. J Am Osteopath Assoc. 2016;116(1):22-34.
- 7 Diehl LA, Souza RM, Gordan PA, Esteves RZ, Coelho ICM. InsuOnline, an electronic game for medical education on insulin therapy: a randomized controlled trial with primary care physicians. J Med Internet Res . 2017;19(3):e6944.
- 8 Araujo TB de, Silveira FR, Souza DLS, Strey YTMH, Flores CD, Webster RS. Impact of video game genre on surgical skills development: a feasibility study. J Surg Res. 2016;201(1):235-43.
- 9 Dalmoro M, Vieira KM. Dilemas na construção de escalas tipo Likert: o número de itens e a disposição influenciam nos resultados? Rev Gest Organ. 2014;6(3).
- 10 Alexandre J, Andrade D, Vasconcelos A, Araujo A, Batista M. Análise do número de categorias da escala de Likert aplicada à gestão pela qualidade total através da teoria da resposta ao item. XXIII Encontro Nacional de Engenharia de Produção. Ouro Preto: Instituição; 2003.
- 11 Cunha LMA da. Modelos Rasch e escalas de Likert e Thurstone na medição de atitudes. Lisboa: Universidade de Lisboa; 2007.
- 12 Silva Júnior SD da, Costa FJ. Mensuração e escalas de verificação: uma análise comparativa das escalas de Likert e Phrase Completion. Rev Bras Pesq Mark. 2014;
- 13 Kumagai AK. A conceptual framework for the use of illness narratives in medical education. Acad Med. 2008;83(7):653-8.
- 14 Gong J, Du J, Hao J, Li L. Effects of bedside team-based learning on pediatric clinical practice in Chinese medical students. BMC Med Educ. 2022;22(1).
- 15 Imran M, Halawa TF, Baig M, Almanjoumi AM, Badri MM, Alghamdi WA. Team-based learning versus interactive lecture in achieving learning outcomes and improving clinical reasoning skills: a randomized crossover study. BMC Med Educ . 2022;22(1).
- 16 Mayel M, Hoseini SH, Teimouri A, Shabouni Z, Rayat-Dost E, Foroughian M. Teaching approach to tachycardia and bradycardia in medical students: a quasi-experimental study to compare team-based learning and lecture method. Front Emerg Med. 2020;4(3):e68-e68.
- 17 Castro Delgado R, Fernández García L, Cernuda Martínez JA, Cuartas Álvarez T, Arcos González P. Training of medical students for mass casualty incidents using table-top gamification. Disaster Med Public Health Prep. 2023;17(4).
- 18 Atkinson RC, Shiffrin RM. Human memory: a proposed system and its control processes. Psychol Learn Motiv - Adv Res Theory. 1968;2(C):89-195.
- 19 Hícaro J, Gonçalves H, Paiva L, Coelho L, Barros M, Samuel I, et al. O uso da estratégia gameficação na educação médica. Rev Bras Educ Med. 2019;43(1):147-56.
- 20 Persuh M, Genzer B, Melara RD. Iconic memory requires attention. Front Hum Neurosci. 2012;6:23354.
- 21 Ioannides AA, Popescu M, Otsuka A, Bezerianos A, Liu L. Magnetoencephalographic evidence of the interhemispheric asymmetry in echoic memory lifetime and its dependence on handedness and gender. Neuroimage. 2003;19(3):1061-75.
- 22 Morris CD, Bransford JD, Franks JJ. Levels of processing versus transfer appropriate processing. J Verbal Learning Verbal Behav. 1977;16(5):519-33.
- 23 Kirkpatrick DL. Techniques for evaluating training programs. Train Dev J. 1978.
- 24 Berbel NAN. As metodologias ativas e a promoção da autonomia de estudantes. Semin, Ciênc Soc Hum. 2011;32(1):25-40.
- 25 Behrens M. A prática pedagógica e o desafio do paradigma emergente. Revista Brasileira de Estudos Pedagógicos. 1999;80(196).
Source: Prepared by the authors.