Problems and potentialities of e-Learning for regular undergraduate courses in emergency medicine

Vantagens e desvantagens do e-Learning para cursos de emergência em medicina

RELATO DE EXPERIÊNCIA

Problems and potentialities of e-Learning for regular undergraduate courses in emergency medicine

Vantagens e desvantagens do e-Learning para cursos de emergência em medicina

William Rafaelo Schlinkert; Sandro Scarpelini; Antonio Pazin-Filho

Universidade de São Paulo, Ribeirão Preto, SP, Brasil

^{Mailing adress}

ABSTRACT

BACKGROUND: E-learning techniques are spreading at great speed in medicine, raising concerns about the impact of adopting them. Websites especially designed to host courses are becoming more common. There is a lack of evidence that these systems could enhance student knowledge acquisition.

GOAL: To evaluate the impact of using dedicated-website tools over cognition of medical students exposed to a first-aid course.

METHODS: Prospective study of 184 medical students exposed to a twenty-hour first-aid course. We generated a dedicated-website with several sections (lectures, additional reading material, video and multiple choice exercises). We constructed variables expressing the student's access to each section. The evaluation was composed of fifty multiple-choice tests, based on clinical problems. We used multiple linear regression to adjust for potential confounders.

RESULTS: There was no association of website intensity of exposure and the outcome - beta-coeficient 0.27 (95%CI - 0.454 - 1.004). These findings were not altered after adjustment for potential confounders - 0.165 (95%CI -0.628 - 0.960).

CONCLUSION: A dedicated website with passive and active capabilities for aiding in person learning had not shown association with a better outcome.

Keywords: Medical Education. Learning. Emergency Medicine.

RESUMO

INTRODUÇÃO: Websites especialmente contruídos para auxiliar cursos são comuns, mas não há dados sobre o impacto na aquisição de conhecimento.

OBJETIVO: Avaliar o impacto da utilização de ferramentas de um website exclusivamente-dedicado no aprendizado cognitivo de alunos de medicina em curso de primeiros socorros.

MÉTODOS: Estudo prospectivo de 184 estudantes de medicina que realizaram curso curricular de primeiros socorros (carga horária - 24horas). Um website foi construído para auxiliar o curso com várias seções (aulas, material de leitura adicional, vídeo e exercícios de múltipla escolha). O acesso a cada seção educativa foi documentado para cada aluno. A avaliação foi composta de cinquenta testes de múltipla escolha, baseados em problemas. Regressão linear múltipla foi utilizada para corrigir possíveis fatores de confusão.

RESULTADOS: Não houve associação entre a intensidade de do uso das seções do website e o desempenho na avaliação - beta-coeficiente 0.27 (95% CI - 0.454 - 1.004). Os achados não se alteraram após o ajuste - 0.165 (95%CI -0.628 - 0.960).

CONCLUSÃO: O use de um website para auxiliar o aprendizado em alunos de medicina não implicou em melhor desempenho.

Palavras chave: Educação Médica. Aprendizagem. Medicina de Emergência.

Introduction

Simulation and e-learning techniques are spreading at great speed in medicine, particularly in emergency medicine^1-3. The lack of case mix for a lot of emergency situations and ethical considerations about training in patients are strong promoters of these techniques already used in other areas, such as commercial aviation^4-6. In addition, traditional lectures are limited in transmitting knowledge, especially when "how to do" is of greater importance than "what to do"⁷.

Regardless of this pressure, there are some concerns about the impact of adopting these techniques². It is true that simulation-based courses for learning cardiac arrest protocols have already shown benefit in patients' survival and that many e-learning software is available^1,2,8-10. Nevertheless, the implementation of these techniques requires more commitment from the teaching staff, which could confound the association between exposure to the new method and the outcome of better knowledge acquisition and retention4. Courses developed to e-learning would be more structured than regular courses, providing better opportunities to learn.

Websites especially constructed to host courses are becoming more common and they can be adapted to the teacher's needs^11-14. They can archive lectures recordings, additional reading material, provide exercises for practice, enhance communication between teacher-students or among students and even provide evaluation resources. Some of them allow taking a course without being present in the classroom. Nevertheless, there is a lack of evidence that these systems could enhance student knowledge acquisition and retention.

The Faculty of Medicine of Ribeirao Preto of the University of Sao Paulo (FMRP-USP) is undergoing an undergraduate medical curriculum improvement and has recently implemented an axis for teaching emergency medicine along the whole medical course. The axis is designed to allow students to have an early contact with emergency medicine through a first aid course provided at the first year (four hours per week during five weeks). The second contact with the axis will be a simulation course in the fourth year (three full week's duration). Subsequently, they will have opportunity to practice in secondary open door emergency department during the fifth year (one full month) and a tertiary reference emergency department in the last year of the course (two months).

Even though these changes constituted an increase in the amount of time dedicated to emergency medicine, it is a challenge to the staff to teach a large amount of knowledge in the allocated time frame, especially for the first aid course. Implementing a a website for the course, including all available resources to increase the exposure of the first year students to emergency medicine would be a reasonable approach. In the previous version of the medical curriculum, the first aid course was traditionally provided to the second year students and with the undergoing curricular revision, it was transposed to the first year. During the time of this study, the staff had the opportunity to conduct the same course for two different populations - first and second year undergraduate medical students. We evaluated the use of the website and its association with course approval while adjusting for possible confounders.

We hypothesized that the intensity of using a dedicated-website would improve the cognitive knowledge acquisition. If this is true, the course could expand the opportunities for practice, delegating the cognitive work acquisition for out-of-class.

METHODS

We performed a prospective study enrolling 184 undergraduate medical students who should take a formal curricular first aid course. The study group was composed of 92 first year and 92 second year students. The school year in Brazil starts in January and all the students took the course in the second semester of 2008. The course had duration of five consecutive weeks, with four hours per week. During this time frame, the student participated in lectures and practice sections.

Participation on the study was voluntary and enrollment required a written consent. The study protocol was approved by the local ethical committee.

EXPOSURE - WEBSITE

We used the Teleduc web platform for generating a website for the course. The Teleduc is free of charge for educational purposes and it was also chosen for being easily handled by the teaching staff participating in the course. One of the authors had previous experience with Teleduc^11,15. The website was closed to general public and the students' access was granted after registration through a login and a password provided after registration in the course by one of the authors. Several students could have been exposed to this platform previously, since its use is increasing in our institution. We explained in the course's inaugural lecture that the platform could include several tools that could have not been available to them in previous courses for not being compatible with the learning objectives. We encouraged all students to use the plataform.

The website had several capabilities for interaction among the students, for correspondence with the teaching staff and to aid studying. All lectures given during the course were available in the website for download and there were also additional reading material and video. Multiple choice exercises for reviewing the lectures content were constructed. The teaching material was related to the course time schedule and was posted after each lecture to incentive regular access to the website. Every time a new educational material was available, an e-mail reminder was sent to students.

Figure 1

The software provided the staff with reports of individual student access to the website for each activity (global access, lecture download, additional material and exercises). We used these reports to characterize the students' access and constructed variables expressing the quartiles of number of times the student accessed each educational section. We considered that a student had used the capability if he was in the fourth quartile. Based on these variables, we constructed a derived variable to express the intensity of exposure to the e-learning process coded as 0 if the student didn't use the website during course, 1 if he has used only for downloading the lectures, 2 if he downloaded the lectures and additional materials and 3 if beyond that he had also taken the exercises provided. We also explored the raw access rate data in the models for multivariate analysis.

Outcome - Approval in the Course Evaluation

Even though the course included skill stations, the course evaluation take into consideration only the multiple choice test applied in the last day of the course. The evaluation was composed of fifty multiple choice tests that covered all the lectures given during the course. All questions were based on clinical problems, had four alternatives and only one of them was correct.

Confounders

Students' characteristics such as mean grade obtained and the number of subjects still pending in the previous semester were recorded. We also obtained data regarding previous experience with e-learning using a form applied in the beginning of the study. In the end of the study, we applied a new questionnaire to evaluate the students' experience with Teleduc and documented any problems they could have faced. Through this form, we also collected the time spent studying for the course and their evaluation of the didactic material, lecture and skill section.

Data Analysis

Categorical variables were expressed as percentages and continuous variables as mean and standard deviations. Fisher's exact or Student's t test were used for group comparison as applicable. Multiple linear regression was used to compare the outcome while adjusting for potential confounders. Incremental multiple linear models parting from a model including only the outcome and the exposure were constructed until the fully adjusted model which included year of medical training (0 - First year; 1 - Second year), last semester mean grade, previous reprobation (0 - no; 1 - yes), previous experience with e-learning (0 - none; 1 - at least one; 2 - more than one), time spent studying (0 - less than one; 1 - from one to five hours; 2 - more than five hours), problems using Teleduc (0 - no; 1 - yes), lecture quality (0 - non-satisfactory; 1 - satisfactory) and practice section quality (0 - non-satisfactory; 1 - satisfactory). Since year of medical training could be a strong confounder we decided to perform a sensitivity analysis, conducting the same models stratified by year of medical training. Statistical significance was considered as p value less than 0.05 for all tests. Data analysis was conducted with Intercool Stata 9¹⁶.

We also performed a qualitative analysis of the student's course evaluation using content summary techniques to identify potentials confounders not evaluated or potential implementations for further research. We used the same techniques of previous work from the authors¹⁷.

RESULTS

Greater intensity of using the website resources was associated in univariate analysis with medical school second year, previous experience with e-learning techniques, spending more time studying and attributing a satisfactory grade to the lectures (Table 1).

No association between exposure and outcome was present either in univariate or multivariate analysis (Table 2).

The student's evaluation provided interesting comments about the experience. First of all, they stated that the platform was very user-friendly and several of the capabilities used were not implemented in other courses-experiences previously taken. They considered very handy to be able to download the lectures in advance, so it was easier to have a general idea of the lectures content, to follow the lecturer reasoning and to take notes.

DISCUSSION

Increasing the amount of time dedicated to a subject besides in person participation during lectures or skill section activities is only one of the possible benefits that website courses can theoretically provide. Besides that, they can standardize the knowledge transmitted to several groups and increase the number of persons reached. They can also provide online immediate feedback^2,9,11-14. Using all the precedent arguments, they are being pressed over the teaching community by a generation of internet users and most of the time the teaching staff is not apt to deal with this new feature^18-20. In developing countries, this problem is magnified by the lack of support to teachers to implement the necessary changes. There are several available software for implementing websites and an ongoing debate of which one is the best¹⁸.

Producing material for these websites can be time consuming and using all the website resources requires training and time^1,11,21. This is viewed by some of the teaching community as a potential threat, since this will be in competition with time that should be employed for research. For those, there is lack of information regarding the benefits that this investment could bring to undergraduate students' performance.

In view of these considerations, the present study findings are very disappointing. Greater exposure to the educational material provided through the dedicated website was not associated with better outcome. The only feature that strongly correlated with the outcome was being in the second year group. We tried to overcome this very strong association including the variable for group (first or second year) in the multivariate linear regression that included all students and also stratifying according first or second year. Both strategies resulted in the same non association findings. Even though previous works have addressed the problem, most of the information was qualitative and did not measure exposure in the same way we performed, being difficult to compare^2,9,22,23.

This lack of association should be evaluated with caution in view of some possible limitations. First, we could not rule out that the students' that had not accessed the website did not have access to the downloaded material. They could have had access through colleagues that have accessed the website.

Second, our exposure was designed to express the intensity of using the website capabilities. For that, we used a combined variable based on the fourth quartile of the number of access to download lectures or the additional reading material and doing the exercises. We assumed that the greater the student used the website capabilities, the better he would perform in the final examination. The association between exposure and outcome could not be as linear as we supposed. We tried to overcome this using logistic regression for the same models, defining the outcome as the fourth quartile of the final evaluation grade (data not shown) without success. Nevertheless, we could not exclude other kind of relationship not addressed by the current strategy.

Finally, the outcome definition took into consideration only theoretical knowledge and it was designed to evaluate both the first and second year groups. This could partially explain the better outcome of the second year students, besides the accumulated knowledge of the medical curriculum. Perhaps, using a more demanding evaluation that included questioning issues only addressed in the website could be a strategy. Nevertheless, we considered that this strategy would penalize the students being ethically unacceptable. One could not forget that the grade obtained in course evaluation would be used for medical school curriculum and participation on the study was voluntary.

Another comment include the qualitative analysis of the course's evaluation. Even though this data is limited, since we only have 25% of descriptive answers to our questionnaire, it seems that we were successful in implementing reasonable tools of different kind, since the students considered the website user-friendly. The comparison with previous courses that have used the same platform should be considered very cautiously, but one possibility is that the resource is not being used to its full capability due to lack of teachers' training.

We should stress that our findings could not be generalized for exclusively website courses. In such cases, the teacher is not relating personally with the student as it was the case in our study. This teacher-student interaction is a strong helper for learning and probably could have played a role in our findings.

In summary, a dedicated website with passive and active capabilities for aiding in person learning had not shown association with a better outcome.

Mailing adress:

Antonio Pazin Filho

R. Bernardino de Campos, 1000

Centro - Ribeirão Preto - CEP 14015-100 SP

E-mail: [e-mail]

Recebido em: 11/06/2009

Reencaminhado em: 03/09/2009

Reencaminhado em: 04/12/2009

Reencaminhado em: 07/12/2009

Aprovado em: 22/11/2009

AUTHORS' CONTRIBUTIONS

The authors participated in the study conceptualization and design, data analysis and interpretation, and writing of the manuscript.

DISCLOSURE

William Rafaelo Schlinkert receiveid an educactional stypend from the Programa Ensinar com Pesquisa - 2008 of the University of São Paulo.

• 1. Carley S, Mackway-Jones K. Developing a virtual learning course in emergency medicine for F2 doctors. Emerg Med J. 2007;24(8):525-8.
• 2. Della CF, La Mura F, Petrino R. E-learning as educational tool in emergency and disaster medicine teaching. Minerva Anestesiol. 2005;71(5):181-95.
• 3. Smolle J, Prause G, Smolle-Juttner FM. Emergency treatment of chest trauma-an e-learning simulation model for undergraduate medical students. Eur J Cardiothorac Surg. 2007;32(4):644-7.
• 4. Pazin-Filho A, Scarpelini S. Simulação: definição. Medicina (Ribeirão Preto) 2007;40(2):162-6.
• 5. Rogers PL. Simulation in medical students' critical thinking. Crit Care Med. 2004;32(2 Suppl):S70-S71.
• 6. Ziv A, Wolpe PR, Small SD, Glick S. Simulation-based medical education: an ethical imperative. Acad Med. 2003;78(8):783-8.
• 7. Pazin-Filho A. Características do aprendizado do adulto. Medicina (Ribeirão Preto) 2007;40(1):7-16.
• 8. Cantillon P, Irish B, Sales D. Using computers for assessment in medicine. BMJ. 2004;329(7466):606-9.
• 9. Garcia Urena MA, Marin Gomez LM, Vega Ruiz V, Diaz Godoy A. Aplicación de las nuevas tecnologías en la enseñaza de la cirugía en la licenciatura de medicina. Cir Esp. 2009;85(3):165-70.
• 10. Moretti MA, Cesar LA, Nusbacher A, Kern KB, Timerman S, Ramires JA. Advanced cardiac life support training improves long-term survival from in-hospital cardiac arrest. Resuscitation. 2007;72(3):458-65.
• 11. Elias Jr J. Simulação em radiologia e diagnóstico por imagem. Medicina(Ribeirão Preto). 2007;40(2):192-8.
• 12. Giacomozzi C, Campobello G, Huijgens B, Ilsbroukx S, Macellari V, Magni R, et al. Tele-rehabilitation and e-learning: the HELLODOC educational experience. Ann Ist Super Sanita. 2008;44(2):145-53.
• 13. Novak IT. [An experience in the use of NTIC (New Technology for Information and Communication) in the teaching process of the immune system]. Rev Fac Cien Med Univ Nac Cordoba. 2005;62(3):63-7.
• 14. Tidia AE. Sobre o projeto TIDIA AE. São Paulo: FAPESP; 2008. [Acess in ??]. Available from: http://tidia-ae.incubadora.fapesp.br/portal/o-projeto
• 15. Pazin-Filho A, Romano MMD. Simulação: aspectos conceituais. Medicina (Ribeirão Preto) 2007;40(2):167-70.
• 16. Data Analysis and statistical Softwares. Stata Statistical Software [computer program]. Version 9.2. College Station, Texas: StataCorp; 2005.
• 17. Pazin-Filho A, Scarpelini S, Schmidt A. Análise qualitativa da elaboração e apresentação de aulas teóricas por alunos de pós-graduação da FMRP-USP. Medicina (Ribeirão Preto). 2007;40(1):51-62.
• 18. Muller M, Duperret S, Viale JP. Le e-learning en médecine : état des lieux et perspectives. Exemple d'un site Internet d'enseignement appliqué à l'échocardiographie en anesthésie, réanimation et urgences : www.echorea.org Ann Fr Anesth Reanim. 2008;27(10):832-9.
• 19. Shah IM, Walters MR, McKillop JH. Acute medicine teaching in an undergraduate medical curriculum: a blended learning approach. Emerg Med J. 2008;25(6):354-7.
• 20. Sparacia G, Cannizzaro F, D'Alessandro DM, D'Alessandro MP, Caruso G, Lagalla R. Initial experiences in radiology e-learning. Radiographics. 2007;27(2):573-81.
• 21. Almeida M. Educação à distância na internet: abordagens e contribuições dos ambientes digitais de aprendizagem. Educ Pesqui. 2003;29(2):327-40.
• 22. Costa-Santos C, Coutinho A, Cruz-Correia R, Ferreira A, Costa-Pereira A. E-learning at Porto Faculty of Medicine. A case study for the subject 'Introduction to Medicine'. Stud Health Technol Inform. 2007;129(Pt 2):1366-71.
• 23. Kukolja-Taradi S, Dogas Z, Dabic M, Drenjancevic P, I. Scaling-up undergraduate medical education: enabling virtual mobility by online elective courses. Croat Med J. 2008;49(3):344-51.

Publication Dates

• Publication in this collection
  06 Jan 2011
• Date of issue
  Sept 2010

History

• Received
  11 June 2009
• Accepted
  22 Nov 2009