Web-Based Undergraduate Medical Education in a Virtual Learning Environment Using an Original Pedagogical Approach : an Observational Longitudinal Study Educação médica de graduação via web em um ambiente virtual de aprendizagem utilizando uma abordagem pedagógica original : um estudo observacional 

43 (1) : 97-104; 2019 I Universidade Federal de Viçosa (UFV), Viçosa, Minas Gerais, Brasil. II Universidade Estadual de Campinas (Unicamp), Campinas, São Paulo, Brasil. Web-Based Undergraduate Medical Education in a Virtual Learning Environment Using an Original Pedagogical Approach: an Observational Longitudinal Study Educação médica de graduação via web em um ambiente virtual de aprendizagem utilizando uma abordagem pedagógica original: um estudo observacional longitudinal


Web-Based Undergraduate Medical Education in a Virtual Learning Environment Using an Original Pedagogical Approach: an Observational Longitudinal Study
Educação médica de graduação via web em um ambiente virtual de aprendizagem utilizando uma abordagem pedagógica original: um estudo observacional longitudinal Lucas Vilas Bôas Magalhães I Li Min Li II

ABSTRACT
Background: Asynchronous Web-based Medical Education in Virtual Learning Environments (VLEs) has grown steadily because of its many advantages.Various configurations and instructional methods are presently available.The existing proposals are poorly structured and/or not very effectively used for teaching diagnostic skills to undergraduate medical students in Brazil.A robust instructional method with positive pedagogical characteristics is needed.Thus, we have proposed a pedagogicallystructured method for VLEs that includes a motivating initial reading (Medical Chronicle -MC), and a knowledge building program, using real cases coupled with audiovisual resources (Diagnostic Workshop -DW).We aimed to verify its acceptance, as well as the efficacy of the MC/DW method in improving the diagnostic ability of medical students, in the long term.Methods: An opinion survey, two MC/DW materials and two Knowledge and Diagnostic Skills (KDS) questionnaires on stroke and epilepsy were developed, and two medical student groups were followed up in this 2013 longitudinal observational study.The students answered a KDS1, and attended a traditional lecture on one of the topics.They also accessed a VLE to apply the MC/DW method on stroke or epilepsy.We applied the same questionnaire (KDS2 and KDS3, respectively), one month and 5-6 months after the KDS1.We analyzed the mean KDS1 score of all the students, and the mean pairwise of those who accessed and those who did not access the VLEs during these three stages.

KEY-WORDS
-Education in Health. -Education.
.WBME is gaining increasing acceptance through activities and courses offered by Virtual Learning Environments (VLEs).These spaces help minimize problems in the classroom, such as material availability, since it is a form of self-directed and potentially interactive study carried out at different times and places, but focused on the same content and same skills 6,7 .However, WBME using VLEs is not a single entity, and various configurations are used, such as e-mail and multimedia 8 , and instructional methods, such as recorded lectures 9 , patient cases 10 , self-assessment, feedback, and interactive discussions 11 .

Contexto
How and when to use specific designs is something that must be learned 12 .Unfortunately, the existing proposals, including massive open online courses (MOOCs), are poorly structured and/or not very effectively used for teaching diagnostic skills to undergraduate medical students, especially in Brazil, being mostly applied in the basic sciences 3,13,14 .VLEs and MOOCs are often used only for classroom "virtualization", privileging technology instead of the pedagogical method 15 .But studying clinical cases is not sufficient; students must also be exposed to a robust instructional method, with contextualized, practical, and interactive experiences, feedback, and repetition 16 .In practice, even modern Web technologies, such as Web2.0 and Second Life, though potentially fit for collective knowledge construction, have played little role in medical education so far 17,18,19 .
We recently proposed an original pedagogical method for teaching of neurology in classroom medical education 20  We adapted our proposal for VLEs, taking into account the growth of WBME in VLEs and its advantages, pedagogical features of VLE learning and the lack of well-structured and effective-proven proposals for WBME under VLEs to improve undergraduates' diagnostic skills.The method was named Medical Chronic/Diagnostic Workshop (MC/DW), and is based on Charles Maguerez's Arch and on constructivist theories 21 .Its aim is to develop the clinical method, with emphasis on the medical interview and relevant physical examination, which account for up to 88% of diagnoses 22 .MC/DW is patient-centered, focused on solving virtual, contextualized and interactive real cases, with feedback, repetition and analysis 23 .
It involves a motivating initial reading (MC), and a knowledge-building structured program (DW).We present the Medical Chronicle (MC) as a simple language narrative.We developed a reflection-inducing, reliable chronological record of events, with a critique by the author, inspired by real-life events in the daily life of the medical services 24 .Several critical incidents are included in the MCs, to open an agenda of topics to be learned.These topics should be those that general practitioners need to know.The MC therefore acts as a learning motivating tool, applicable to VLE 25 .The second tool used in the method is a Diagnostic Workshop (DW), focusing on topics defined by the agenda.It consists of digitized real medical cases, presented in ascending order of complexity, in a trans-disciplinary mode, allowing a gradual and cumulative, mainly diagnostic, construction of knowledge.Each question contains one or more virtual "stations" (e.g.images, scientific article excerpts, videos, and complementary exams) for its resolution.We do give frequent summaries and we approach diagnostic heuristics.We allow classroom or virtual interaction among the participants.Feedback (with post-references) is provided at the end of the DW, always seeking the best available scientific evidence.We encourage the students to return to the questions/"stations" several times, thus promoting repetitive study and respect for individual learning time, an aspect that is severely neglected in class.We respect copyright law and restrict VLE access.All these characteristics make DW an original tool with many differential characteristics for use in VLEs 20 .
The questions that motivated this research were: Would students access this method in the VLE?How would they subjectively analyze our proposal?and Would the MC/DW method be effective in improving the diagnostic skills of the medical students in a long term, if applied after a traditional lecture class?This work aimed to verify the acceptance of our proposal by the students, as well as the efficacy of the VLE MC/DW method in improving the diagnostic ability of medical students in the long term.

Elaboration of the research material
We developed two MC/DW materials, one on stroke and the other on epilepsy, both prevalent topics that need to be mastered by general practitioners.These topics belong to the authors' area of specialization (neurology).Both materials were inserted into a VLE -Moodle (modular object-oriented dynamic learning environment) called PVANet at the Universidade Federal de Viçosa (UFV/MG, Brazil).We developed an opinion survey and two Knowledge and Diagnostic Skills (KDS) questionnaires, one on stroke and the other on epilepsy.The opinion survey contained alternative responses using numerical or Likert scales, aiming to gather the student's subjective reactions.The KDS questions assessed not only the students' theoretical knowledge but also, in particular, their "know-how" (diagnosis), as well as their attitudes and practices.The KDS and opinion survey scales were constructed based on Bunchaft & Cavas 26 .The final stroke questionnaire consisted of ten open-ended questions, while the epilepsy questionnaire contained seven open-ended and three multiple-choice questions.Half of the questions were devised by an experienced neurologist, rather than a researcher.The open-ended questions (e.g., "What is the diagnosis?"),required objective answers.

Ethics approval and consent to participate
This research work was approved by the Research Ethics Committee at Unicamp and by the UFV (Ruling no.468/2010, CAAE 0358.0.146.000-10).The participants signed a consent form.

Application and pre-experimental results
Three medical doctors connected to the university (a general practitioner, an internist, and a neurologist) initially evaluated the two MC/DW materials and the KDS questionnaires.The medical researcher asked experts to analyze the content, which was found to be comprehensive and cumulative, but too extensive, and with navigation difficulties (the experts therefore suggested it be reduced and divided into topics to make it easier to handle).The teaching method was considered educational, thought-provoking, and trans-disciplinary, stimulating preliminary study and highlighting the clinical method.The professors considered KDSs (evaluation methods of the proposed MC/DW) to be adequate, presenting practical and routine situations to the general practitioner.The DWs on stroke and epilepsy contained 15 and 26 questions, respectively, some with sub-items.Brazil) accessed the VLE and rated the MC/DW sets and the opinion survey.Twenty-nine students (90.6%) considered the MCs at least good.The DWs' feedbacks, with post-references, were found to be sufficient to clear up the doubts of 26 students (81.2%).All students (100%) reported that the feedback complemented learning and prompted a search for new information, providing a few suggestions to improve the material.After adjustments, the materials and KDS/opinion surveys were found to be suitable for application in a longitudinal observational study.

Design and participants
A longitudinal observational study involving two groups of undergraduate medical students at the UFV during 2013 (2 nd and 3 rd year students, familiar with basic science and clinical examination, were selected).The research was conducted using the curriculum course.

Interventions
We applied the KDS (KDS1) stroke questionnaire to the 2 nd year medical students and the epilepsy questionnaire to the 3 rd year medical students.We further gave a traditional lecture on stroke or epilepsy.We then invited all the students to access the VLE and to apply the MC/DW method.The students had 30 days to access the VLE, being allowed to access it as many times and for as long as they found necessary, outside the classroom.The researcher in charge controlled access to the VLEs and answered questions by mail, sending a prompt to those who had not yet entered the system.All the students were invited to respond, without previous notice, to the same KDS (KDS2) and the opinion survey, 30 days after KDS1, and the same KDS (KDS3) five to six months after KDS1.Throughout the course, there was no formal exposure to these topics between KDS1 and KDS3.The researcher attributed the scores to KDS1, 2, and 3 questionnaires using a checklist.Each KDS was worth ten points (1 point for each correct answer, with partially correct answers being accepted).We must emphasize that the KDS questions were not included in the VLE or discussed in the classroom.

Statistical analysis
We tabulated the scores of the KDS1, 2, and 3 questionnaires in Microsoft Excel 2007 as a single set (stroke KDS + epilepsy KDS) with paired data.We used the software Statistical Package for the Social Sciences -SPSS (version 20) (SPSS Inc., USA), performing an initial descriptive analysis of the sample, and obtaining the mean and standard deviation in the KDS1 of the students who completed the study.Next, we calculated the means of those who accessed and did not access the VLE, during those three instances.We conducted a descriptive analysis to characterize the samples, after the classification made by the students, and applied the Anova statistics for repeated measures, to verify whether there was a significant difference between the scores of the KDS1, KDS2 and KDS3 survey questionnaires.The assumptions for performing Anova were met (N≥30; allowing to apply the Central Limit Theorem; equal variances in the different groups; observations made by each group independent from one another; and to measure the variable under study at an interval scale).Once the differences in KDS1, 2, and 3 had been confirmed, we applied Bonferroni post hoc analysis to verify whether there was a significant difference between the scores of those who accessed and those who did not access the VLE.We analyzed the opinion survey data tabulated in Microsoft Excel 2007 using descriptive statistics tools.

Analysis of the KDS questionnaires
Ninety-two 2nd and 3rd year medical students were invited to participate in this 2013 study; 87 students participated, but six were excluded because they did not answer all the questionnaires.Thus, we evaluated data obtained from questionnaires KDS1, 2, 3 and opinion survey of 81 students, 41 males (50.1%), mean age of 21.9 years (19-34, Standard Deviation or SD 2.41).We obtained the mean of the 81 participants in KDS1, 1.59 points (SD 0.71).We confirmed that 66 students (81.5%) accessed the VLEs and 15 students (18.5%) did not; we sorted the responses, separated them, and obtained the means and standard deviations, as summarized in Table 1.The reasons why 15 students did not access the VLE were not verified.We used Anova for repeated measures and verified a significant difference between the scores of the students obtained in the three KDS questionnaires [F (1.76; 139.33) = 72.76,p<0.01].We found that, statistically, there was an interaction 101 Lucas Vilas Bôas Magalhães et al.  http://dx.doi.org/10.1590/1981-52712015v43n1RB20180039ING between the scores obtained from the questionnaires applied and access to VLE [F (1.76; 139.33) = 48.43,p<0.01].We then used the Bonferroni post hoc analysis, showing no significant difference in the KDS1 grades between those who accessed and did not access the VLE (t = 1.48, p>0.05).However, the students who accessed the VLE had higher scores in KDS2 (t = 8.88, p<0.05) and in KDS3 (t = 8.39, p<0.05).These results indicate that access to the VLE had a direct influence on the KDS2 and KDS3 grades (Figure 1).Of the 66 participants who accessed the VLE, 34 (51.5%) were female.We found that 37 students (56.1%) completed the entire activity (reading the MC and at least one primary reference, answering all the DW questions and checklist), with 62 students (94%) considering the MC at least good.We verified that 62 students (94%) checked their answers; 52 (78.8%) considered that it was sufficient to clear up all their doubts; and 57 students (86.4%) considered it necessary to hold a final collective discussion on the subject.

DISCUSSION
We aimed to verify the acceptance of our proposal by the students, as well as the efficacy of the VLE MC/DW method in improving the diagnostic ability of medical students in the long term.The vast majority (81.5%) of the students accessed the VLE MC/DW method.The MC was considered at least good by 62 students (94%); 52 (78.8%) considered the checklist sufficient to clear up all their doubts; and 57 students (86.4%) considered it necessary to hold a final collective discussion on the subject.The results of the KDS questionnaire showed that those who accessed the VLE had a mean score of 4.57 out of 10, while those who did not access it had a mean score of 1.57 out of 10, after 5-6 months of access.These data corroborate the assumption that the MC/DW method is feasible, even if performed in isolation in subjects or modules 27 , stimulating students to study before the activity.It was well accepted, with the high participation rate indicating that students showed great interest, although the Hawthorne effect may have played a role 28 .The answer key sheet cleared up any doubts, providing considerable individual learning, but most of the students expressed the need for a final collective discussion, to clear up any remaining doubts and reinforce the learning.We understand this would be a very good discussion, but only after individual doubts have been cleared up, through e-mails or chat.The MC/DW method led to a statistically significant improvement in the ability to diagnose routine neurological problems in the long term.This improvement in performance may be explained by the fact that MC/DW is patient-centered method, focused on solving virtual, contextualized and interactive real cases.Feedback and repetition promote greater motivational and critical thinking, and emphasize differential diagnosis of the so-called minor symptoms (nonspecific symptoms, a real epidemic of the 21 st Century) 29,30,31 .According to Kirkpatrick and Kirkpatrick 32 , training systems evaluation takes place at four levels: reaction (satisfaction), learning, behavioral change, and results.The evaluation of the last two stages is only possible with a consolidated course.Thus, we evaluated the reactions of students exposed to the MC/DW method through the opinion survey, and long-term learning through analysis of questionnaires KDS1, 2, and 3.The real change in behavior and outcomes in patients is rarely evaluated in undergraduate research.It is unlikely that the students sought answers for the KDS, since they had no access to the questions and answers, and were not told they would be taking tests.Our study did not aim to compare our method with other WBME methods, since our proposal is original and existing methods (WBME and faceto-face) are not mutually exclusive 33,34 .A longer exposure time to any methodology can produce a positive effect 35 , but we showed that long-term improvement in diagnostic skills was better for those who accessed the VLE, with exposure time to classes having little influence on the results.
.However, it seems that a strong instructional method, like MC/DW, minimizes these effects 40 .Furthermore, MC helps identify medical heuristics (processes through which medical reasoning is made), especially diagnostic heuristics, thus attempting to minimize reasoning biases by the students, since not every medical decision is based on scientific data, not always available 41 .An updated answer-key sheet is the only task the professor must provide in the long-term to maintain the MC/DW method.We found this to be a positive point, as it contributes to continuing medical education (with one of the greatest challenges for many doctors being to find the best evidence at the appropriate time) 42 .
We recognize that some topics may seem to be better taught through regular teaching models.The MC/DW method does not replace face time learning, such as patient assessments, internships, and skills labs 43 .We carried out an observational (not randomized) study in two separate curriculum subjects at the same college.Further multi-centric randomized studies are necessary to compare the MC/DW method with other proposals structured for WBME in VLEs to measure the actual effect size for improved diagnostic capability in the long term.

CONCLUSIONS
The MC/DW method for WBME in VLE was found to be effective in improving diagnostic skills of undergraduate medical students in the long term.It was well accepted and positively evaluated by the students, and applied to isolated subjects.Since it presents several pedagogical characteristics needed for greater VLE learning, the MC/DW method seems to be a good choice of instructional structured method by any professor or researcher who applies VLE.The description of the proposal allows its replication.

Figure 1
Figure 1 Mean of the KDS1 scores of all students and of KDS2 and 3 of those who accessed and did not access the VLE, confidence interval of 95%

Table 1 Mean and standard deviations of the scores of the students who accessed and who did not access the VLE
The use of MC/DW can foster the development of minimum competencies generated 102 Lucas Vilas Bôas Magalhães et al. http://dx.doi.org/10.1590/1981-52712015v43n1RB20180039INGfrom an exponentially growing knowledge and scarce medical course time.It is still uncertain whether individual learning styles influence WBME learning