Virtual learning object about oral ulcerative lesions: controlled educational intervention study

Abstract

The aim of this study was to develop a virtual learning object (VLO) to teach undergraduate dental students about the diagnostic and therapeutic approaches to oral ulcerative lesions. The VLO was developed with information on the diagnostic process, lesion classification, and clinical-surgical management of oral ulcerative lesions. The VLO content was initially validated by a group of specialists. Learning was evaluated in a sample of 58 undergraduate dental students, divided into control group (conventional theoretical class, n = 29) and intervention group (interaction with VLO, n = 29). All students answered a pre-test and post-test questionnaire. The VLO group also answered a specific questionnaire on the evaluation of the VLO. Both quantitative and qualitative descriptive analyses were performed. The validation showed that professors and students considered the VLO adequate. The use of the VLO was recommended by 100% of professors and 86.6% of students. In the intervention group, the results showed a significantly higher number of correct answers in the post-test (p < 0.01). In conclusion, the VLO proved to be a useful tool for teaching oral medicine, contributing significantly to the knowledge of ulcerated lesions in the mouth.

Teaching; Educational Measurement; Oral Medicine; Telemedicine

Introduction

Ongoing technological development is driving the emergence of new alternatives in education that make the teaching-and-learning process more enjoyable and innovative for students while promoting their effective participation.¹,² In this context, virtual learning objects (VLOs) are defined as digital resources that support learning.³ The main goal of VLOs is to “break down” the disciplinary educational content into small pieces that can be reused in different environments, making the learning flexible, participatory, and effective, ultimately promoting independent and autonomous learning.⁴,⁵ Numerous technological and operational characteristics can be found in VLOs, such as: reuse, usability, interoperability, recoverability, flexibility, accessibility, durability, and autonomy.⁵ VLO consist of electronic texts and animations (videos, sounds, figures, and photos), and are important didactic tools to help students who struggle with abstract concepts, as they produce a playful environment for developing educational content, while stimulating cognitive processes such as perception, memory, and language.¹,⁵

Several studies have discussed the benefits of using VLOs in the teaching-learning process considering student knowledge gain, which is noted through the impact on learning averages.¹-³ Teaching methods must stimulate student initiative and responsibility for meaningful learning to take place. Educator engagement that is creative, motivating, challenging, conscious, and critical is also essential.⁶,⁷ Virtual objects must replicate real-world situations to contextualize knowledge and stimulate and motivate students to solve problems. The interaction and liveliness of virtual objects allows people to perceive the virtual experience as a real experience, making learning more effective.⁸ VLOs have proven to be effective tools, as they consist of a particular form of learning in which knowledge must go beyond simple memorization of points and problems to assimilation of a range of diversified and in-depth knowledge to solve real-world problems.⁶ As far as we know, there are no previous studies on the effectiveness of VLOs in learning diagnostic and therapeutic skills in oral ulcerative lesions.

In the literature, there are few studies on health-related VLOs. VLO in health subjects are still recent and thus must be further explored, given their great potential to contribute to the academic formation of health professionals. Providing education that promotes decision-making skills and imparts both knowledge and expertise is a challenge to health education institutions.¹,²,⁹-¹³ The primary aim of this study was to develop, apply, and evaluate a VLO for teaching diagnostic and therapeutic knowledge of ulcerative oral lesions. The secondary aim was to assess the characteristics and contributions of this new instrument in teaching and its importance as an educational tool.

Methodology

This study was approved by the Dentistry Research Commission (COMPESQ-ODO) and the Ethics Committee on Human Research (CEP) of the Federal University of Rio Grande do Sul (UFRGS) with the report no. 2.546.538. The methodology used was based on studies previously published in the literature.¹,⁷

This controlled educational intervention study consisted of 4 main steps: a1) VLO development, b) Validation of the VLO by experts, c) Assessment of learning, and d) Assessment of the VLO by students (Figure 1).

Figure 1
Study flowchart.

VLO development

Initially, a “story-board” of the VLO was developed with the following contents: introduction, acute ulcers (traumatic ulcer, acute herpetic gingivostomatitis, mucositis, erythema multiforme, and primary syphilis), and chronic ulcers (recurrent herpes, recurrent aphthous stomatitis, pemphigus vulgaris, cicatricial pemphigoid, lichen planus, paracoccidioidomycosis, tuberculosis, secondary syphilis, squamous cell carcinoma, lymphomas, sarcomas, and malignant salivary gland neoplasms). The lesson plan addressed, for each lesion, disease history, clinical aspects, diagnostic process, and appropriate clinical-surgical management. Avatars of a professor and a student were developed to simulate the care of patients within each lesion. The development of diagnostic reasoning was based on simulated clinical cases. The VLO was developed using three programs: Microsoft Word (version 19, Microsoft, Redmond Washington, USA), Adobe Photoshop (version 21.1.0, Adobe Inc, San Jose, USA), and Articulate Storyline (Articulate Global LLC, New York City, USA), with the collaboration of the specialized team of the TelessaúdeRS-UFRGS and of the Social Dentistry Research Center (Faculty of Dentistry-UFRGS).

Expert validation

After the development phase, the VLO was submitted to assessment by oral medicine professors to verify the relevance of the contents of the created instrument. Schools from three public universities across the country were invited to participate. All of the selected universities had graduate programs in oral pathology and oral medicine ranked as excellent by the Coordination for the Improvement of Higher Education Personnel (a foundation affiliated with the Brazilian Ministry of Education). One expert from each university with at least 5 years of teaching experience in oral medicine participated in the validation process. After using the VLO, the professors who agreed to take part answered an assessment questionnaire with questions related to educational and technical aspects of the VLO. For each question, a score from 1 to 5 was assigned: (1) Very adequate; (2) Considerably adequate; (3) Very little adequate; (4) Somewhat adequate; (5) Not adequate at all.⁷The weighted average of the above scores was calculated, in which 1.00 was assigned for score 1, 0.75 for score 2, 0.50 for score 3, 0.25 for score 4, and 0 for score 5. Items with a weighted mean score greater than or equal to 0.80 were classified as appropriate, those with a weighted mean between 0.80 and 0.50 as moderately appropriate, and those with weighted mean less than or equal to 0.50 as not appropriate.⁷

The professors also validated the questionnaire (pre- and post-test) using a 6-item scale with scores ranging from one to three: a) Adequate, b) Moderately adequate, and c) Inadequate. At the end of the questionnaire, respondents had the opportunity to make suggestions and recommendations for possible improvements to the VLO. Adjustments were made when the experts considered an item to be inadequate. Questions assessed as moderately adequate were reassessed by the team and modified if they caused comprehension problems.

Assessment of learning

All undergraduate students from the 4^th and 10^th semesters (initial and final stages) at the Faculty of Dentistry of the UFRGS were invited to participate in the study. The semesters were selected based on students having recently had oral pathology class on oral ulcers in the 4^th semester and having completed all required courses and doing in-service training prior to graduation in the 10^th semester. All participants signed an informed consent form and were divided into two groups (control group and intervention group). The control group consisted of students who attended a 1h 10 min traditional lecture with a multimedia resource with PowerPoint slides, delivered by an oral medicine professor of the Faculty of Dentistry. Before and after the lecture, students answered similar pre- and post-tests. The test consisted of questions covering all the proposed content, containing 10 multiple-choice questions and 10 true-or-false questions. Among these, 20% were rated as easy, 20% as difficult, and 60% as intermediately difficult. The class content contained the same script, objectives, and clinical photos as the VLO. The intervention group consisted of students who received the class content via the VLO. Likewise, the students answered the pre- and post-test questionnaires that were similar to those answered by the control students.

The statistical analyses were conducted with SPSS for Windows, version 21.1 (Statistical Package for the Social Sciences, IBM, New York, USA). Initial descriptive data were analyzed considering all quantitative (mean and standard deviation) and qualitative (frequencies and percentages) variables measured in the study. The data distribution pattern was established, which indicated that non-parametric tests should be used in subsequent analyses to assess the association between independent variables and outcomes. Pre- and post-test scores was compared within and between groups, and the level of significance was set at 5% (13.4 versus 15.5, Wilcoxon test).

Assessment of VLO by students

Dental students assessed the VLO attributes using the same questionnaire that was answered by the oral medicine professors. In addition, two questions on the questionnaire referred to self-assessment of knowledge on ulcerative lesions in the oral cavity before and after using VLO, using a 5-point scale: (1) very limited; (2) limited; (3) substantial; (4) very substantial; (5) extensive. For result comparisons, categories 1 – 2 and 3 – 4 were grouped.

Results

Development of the VLO

The VLO consisted of a main screen with icons presenting the following content: introduction icons (7 min), acute ulcers (16 min), chronic ulcers part 1 (17 min), part 2 (10 min), and part 3 (18 min), and case simulation, totalizing 1h08min (Figure 2). The VLO is hosted at https://360.articulate.com/review/content/3a9aad68-b6dd-4cad-b646-61788dd1de63/review.

Figure 2
Layout of the VLO on oral ulcers: A) Initial VLO screen, containing recommendations on how to use it; B) Screen detailing how the content is presented, through audio, image, and text videos, with interactions between professor and student on the simulated case. C) Screen showing the clinical examination of a case. D) Virtual case simulation containing clinical data and questions about the case afterwards.

Expert validation

Three oral medicine professors agreed to take part in the study by evaluating the lecture instrument, 2 males and 1 female. All had Ph.D. in oral medicine and experience teaching the subject in Brazilian universities. The average age of the professors was 43 years (± 7.21), ranging from 37 to 51 years. The mean time of professional training in oral medicine was 19.6 years (± 9.29), with a minimum of 12 years and a maximum of 30 years.

Table 1 describes the results of the validation of the VLO by the professors using weighted mean analysis. In the analysis, of the 21 questions that composed the validation questionnaire, 18 (85.7%) were classified as adequate (weighted average equal to or above 0.80) and only 3 (14.3%) questions as moderately adequate (weighted average between 0.50 and below 0.80). The analysis of the overall weighted average revealed that the VLO was considered adequate by all experts, with a mean value of 0.90. Within the validation questionnaire, two questions (number 22 and 23 – yes / no categories) assessed the use of the VLO: all 3 professors answered they recommended the use of the virtual environment as a teaching strategy in undergraduate dental education and would be willing to use other VLOs.

Table 1
Analysis of the virtual learning object (VLO) by oral medicine professors and dental students.

Assessed items	Weighted average faculties	Weighted average students
	(n = 3)	(n = 29)
Interaction and stimulation
1. Ease of navigation, quality of resources and user help	1.00	0.87
2. The virtual environment proposes learning situations	0.91	0.92
3. The activities are relevant and address the proposed objectives	0.83	0.93
4. The access to the modules is easy	1.00	0.91
5. VLO prompts action and arouses the interest of students	0.91	0.80
6. Design of visual and audible information for enhanced learning and efficient mental processing	0.75	0.77
7. Ability to study using the Virtual Learning Object (VLO)	0.91	0.89
Dedication, discipline and time management
8. Prompts the search for other information in different research sources	0.75	0.86
9. Time organization for the VLO activities	0.91	0.83
10. Self-discipline for online activities	0.66	0.83
Communications tools
11. The environment stimulates information exchange between faculties and students	0.83	0.81
12. The environment provides feedback to the learner during the interaction	1.00	0.84
13. The links provided are relevant to learning content	1.00	0.93
14. Use of a method to clarify doubts about the content	0.91	0.81
Educational material
15. Veracity, accuracy and balanced presentation of ideas	0.83	0.90
16. VLO is explanatory and easy to understand	1.00	0.93
17. The information is presented in a logical and coherent manner	0.91	0.93
18. The writing style is easy to understand	1.00	0.93
19. It has alternatives for presenting information such as reading, animations and videos	0.91	0.88
20. The media are correlated with the content and complement the texts	0.83	0.90
21. Possibility to use in different learning contexts and with students from different backgrounds	1.00	0.88
Total	0.90	0.87

Regarding the assessment of pre- and post-test questionnaires by oral medicine professors, two participants rated all 6 questions (100%) as adequate and one participant rated 5 questions (83.3%) as adequate and 1 question (16.7%) as moderately adequate, suggesting grammatical/content corrections to the questionnaire. This suggestion was taken up and the changes were made before the questionnaire was applied to the students.

All suggestions provided by the experts were assessed and, when necessary, implemented in the final version.

Assessment of learning

A total of 58 students participated in the study, 24.1% were male (n = 14) and 75.9% were female (n = 44). Of these, 50% (n = 29) were assigned to the control group and 50% (n = 29) to the VLO group. Most of the students were from the 10th semester (60.3%, n = 35). Table 2 shows the participants’ performance before and after the two teaching approaches and the semester in which they were enrolled. The between-group comparison revealed that the VLO group had a significant increase in the mean number of correct answers (p < 0.01), which was not observed in the group that received a traditional, in-class lecture (Table 2).

Table 2
Assessment of dental students’ knowledge pre- and post-intervention tests, by group and semester.

Variable	Pre-test	Post-test	Within group p value**
	Mean (SD)	Mean (SD)
4 th semester (n = 23)
Control group (n = 10)	13.0 (2.3)	14.7 (2.94)	0.08
VLO (n = 13)	12.8 (2.0)	15.38 (1.5)	< 0.01
p-value* (between groups)	0.78	0.92
10th semester (n = 35)
Control group (n = 19)	14.78 (1.75)	14.84 (2.26)	0.85
VLO (n = 16)	13.87 (1.74)	15.68 (1.7)	< 0.01
p-value* (between groups)	0.11	0.29
All (n = 58)
Control group (n = 29)	14.8 (2.1)	14.8 (2.5)	0.14
VLO (n = 29)	13.4 (1.9)	15.0 (1.6)	< 0.01
p-value* (between groups)	0.11	0.34

• *Mann-Whitney; ** Wilcoxon

Table 3 shows the students’ performances in each question. The conventional class group showed a significant increase of correct answers in 2 questions (10%). In the VLO group, 9 questions (45%) had an increase in the number of correct answers in the post-test. In both groups, 1 question (5%) had a significant decrease in the number of correct answers after the educational intervention.

Table 3
Performance (percentage of correct answers) in each question, pre- and post-test by group.

Control group				VLO group
Question	Pre test n (%)	Post-test n (%)	p-value*	Question	Pre test n (%)	Post-test n (%)	p-value*
Q1	22 (75.9)	26 (89.7)	0.10	Q1	25 (86.2)	28 (96.6)	0.08
Q2	28 (96.6)	29 (100)	0.32	Q2	27 (93.1)	28 (96.6)	0.32
Q3	27 (93.1)	28 (96.6)	0.32	Q3	20 (69.0)	28 (96.6)	0.01
Q4	9 (31.0)	4 (13.8)	0.03	Q4	3 (10.3)	8 (27.5)	0.03
Q5	27 (93.1)	27 (93.1)	1.00	Q5	25 (86.2)	22 (75.9)	0.18
Q6	22 (75.9)	24 (82.8)	0.48	Q6	21 (72.4)	27 (93.1)	0.03
Q7	23 (79.3)	25 (86.2)	0.41	Q7	22 (75.9)	26 (89.7)	0.10
Q8	21 (72.4)	23 (79.3)	0.32	Q8	19 (65.5)	27 (93.1)	< 0.01
Q9	14 (48.3)	15 (51.7)	0.71	Q9	16 (55.2)	18 (62.1)	0.16
Q10	20 (69.0)	25 (86.2)	0.03	Q10	18 (62.1)	25 (86.2)	0.04
Q11	27 (93.1)	26 (89.7)	0.32	Q11	28 (96.6)	27 (93.1)	0.32
Q12	28 (96.6)	23 (79.3)	0.06	Q12	26 (89.7)	26 (89.7)	1.00
Q13	17 (58.6)	26 (89.7)	0.01	Q13	19 (65.5)	29 (100)	< 0.01
Q14	29 (100)	29 (100)	1.00	Q14	27 (93.1)	29 (100)	0.16
Q15	24 (82.8)	26 (89.7)	0.32	Q15	24 (82.8)	24 (82.8)	1.00
Q16	25 (86.2)	26 (89.7)	0.56	Q16	29 (100)	28 (96.6)	0.32
Q17	10 (34.5)	8 (27.6)	0.41	Q17	6 (20.7)	5 (17.2)	0.66
Q18	13 (44.8)	11 (37.9)	0.56	Q18	12 (41.4)	5 (17.2)	0.04
Q19	4 (13.8)	5 (17.2)	0.71	Q19	5 (17.2)	14 (48.3)	< 0.01
Q20	21 (72.4)	23 (79.3)	0.48	Q20	17 (58.6)	27 (93.1)	< 0.01

• *Wilcoxon test

Assessment of the VLO by the students

The analysis of the VLO by the students is shown in Table 1. The results indicate that the students considered the VLO adequate, since of the 21 questions, 20 (95.2%) were considered adequate (weighted average equal to or greater than 0.80). Only 1 question (4.8%) was considered moderately adequate (average between 0.50 and below 0.80) by students.

Of the 29 students who took part in the VLO group, 26 responded to questions about the use of the Oral Ulcers VLO in undergraduate dental education and whether they would be willing to use other types of VLOs. The results showed that 22 participants (86.6%) recommend the use of the Oral Ulcers VLO and 24 participants (92.3%) would be willing to use other VLOs.

The results concerning self-assessment of knowledge on ulcerative oral lesions before and after using the VLO are shown in Table 4. There was a significant difference in frequencies (p = 0.02), demonstrating a knowledge increase after the use of the VLO compared to before, mainly associated to more students considering their knowledge extensive.

Table 4
Self-assessment of knowledge about ulcerated lesions in the mouth before and after the application of the VLO.

Variable	Pre-VLO n (%)	Post-VLO n (%)	p-value *
Limited	5 (17.2)	1 (3.4)	0.02
Substantial	21 (74.4)	20 (68.9)
Extensive	0 (0.0)	5 (17.2)
No answer	3 (10.3)	3 (10.3)
Total	29 (100)	29 (100)

• *Chi-square test (test applied among valid answers)

Discussion

The increasing use of technology in daily life brings the need for innovation in teaching and learning processes. Consequently, VLOs that can recreate real-life situations and stimulate cognitive processes, such as perception, memory, and language, are decisive tools in this process.¹,³,⁵ In the present study, we developed the first VLO applied to oral medicine on ulcerative lesions. The results reveal that the use of the VLO has promoted a significant increase in student knowledge compared with a traditional lecture. In addition, it resulted in an increase in self-perceived knowledge about the topic.

Ideally, VLOs should go through a validation stage after their development, as this is central for the identification of problems. This allows improvements to be made that have a positive impact on success.¹⁴,¹⁵ In this study, the content developed was validated by three oral medicine professors before being used as an educational tool for dental students. All recommendations were evaluated and most of them were implemented. Importantly, the professors evaluated the VLO as appropriate for use in oral medicine teaching. The participating professors were carefully selected for the validation step based on their years of clinical and research experience, international recognition in oral medicine, and the excellence of their institutions according to the official Brazilian educational bodies. Yet, this process might have been biased as there was no sample size calculation for the validation step, which might be a deficiency in this step.

The results of this educational intervention study showed a significant increase in knowledge of students in the 4^th and 10^th semesters after using the VLO. Therefore, the VLO had a positive impact on student knowledge both in the early stages and in the final year of study. It should be noted that the students in the control group and the VLO group did not show any difference in the level of knowledge in the pre-test assessment. This result is important to show the homogeneity of the groups at baseline.

The control group had a mean increase in correct answers in 2 questions in the post-test, while the VLO group had a mean increase in correct answers in 9 questions, with a significantly positive impact on learning. It can be inferred that the VLO is a good tool for learning the diagnosis and therapy of mouth ulcerative lesions. In the teaching of oral medicine, especially on the subject of ulcerative lesions, no studies evaluating the use of VLOs were found. Our results corroborate the findings of other studies that used VLOs in teaching and learning in other subjects, revealing that students’ knowledge improved after the intervention.¹-⁴,⁹,¹⁰,¹⁵,¹⁶

A study that addressed the use of VLO for teaching radiographic cephalometry in dental education, also with a control group, showed that students in the VLO group achieved better results, with greater knowledge consolidation and better performance in identifying radiographic landmarks [10]. Recently, another study was conducted to develop an interactive VLO with videos, images, stories, and virtual simulation in dental materials to improve theoretical knowledge and laboratory skills of dental students.¹ Other reports also emphasize the positive effect of using virtual media such as games, VLOs, and apps in increasing learning in several fields.¹¹-¹³,¹⁵,¹⁶

Another important finding of the present study is the assessment of the VLO by students. This step has also been highly recommended, since poorly planned multi-media materials have been associated with student disorganization and demotivation.⁵ Our results showed that students considered the VLO appropriate, with mean score above 0.80 of a possible maximum of 1. Also, most students recommended the use of VLO in undergraduate education. The VLO developed in this study enabled the simulation of real-life scenarios (clinical cases), thus enhancing the contextualization of knowledge, stimulating and motivating students to solve problems. In addition to significantly increasing the mean post-test scores, the VLO also improved students’ self-perception of knowledge.

VLOs encourage a more active student engagement by generating a playful environment for the development of content that stimulates cognitive processes such as perception, memory, and language.³ The role that technology plays in optimizing collective knowledge production is clear, as it makes the teaching-learning process more innovative for students, favoring effective participation.²-⁵,¹⁴ Our results suggest that new teaching-learning tools must follow the changes in the profile of students who have a preference for differentiated methodologies, who use the internet, and who show interest and knowledge in using virtual objects.¹⁵

One of the limitations of this study is the use of the VLO in a classroom. Given that the instrument takes one hour to complete, it would be ideal if it could be done remotely, so that students can use the VLO on their own time, splitting the content and viewing it at different times, setting schedules and taking breaks, as is the idea of the study, to reduce fatigue while using it. Instruments with shorter duration may have a greater impact on learning. Another shortcoming is that we did not assess student performance in the oral medicine clinic after using the instrument. The study by Tubelo and colleagues (2016) evaluated the influence of the VLO using both theoretical and practical approaches (laboratory skills). Evaluating how students apply the acquired knowledge in the clinic would be important for demonstrating the applicability of our instrument. In addition, including students from different institutions across Brazil would be a great benefit. The assessment was performed with students from a single institution, which could lead to selection bias.

Conclusion

The VLO has proven to be a useful tool for teaching oral medicine, contributing to the knowledge of ulcerative lesions of the oral cavity. Therefore, we can recommend the use of the VLO in teaching-learning to promote active participation of students and stimulate cognitive processes through the use of audio, image, and virtual simulations.

References

• ¹
  Tubelo RA, Branco VL, Dahmer A, Samuel SM, Collares FM. The influence of a learning object with virtual simulation for dentistry: a randomized controlled trial. Int J Med Inform. 2016 Jan;85(1):68-75. https://doi.org/10.1016/j.ijmedinf.2015.11.005
  » https://doi.org/10.1016/j.ijmedinf.2015.11.005
• ²
  Fonseca LM, Leite AM, Mello DF, Dalri MC, Scochi CG. Semiotics and semiology of the preterm newborn: evaluation of an educational software application. Acta Paul Enferm. 2008 Jan;21(4):543-8. https://doi.org/10.1590/S0103-21002008000400002
  » https://doi.org/10.1590/S0103-21002008000400002
• ³
  Corradi MI, Silva SH, Scalabrin EE. Virtual objects to support the teaching-learning process of physical examination in nursing. Acta Paul Enferm. 2011 Jan;24(3):348-53. https://doi.org/10.1590/S0103-21002011000300007
  » https://doi.org/10.1590/S0103-21002011000300007
• ⁴
  Tamashiro LM, Peres HH. Development and assessment of learning objects about intramuscular medication administration. Rev Lat Am Enfermagem. 2014 Oct;22(5):716-23. https://doi.org/10.1590/0104-1169.3647.2472
  » https://doi.org/10.1590/0104-1169.3647.2472
• ⁵
  Boyle T. Design principles for authoring dynamic, reusable learning objects. Australas J Educ Technol. 2003 Mar;19(1):46-58. https://doi.org/10.14742/ajet.1690
  » https://doi.org/10.14742/ajet.1690
• ⁶
  Schibeci R, Lake D, Phillips R, Lowe K, Cummings R, Miller E. Evaluating the use of learning objects in Australian and New Zealand schools. Comput Educ. 2008 Jan;50(1):271-83. https://doi.org/10.1016/j.compedu.2006.05.006
  » https://doi.org/10.1016/j.compedu.2006.05.006
• ⁷
  Holanda VR, Pinheiro AK, Holanda ER, Santos MC. Teaching and learning in a virtual environment: nursing students’ attitude. REME. 2015 Jan;19(1):141-7. https://doi.org/10.5935/1415-2762.20150012
  » https://doi.org/10.5935/1415-2762.20150012
• ⁸
  Kwon C. Verification of the possibility and effectiveness of experiential learning using HMD-based immersive VR technologies. Virtual Real (Walth Cross). 2019 Aug;23(1):101-18. https://doi.org/10.1007/s10055-018-0364-1
  » https://doi.org/10.1007/s10055-018-0364-1
• ⁹
  Warmling CM, Pires FS, Baldisserotto J, Levesque M. Teaching Bioethics: evaluation of a virtual learning object. Rev Bioet. 2016 Sep;24(3):503-14. https://doi.org/10.1590/1983-80422016243150
  » https://doi.org/10.1590/1983-80422016243150
• ¹⁰
  Silveira HL, Gomes MJ, Silveira HE, Dalla-Bona RR. Evaluation of the radiographic cephalometry learning process by a learning virtual object. Am J Orthod Dentofacial Orthop. 2009 Jul;136(1):134-8. https://doi.org/10.1016/j.ajodo.2009.03.001
  » https://doi.org/10.1016/j.ajodo.2009.03.001
• ¹¹
  Buttussi F, Pellis T, Cabas Vidani A, Pausler D, Carchietti E, Chittaro L. Evaluation of a 3D serious game for advanced life support retraining. Int J Med Inform. 2013 Sep;82(9):798-809. https://doi.org/10.1016/j.ijmedinf.2013.05.007
  » https://doi.org/10.1016/j.ijmedinf.2013.05.007
• ¹²
  Reis LO, Ikari O, Taha-Neto KA, Gugliotta A, Denardi F. Delivery of a urology online course using Moodle versus didactic lectures methods. Int J Med Inform. 2015 Feb;84(2):149-54. https://doi.org/10.1016/j.ijmedinf.2014.11.001
  » https://doi.org/10.1016/j.ijmedinf.2014.11.001
• ¹³
  Hansen MM. Versatile, immersive, creative and dynamic virtual 3-D healthcare learning environments: a review of the literature. J Med Internet Res. 2008 Sep;10(3):e26. https://doi.org/10.2196/jmir.1051
  » https://doi.org/10.2196/jmir.1051
• ¹⁴
  Firmstone VR, Elley KM, Skrybant MT, Fry-Smith A, Bayliss S, Torgerson CJ. Systematic review of the effectiveness of continuing dental professional development on learning, behavior, or patient outcomes. J Dent Educ. 2013 Mar;77(3):300-15. https://doi.org/10.1002/j.0022-0337.2013.77.3.tb05471.x
  » https://doi.org/10.1002/j.0022-0337.2013.77.3.tb05471.x
• ¹⁵
  Gadioli B, Fulquini FL, Kusumota L, Gimenes FRE, Carvalho EC. Construction and validation of a virtual learning object for the teaching of peripheral venous vascular semiology. Anna Nery School journal of nursing. 2018 Aug; 22(4). https://doi.org/10.1590/2177-9465-ean-2018-0043
  » https://doi.org/10.1590/2177-9465-ean-2018-0043
• ¹⁶
  Alvares AG, Dal Sasso GT. Virtual learning objects: contributions to the learning process in health and nursing. Acta Paul Enferm. 2011 May;24(5):707-11. https://doi.org/10.1590/S0103-21002011000500018
  » https://doi.org/10.1590/S0103-21002011000500018

• Erratum

  Virtual learning object about oral ulcerative lesions: controlled educational intervention study. Braz. Oral Res. 2023:37:e118.
  Legend, p. 3, 5, 7, 9 Where is read: Zueger RA, Should read: Zieger RA,

Publication Dates

• Publication in this collection
  10 Nov 2023
• Date of issue
  2023

History

• Received
  24 May 2022
• Accepted
  28 Mar 2023
• Reviewed
  02 May 2023
• Corrected
  22 Mar 2024