Technical procedures in clinical and surgical urgencies/emergencies for simulation-based teachingABSTRACT
Introduction: for a considerable period of time, medical education was characterized by a traditional teaching model, with questionable knowledge retention capacity and applicability. In recent years, structural changes have been implemented in the curricula and teaching plans of medical courses aiming to make undergraduate training more modern and effective. Simulation-based teaching is a possible active educational tool for this purpose.
Objective: Given the dearth of literature on the subject, a study was designed, based on the opinions of teachers at Universidade do Estado do Amazonas and the development of a prioritized list of ten technical procedures in adult clinical and surgical urgencies/emergencies to be taught to students using simulation.
Method: To achieve this objective, the modified Delphi method was employed in a three-stage process involving the administration of three sets of questionnaires to 22 teachers. The initial questionnaire comprised an open-ended question requiring the participants to identify at least 12 of the procedures in question. In the second round, the participant received a list of the 17 procedures chosen in the initial questionnaire and was asked to evaluate each one using the modified CAMES-NAF (Copenhagen Academy for Medical Education and Simulation - Needs Assessment Formula). In the third round, the teacher received the preliminary prioritized list of procedures according to the evaluation contained in the second questionnaire and, using a Likert scale, was able to state their degree of agreement with the list of procedures presented. A prioritized and validated list was then drawn up using the Content Validity Index (CVI), containing the 10 ranked procedures to be taught to medical students through simulation (CVI = 0.95).
Results: The five procedures on the list that obtained the highest scores in the CAMES-NAF formula were endotracheal intubation, cardiopulmonary resuscitation, basic airway management, central venous access, and superficial sutures.
Conclusion: A prioritized list of medical technical procedures, used in clinical and surgical urgency/emergency scenarios, which should be taught with de aid of simulation techniques to medical students was created based on the opinions of medical school teachers of Universidade do Estado do Amazonas.
Keywords:
Medical education; Simulation training; Emergency medicine; Delphi Method
RESUMO
Introdução: A educação médica, durante longo período de tempo, caracterizou-se por um modelo de ensino tradicional, com capacidade de retenção de conhecimentos e aplicabilidade questionáveis. Nos últimos anos, mudanças estruturais foram implementadas nos currículos e planos pedagógicos dos cursos de Medicina, a fim de tornar o treinamento do graduando moderno e efetivo. O ensino baseado em simulação é uma ferramenta educacional ativa possível para esse propósito.
Objetivo: Diante da escassez da literatura a respeito do assunto, idealizaram-se um estudo baseado na opinião de docentes da Universidade do Estado do Amazonas e a produção de uma lista priorizada de procedimentos técnicos em urgências clínicas e cirúrgicas em adultos a serem ensinados aos alunos por meio de simulação.
Método: Para tanto, foi utilizado o método Delphi modificado, um processo de três séries de questionários aplicados a 22 docentes. O questionário inicial foi constituído por uma questão aberta, na qual o participante deveria citar no mínimo 12 desses procedimentos. Na segunda rodada, o participante recebeu uma lista com os 17 procedimentos eleitos no questionário 1, e foi solicitada a avaliação de cada um deles por meio da fórmula Copenhagen Academy for Medical Education and Simulation - Needs Assessment Formula (CAMES-NAF) modificada. Na terceira rodada, o docente recebeu a lista preliminar priorizada de procedimentos de acordo com a avaliação contida no segundo questionário e, por meio de escala Likert, pôde informar o seu grau de concordância com a lista de procedimentos apresentada. Com a utilização do Índice de Validade de Conteúdo (IVC), foi então elaborada uma lista priorizada e validada (IVC = 0,95), contendo os dez procedimentos ranqueados a serem ensinados aos estudantes de Medicina por meio da simulação.
Resultado: Os cinco procedimentos da lista que obtiveram as maiores notas na fórmula CAMES-NAF foram: intubação endotraqueal, reanimação cardiopulmonar, manuseio básico das vias aéreas, acesso venoso central e suturas superficiais.
Conclusão: No presente estudo, foi consensuada uma lista priorizada dos procedimentos técnicos em urgências clínicas e cirúrgicas em adultos a serem ensinados por meio de simulação aos alunos do curso de Medicina com base na opinião dos docentes da Universidade do Estado do Amazonas.
Palavras-chave:
Educação Médica; Treinamento por Simulação; Medicina de Urgência; Método Delphi
INTRODUCTION
Medical education, for a long period, was characterized by a traditional teaching model, based on a passive method, centered on the teacher and their individual knowledge, on the reading of evidence, on lectures and at the bedside with the patient. It has been proven that passive learning techniques promote questionable levels of knowledge retention and applicability in practical life1.
In recent decades, medical courses have undergone structural changes in their curricula and pedagogical plans, aiming to become more modern and effective, based on the general and specific competencies that must be acquired to treat patients’ real needs2. The National Curriculum Guidelines (DCNs, Diretrizes Curriculares Nacionais )3, implemented in 2014, direct changes in the pedagogical practices of institutions with the objective of bringing social reality closer to the new knowledge network4.
An alternative educational tool for health courses that enables a more dynamic performance than traditional teaching is simulation-based teaching (SBT), a model that not only encompasses the teaching and learning of technical skills but also crisis management, clinical reasoning and teamwork, without the possibility of real harm to the patient. Simulation is a student-centered form of teaching, and the student actively participates in the entire process, optimizing learning and knowledge retention for a longer period of time5. Technical procedures, defined as: “Clinical or surgical procedures with equipment that involve direct contact with the patient”6 can be largely taught and trained through simulation, such as cardiopulmonary resuscitation and endotracheal intubation.
The SBT qualities meet the need to change the teaching-learning model of the undergraduate medical course to a more participatory and effective method, in which students can experience and practice a greater number of experiences, with a homogeneous offer to all, allowing the repetition of practice and in a safe environment. Therefore, simulation-based training can complement the traditional teaching-learning approach, making undergraduate students safer and better prepared1.
Although the effectiveness of simulation-based teaching has been widely demonstrated, each medical school has different particularities and requirements for its undergraduate curriculum, and it is important to conduct research to assess each reality7. Studies of needs in clinical and procedural training are mostly focused on postgraduate studies, with few studies directed to undergraduate studies.
In view of the above, with the need for change in the teaching-learning method observed in the undergraduate medical course and due to the scarce literature regarding procedural training directed to undergraduate students, this study aimed to develop in an innovative way a prioritized list of technical procedures in clinical and surgical urgencies/emergencies in adults, using the Delphi methodology. This method is known worldwide for providing great and important decisions through consensuses established among experts’ opinions. It was initially widely used by the military and currently by large companies in the most diverse areas. The findings of this study will help the management and faculty in the inclusion of simulation as an educational strategy in the medical curriculum and guide the practice, in fact, of procedural teaching through simulation.
METHOD
This was a qualitative and quantitative, prospective and non-probabilistic study, using the modified Delphi method through the creation and application of three consecutive questionnaires to teachers from Universidade do Estado do Amazonas in the disciplines of Internal Medicine, Surgical Clinic, Anesthesiology and Urgency/Emergency, regarding technical procedures in clinical and surgical urgencies/emergencies. The project was submitted to the Research Ethics Committee (REC) of Universidade Federal do Amazonas (UFAM) and was approved.
According to the Delphi methodology, there is no recommendation for statistical calculation of sample size. The method prioritizes the selection of participants with extensive knowledge and experience in the subject to be researched8. The number of required specialists varies widely, but studies indicate that an optimal number of such specialists should not be lower than ten9. The method encourages the cooperation of a small number of participants10, suggesting the creation of groups with 6 to 30 members11.
The inclusion criteria for this study were: being a teacher of internal medicine, clinical surgery, anesthesiology, and urgency/emergency medicine; teachers who have experience in clinical or surgical urgencies/emergencies; agreeing with and signing the Free and Informed Consent Form (FICF).
The exclusion criterion of this study was: not currently exercising teaching activities due to leave and/or personal reasons.
The final sample consisted of 22 participants and the study took place in three stages, as described below.
Stage 1: Formulation and application of Questionnaire 1.
Initially, individual contact was made, via WhatsApp, with the teachers to ascertain their interest in participating in the study. According to the teacher’s acceptance, an in-person meeting was scheduled for a detailed explanation of the study, as well as important topics, such as the participant’s anonymity and the need for effective participation to achieve the desired result, the prioritized list of procedures.
In the first in-person meeting, after the presentation of the research and clarification of any doubts, the FICF was handed out for reading and signing. After that, the first questionnaire was given to the participating teacher and the day for its return was scheduled.
The initial questionnaire was prepared by the team of researchers and consisted of an open question in which the participant was asked to mention, in writing, at least twelve technical procedures in clinical and surgical urgencies/emergencies in adults, which they thought the undergraduate medical student should be able to perform before completing the course.
After the application of the first questionnaire, the technical procedures mentioned were analyzed and grouped through descriptive content analysis. Taking into account the definition of technical procedures, a qualitative analysis was carried out with the exclusion of non-technical procedures, as well as duplicate procedures and the grouping of those that were written in similar ways but had the same meaning.
Stage 2: Creation and application of Questionnaire 2 - Evaluation
In this second moment, the participating teacher was given a form with 17 technical procedures elected in questionnaire 1 and, for each of them, a formula for evaluating the procedures was presented.
This evaluation was based on the modified CAMES-NAF - Copenhagen Academy for Medical Education and Simulation - Needs Assessment Formula6, with modifications aligned with the reality of the study. Through this formula, we sought to quantify the need for simulation-based procedural training (SBPT) through the scoring of each procedure, on a scale of 1 to 5 points, considering the items mentioned in Table 1.
Using the modified CAMES-NAF evaluation formula, each procedure mentioned in the first round was evaluated and a descriptive analysis was performed. The CAMES-NAF score modified for the individual procedures was given by the sum of the scores (1-5) of the four factors (frequency, number of physicians, impact, and feasibility), with each factor having a weight of 25%. The total score achieved, resulting from 4 to 20 points, determined the classification of each procedure in the preliminary prioritized list.
Stage 3: Creation and application of questionnaire 3 - Content validation
In the third round, the preliminary list of procedures resulting from the second stage was presented to the teachers and they were asked to evaluate and demonstrate their degree of agreement with it for content validation through a Likert Scale with five options: “I fully agree”, “I agree”, “I neither agree nor disagree”, “I disagree” and “I fully disagree”. To calculate the degree of agreement, the Content Validity Index (CVI)( was used, obtained by adding the number of answers in which the teachers marked option 4 “I agree” and 5 )“I fully agree” and dividing the number found by the total number of answers, stipulating the acceptable agreement rate among the teachers as ≥ 0.78. An open question was asked if the teacher wished to leave any observation about the list of procedures.
Stage 4: Preparation of the final prioritized list of technical procedures
After the third round of questionnaires, the final prioritized list of technical procedures in clinical and surgical urgencies/emergencies in adults considered essential for the training of undergraduate medical students was prepared, in agreement with the teachers. Figure 1 illustrates the study flowchart, following the Delphi methodology.
Stage 5: Characterization of the participants’ profile
After the end of the rounds and the creation of the prioritized list of procedures, the study authors verified the need to outline the profile of the research participants. For this purpose, contact via WhatsApp was used again and four additional questions were asked: How long has it been since you graduated? How long have you taught? In your weekly routine, do you have any workload in the urgency/emergency and/or intensive care? If the answer to question 3 is NO, how long has it been since you have had urgency, emergency and/or intensive care in your routine?
RESULTS
Twenty-six teachers were contacted; however, 2 did not continue the contact via WhatsApp and 2 received but did not answer the first questionnaire. The participants included in the study who completed all stages totaled 22 teachers, distributed among the disciplines, as shown in Table 1, with internal medicine being the most frequent one (45.5%).
The participants’ profile regarding the time since graduation in medicine ranged from 17 to 37 years, with a mean of 24.82 years. Teachers who had graduated between 21 and 31 years before comprised the largest portion of the sample (63.6%).
When analyzed from the perspective of time working as a teacher, there was a variation between 9 and 34 years, with an average of 17.27 years. Teachers who had been teaching for 21 to 31 years comprised the majority of the sample (63.6%).
Regarding the analysis of the teachers in relation to the presence of an urgency/emergency and/or intensive care workload in their current routines, 12 participants (54.5%) said they had it and 10 (45.5%) said they no longer had it. When asked to those who had answered NO, how long they had been without this experience, 50% said they had not had that routine for more than 10 years.
In the first stage of the study, the technical procedures in clinical and surgical urgencies/emergencies in adults that the teachers considered that the medical student should be able to perform were analyzed quantitatively by counting the number of citations they received and qualitatively grouped and summarized by similarity. The lowest number of procedures cited per questionnaire was 12 and the highest number, 18. The total number of responses in the first stage was 304. After the qualitative analysis, in which the mentioned procedures that were not technical and/or not urgency/emergency procedures were excluded, and after grouping those that were written in similar ways, but had the same meaning, a list of 31 procedures was reached, of which 17 were selected first, according to the number of citations (Table 2).
Taking into account the frequency of the procedure in the different work environments of the teachers included in the research, the 10 most frequently mentioned procedures were: peripheral venous access, basic airway management, superficial sutures, central venous access, digestive catheterization, arterial puncture, orotracheal intubation, urinary catheterization, cardiopulmonary resuscitation, and drainage of superficial skin and soft tissue abscesses. The frequency of these procedures varied mostly from a few times a month to a few times a week.
When evaluating the number of university graduates that the teachers thought would be capable of performing a certain procedure in their work environment in the future, it was possible to see that the teachers believe that 61 to 80% of the physicians who graduated from the university should know how to perform the following procedures: superficial sutures, cardiopulmonary resuscitation, orotracheal intubation, basic airway management, drainage of superficial skin and soft tissue abscesses, central venous access, peripheral venous access, urinary catheterization, arterial puncture, digestive catheterization. The means of superficial sutures and cardiopulmonary resuscitation procedures were very close to the higher category, in which 80-100% of the graduates should know how to perform these procedures.
Regarding the impact of the procedure, assessed according to the following information: “This procedure is uncomfortable or risky for the patient if performed by an untrained physician”, the procedures that the teachers most agreed to be in accordance with the above statement are the following: tracheostomy, closed chest drainage, cricothyroidotomy, thoracentesis, orotracheal intubation, central venous access, lumbar puncture, venous dissection, cardiopulmonary resuscitation and paracentesis, and it can be observed that the procedures evaluated as the most likely to have a negative impact on the patient’s life are, for the most part, those that are more invasive and require more training and skill for their performance. With the exception of endotracheal intubation and cardiopulmonary resuscitation, the others did not appear among the ten most frequent procedures.
The feasibility of simulation-based procedural training was assessed and the five procedures that received the highest scores are the following: cricothyroidotomy, cardiopulmonary resuscitation, orotracheal intubation, tracheostomy, and thoracentesis. It was found that ten of the seventeen assessed procedures received a score ≥ 3, indicating that they can be taught both in a clinical environment and in a simulation environment.
After evaluating each procedure using the modified CAMES-NAF evaluation formula, the prioritized preliminary list of 17 procedures was obtained. And after the third round of questionnaires and the validation of the content through CVI = 0.95, the final prioritized list was prepared, which was the same as the preliminary list, containing the first ten technical procedures in clinical and surgical urgencies/emergencies in adults considered essential for the training of medical students, in agreement with the teachers, and the educational tool of simulation-based teaching can be used to train these procedures (Table 3).
As for the suggestions given by the participants in the third stage of the research, there was none that was common to more than three participants.
The data were analyzed using the IBM SPSS Statismtics software, version 22.
DISCUSSION
The prioritized list of technical procedures to be taught to undergraduate medical students consisted primarily of those related to airway management (basic, endotracheal intubation, and cricothyroidotomy), cardiopulmonary resuscitation maneuvers, venous access, digestive and bladder catheters, and superficial sutures.
In the current literature, there are few similar studies on the assessment of needs directed to medical students. The Australian study by Green et al.12 (2022), which also used Delphi methodology in a series of three rounds of online questionnaires, with a panel of participants comprising clinical physicians from various specialties and medical education professionals, had as one of the objectives to verify the basic competencies of procedural skills in general, for the medical student, reaching a number of 46 procedures, divided into 10 categories (cardiovascular, diagnostic, gastrointestinal, injections/accesses, ophthalmological, respiratory, surgery, trauma, women’s health and urogenital). Among the elected procedures, those that showed the highest level of agreement included: cardiopulmonary resuscitation, airway management, asepsis, and surgical gowning. Although the present study focuses on clinical and surgical urgencies/emergencies and the study by Green et al.12) (2022) covers medical fields in a more general way, similar results were found across the studies.
Breindahl et al.7 (2023), using the Delphi methodology in three series of questionnaires, performed a study exploring the general training needs for newly graduated physicians and thus arrived at the proposal of a simulation-based curriculum for graduation. Nineteen procedures were included and prioritized, the five main ones being: peripheral venous access, gowning with personal protective equipment, basic airway management, basic life support, and radial artery puncture.
Bessmann et al.6 (2019) used a methodology very similar to that of the present study to establish a consensus on the technical procedures for simulation-based training for anesthesiology residents. In this study, the final prioritized list of 30 groups of procedures to be trained in simulation by anesthesiology residents had as the first five: cardiopulmonary resuscitation, direct laryngoscopy and videolaryngoscopy, use of defibrillator, emergency cricothyroidotomy, and intubation with fiber optics. Although the study by Bessmann et al.6 (2019) was carried out for a different target audience, it could be observed that the cited procedures are in common with those listed in this study.
Regarding the evaluation of the percentage of graduates who the teachers believe should know how to perform certain procedures in their future work environments, the procedures that reached the highest percentages (61-80%) were superficial sutures, cardiopulmonary resuscitation, endotracheal intubation and basic airway management, with the first two showing scores very close to the maximum (4.9), demonstrating the real need for these procedures to be taught to students, either in a clinical environment and/or through simulation.
It was interesting to note that procedures widely cited in the first brainstorming phase, such as paracentesis and closed chest drainage, did not appear among the first 10 procedures after the application of the modified CAMES-NAF formula, whereas items that were rarely mentioned in the first phase gained higher scores in the second phase, such as basic airway management and cricothyroidotomy.
When assessing the feasibility of simulation-based procedural training, ten of the seventeen procedures evaluated received a score ≥ 3, meaning that they should be taught in both clinical and simulation settings. The most invasive procedures obtained the highest scores, with cricothyroidotomy obtaining the highest number of scores 5 (12/22), suggesting the need for its training in a simulation environment, in this case, most likely associated with its impact on the patient’s life.
The other procedures obtained a score below 3, suggesting that they can be learned safely in a clinical environment (paracentesis, peripheral venous access, superficial sutures, passage of a digestive tube, arterial puncture and drainage of superficial abscess).
In the third and final stage, the list obtained in the second stage was validated, using a Likert scale to assess the teachers’ level of agreement with the presented list. The Content Validity Index (CVI) was used for validation, the most often used index in the health area13, which has 0.78 as the lowest acceptable value. The study CVI was 0.95, which shos a high degree of agreement among the teachers regarding the list and the consequent validation of the presented content.
The study had weaknesses and limitations, such as: the scope of the topic, which included technical procedures in clinical and surgical urgency/emergency in specific disciplines which, in theory, have this experience in their daily routine. However, other disciplines that also deal with urgency and emergency could also have been included, such as obstetrics, orthopedics and pediatrics; the sample of the study participants comprised teachers from a single higher education institution, UEA. Therefore, the composition of the panel of experts did not include teachers from other medical schools, which may have a different teaching reality from that of the participating institution. In addition to the abovementioned fact, the study could have selected only those teachers who have urgency/emergency and/or intensive care experiences in their current daily routine. Another point to be analyzed is the modified CAMES-NAF evaluation formula, which, although widely used in its original form in studies to assess needs in Denmark, has not been validated into Portuguese, and was freely translated by the authors and adapted to the reality of the study.
CONCLUSIONS/FINAL CONSIDERATIONS
In the present study, a prioritized list of technical procedures in clinical and surgical urgencies/emergencies in adults to be taught through simulation to medical students was consensual, based on the opinion of UEA teachers.
It was found that, although few similar studies are available in the current literature, the results found are in agreement for the most part with those of existing studies, with airway management and cardiopulmonary resuscitation being present in all of them with high degrees of agreement.
The findings of this study may help management and faculty in the inclusion of simulation as an educational strategy in the medical curriculum and guide the practice, in fact, of teaching procedures through simulation.
References
- 1 Brandão CFS, Collares CF, Marin HF. A simulação realística como ferramenta educacional para estudantes de medicina. Sci Med. 2014;24(2):187-92.
- 2 Troncon LEA, Maffei CML. A incorporação de recursos de simulação no curso de graduação em Medicina da Faculdade de Medicina de Ribeirão Preto - USP. Medicina (Ribeirão Preto). 2007;40(2):153-61.
-
3 Brasil. Diretrizes Curriculares Nacionais para os Cursos da Graduação em Medicina (DCN). Brasília: Ministério da Educação, Câmara de Educação Superior do Conselho Nacional de Educação; 2001 [acesso em 22 jun 2022]. Disponível em: Disponível em: http://portal.mec.gov.br/cne/arquivos/pdf/CES04.pdf
» http://portal.mec.gov.br/cne/arquivos/pdf/CES04.pdf - 4 Mitre SM, Siqueira-Batista R, Girardi-de-Mendonça JM, Morais-Pinto NM, Meirelles CAB, Pinto-Porto C, et al. Metodologias ativas de ensino-aprendizagem na formação profissional em saúde: debates atuais. Cien Saude Colet. 2008;13(supl 2):2133-44.
- 5 Brim NM, Venkatan SK, Gordon JA, Alexandre, EK. Long-term educational impact of a simulator curriculum on medical student education in an internal medicine clerkship. Simul Healthc. 2010;5(2):75-81.
- 6 Bessmann EL, Ostergaard HT, Nielsen BU, Russel L, Paltved C, Ostergaard D, et al. Consensus on technical procedures for simulation‐based training in anaesthesiology: A Delphi‐based general needs assessment. Acta Anaesth Scand. 2019;63(6):1-10.
- 7 Breindahl N, Khan F, Skipper M, Nielsen AB, Friis ML, Paltved C, et al. Exploring training needs of newly graduated medical doctors to inform the undergraduate simulation-based curriculum: a national Delphi consensus study. Postgrad Med J. 2023;99(1167):37-44.
-
8 Yousuf MI. Using experts’ opinions through Delphi technique. Practical Assessment, Research & Evaluation. 2007;12(4):1-7 [acesso em Fev 2023]. Disponível em: Disponível em: http://pareonline.net/getvn.asp
» http://pareonline.net/getvn.asp - 9 Marques JBV, Freitas D. Método Delphi: caracterização e potencialidades na pesquisa em educação. Pro-Posições. 2018;29(2):389-415.
- 10 Hsu C, Sandford B. The Delphi technique: making sense of consensus. Pract Assess Res Eval. 2007;12(10):1-8.
- 11 Reguant-Álvarez M, Torrado-Fonseca M. El método Delphi. Revista d’ Innovació i Recerca en Educació. 2016;9(1):87-102.
- 12 Alexandre NM, Coluci MZO. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Cien Saude Colet . 2011;16(7):3061-8.
- 13 Green P, Edwards J, Tower M. Core procedural skills competencies and the maintenance of procedural skills for medical students: a Delphi study. BMC Med Educ. 2022;22(1):1-13.
Source: Prepared by the authors (2022).