Realistic simulation is associated with healthcare professionals’ increased self-perception of confidence in providing acute stroke care: a before-after controlled study

ABSTRACT Background: Simulations are becoming widely used in medical education, but there is little evidence of their effectiveness on neurocritical care. Because acute stroke is a neurological emergency demanding prompt attention, it is a promising candidate for simulation training. Objective: To assess the impact of a stroke realistic simulation course on clinicians’ self-perception of confidence in the management of acute stroke. Methods: We conducted a controlled, before-after study. For our intervention, 17 healthcare professionals participated in a stroke realistic simulation course. As controls, participants were chosen from a convenience sample of attendees to the courses Emergency Neurologic Life Support (ENLS) (18 participants) and Neurosonology (20 participants). All participants responded pre- and post-test questionnaires evaluating their self-perception of confidence in acute stroke care, ranging from 10 to 50 points. We evaluated the variation between pre- and post-test results to assess the change on trainees’ self-perception of confidence in the management of acute stroke. Multivariate analysis was performed to control for potential confounders. Results: Forty-six (83.63%) subjects completed both questionnaires. The post-test scores were higher than those from the pretests in the stroke realistic simulation course group [pretest median (interquartile range - IQR): 41.5 (36.7-46.5) and post-test median (IQR): 47 (44.7-48); p=0.033], but not in the neurosonology [pretest median (IQR): 46 (44-47) and post-test median (IQR): 46 (44-47); p=0.739] or the ENLS [pretest median (IQR): 46.5 (39-48.2), post-test median (IQR): 47 (40.2-49); p=0.317] groups. Results were maintained after adjustment for covariates. Conclusions: This stroke realistic simulation course was associated with an improvement on trainees’ self-perception of confidence in providing acute stroke care.


INTRODUCTION
Simulation techniques in medical education are gaining recognition and popularity due to their capacity for training clinical skills in a practical and realistic way, without putting patients or participants at risk 1 . Simulation scenarios allow students to repeatedly practice their abilities and improve their efficiency, decision making, leadership and communication skills in a time-pressure environment 1,2,3 . The longstanding experience of simulation-based learning in anesthesiology training is well known for its positive results 4 , but there are few data on the current use of simulation techniques in neurology training 5 .
Stroke is a leading cause of mortality and morbidity worldwide 6 . Proper acute stroke management must be fast, well coordinated and effective 7 . The need for quick and effective stroke care is widely recognized in the literature and is represented by the aphorism "time is brain". On the other hand, a large scale of fatal errors is mainly caused by deficits in communication skills, interaction and decision making by the care team 7 . Furthermore, acute stroke is an emergency and there is a limitation of practical teaching to the observation and demonstration of care by the fully trained professional 8 . As such, acute stroke training is a promising candidate for realistic simulation approaches.
In this study we aimed to determine the impact of a stroke realistic simulation course on the self-perception of confidence by clinicians in the management of acute stroke care as compared with other neurology courses. We also aimed to describe the structure of this didactic resource and to evaluate the acceptance of the training amongst participants.

Study description
This was a pilot-controlled, before-after study.

Study population
We enrolled 55 healthcare professionals and medical students attending the XI Brazilian Congress of Cerebrovascular Diseases on October 8 th , 2017.
The inclusion criterion was participation in one of the courses described below. Participants were excluded if they did not consent to participate.
Seventeen subjects were submitted to the intervention and participated in the stroke realistic simulation course.
As controls, 18 participants of the Emergency Neurological Life Support (ENLS) course and 20 participants of the Neurosonology course were chosen from a convenience sample. Both courses were conducted during the congress. We chose to include controls from both of these because that would enable us to compare the intervention with an "active control arm" (i.e., the ENLS, which focuses on neurologic emergencies, including stroke) and a "placebo arm" (i.e., the Neurosonology course, which does not address stroke management).

Outcomes and definitions
Our primary outcome was a variation in the self-perception of confidence when treating acute stroke patients before and after the courses. This was measured with the use of preand post-test questionnaires, comprising 11 questions, translated and adapted from previously validated versions 7 .
The questionnaires were provided 30 minutes before and after the courses, with answers adding up to a score RESUMO Introdução: Simulações são amplamente utilizadas na educação médica, mas há pouca evidência de sua eficácia no tratamento de pacientes neurocríticos. Como o acidente vascular cerebral agudo (AVC) é uma patologia que requer atendimento imediato, o uso de simulação pode ser uma ferramenta útil no treinamento do manejo desses pacientes. Objetivo: Avaliar o impacto do uso de simulação realística na autopercepção de segurança no atendimento a pacientes vítimas de AVC agudo. Métodos: Estudo antes-depois controlado. No grupo da intervenção, 17 profissionais da área de saúde participaram de um curso de simulação realística de atendimento a pacientes com AVC. Como controles, os participantes foram escolhidos a partir de uma amostra de conveniência composta por 18 participantes do curso Emergency Neurologic Life Support (ENLS) e 20 participantes de um curso de Neurossonologia. Foram respondidos questionários antes e após o curso para avaliar a autopercepção de segurança no atendimento a pacientes vítimas de AVC agudo, variando de 10 a 50 pontos. Foi avaliada a variação entre os resultados pré-e pós-teste, para avaliar a mudança na autopercepção de confiança do trainee no manejo do AVC agudo. Análise multivariada foi realizada para controlar possíveis fatores de confusão. Resultados: Quarenta e seis (83,63%) participantes responderam aos questionários. A pontuação no questionário pós-curso foi maior do que a obtida no questionário pré-curso no grupo de participantes do curso de simulação realística em AVC between 10 and 50. The first 10 questions consisted of a Likert scale varying between 1 (complete disagreement) to 5 (complete agreement), and the last question had two options as possible answers. Information was also collected on age, gender, occupation and whether the subject had already participated in other realistic simulation courses on acute stroke care. The questionnaires were self-administered and participants had 30 minutes to answer it both before and after the courses.

Description of the intervention
The stroke realistic simulation course consisted of two main parts. Firstly, participants were introduced to the institutional acute stroke management protocol of São Rafael Hospital (HSR) and watched the "Acute stroke aid" movie, which includes every step of the protocol assistance and was recorded in the same institution. The movie portrays the ideal care given to acute stroke patients, emphasizing critical points such as the contraindications to thrombolysis and the importance of an integrated and well-trained multidisciplinary team.
Then, professionals were split in three groups of six persons per instructor. Each team was composed of six participants, role-playing one nurse, one nursing technician, two emergency physicians, one observer and one reporter. Before the simulations, all participants were introduced to their functions according to their roles. The observer used a white board to draw an impartial timeline of all events occurring during the simulation sets.
There were three stroke realistic simulation sets per group. The first covered a patient with acute stroke and contraindications for thrombolytic therapy, the second had a patient with acute stroke but out of the time window for thrombolysis and the last involved a patient with acute stroke and hypertension.
Simulations took place in an environment fully equipped and faithfully adapted to portray an acute stroke picture in the best way possible. It included a vital signs monitor attached to an iPad (Apple, United States of America) broadcasting the content of a Defibrillator Automatic Rhythm Training Simulator (DART) Sim App © (DART Sim Inc.), a vital signs simulator controlled by the simulation instructor. Participants were provided with a standardized kit containing: the "HSR Step-by-Step Guide for Assisting Suspected Acute Stroke Patients", elaborated according to the most recent stroke guidelines 9,10,11,12 ; the National Institute of Neurological Disorders and Stroke Scale (NIHSS), translated and adapted into Portuguese 13 ; a list of eligibility criteria for thrombolysis therapy 9,10,11,12 ; and a list of the amount of alteplase/ body weight to be infused in each patient. A thrombolysis kit was also included, with two alteplase ampoules, infusion pump equipment, a puncture kit, a tourniquet, a calculator, gloves, plasters, a bolus syringe, a peripheral intravenous catheter, a spatula and 0.9% 10 mL saline solution ampoules.
There was also a bioimaging station providing all necessary information to participants, such as computed tomography (CT) and magnetic resonance imaging (MRI) results and their follow-up images.
After each simulation, a feedback and debriefing session of highlights, improvements needed and key points was conducted.
No specific interventions were made in the control groups, other than having them respond to the questionnaire. Courses were taught as usual 14,15 .

Statistical analysis
Continuous variables were reported as mean and standard deviation or median (interquartile range -IQR), and the categorical variables were reported as counts and frequencies (percentage).
Categorical variables were compared with the chisquare test. Continuous variables were evaluated using the Friedman's two-way analysis of variance by ranks to evaluate which group had the most significative variation when comparing pre-and post-test results. After this the Wilcoxon's signed rank test was conducted on the data to follow up this finding and evaluate the features of variation in each group. Groups were compared independently.
Variation between pretest and post-test scores was also assessed as a binary variable, with an increase in post-test scores being labeled as "positive variation", while neutral or negative variations in post-test scores being labeled as "nonpositive variation". Multivariable logistic regression analysis was performed for adjustment for potential confounders. Variables were selected for the final model based on the theoretical association with the outcome of interest. Goodnessof-fit was assessed with the Hosmer-Lemeshow test.
Statistical analyses were conducted with the IBM Statistical Package for the Social Sciences (SPSS ® , Chicago, IL, USA) 25.0, R (R Programming Language) and Microsoft Excel ® 2016 software. Values of p<0.05 were considered statistically significant.

Ethical concerns
This study was submitted and approved by the Ethics Committee in Human Research of São Rafael Hospital, and all participating subjects signed a written consent form.

Characteristics of the participants
Of the 55 participants, 46 (83.6%) completed the pre-and post-test questionnaires, with the following distribution: 14 subjects (30.4%) from the stroke realistic simulation course (intervention group), 18 (39.1%) from the Neurosonology course and 14 (30.4%) from the ENLS course (control groups), as shown in Figure 1.
The participants' characteristics are shown in Table 1. Overall, most subjects (30 or 65.2%) were neurologists or neurology fellows and the mean age was 30.9±6 years.

Impact of the intervention on the self-perception of confidence in the management of acute stroke
There was an increase in the subjects' self-perception of confidence in the management of acute stroke of participants in the stroke realistic simulation course (intervention), but not in the control groups, i.e., the Neurosonology and ENLS groups (Figure 2). Post-test scores were higher than pretest scores in the stroke realistic simulation course group Overall, median (IQR) variation in the pre-and posttest scores was 0 (-0.25 to +4.25), ranging from -16 to +15. Twenty-one (45.7%) participants had a positive variation in the scores, distributed as 11 (78.6%) in the stroke realistic simulation course, six (42.9%) in the ENLS group and four (22.2%) in the Neurosonology group, p=0.006. After adjustment for neurology specialization and previous participation in stroke simulation courses, allocation to the stroke realistic simulation group was associated with a positive variation between pretest and post-test scores [OR (95%CI)=10.6 (1.68-67.35); p=0.012), as shown in Table 2.

Acceptance of the stroke realistic simulation course among participants
All 14 participants (100%) in the stroke realistic simulation course acknowledged its contribution to amplify their knowledge and skills on acute stroke care, agreed that the simulated situations were realistic and recognized their applicability to the medical routine.    Table 2. Multivariable analysis for association with positive variation between pretest and post-test scores.

DISCUSSION
Our research has shown that a high-fidelity simulation course on stroke care was associated with an improvement in the level of confidence involved in the management of this situation.
The course structure ensured that subjects had close contact with the most current stroke guidelines/recommendations. The main concepts regarding the initial treatment and management of acute stroke were transmitted in a repetitive and accessible way.
The importance of providing adequate conditions for training in neurological emergencies is widely acknowledged among the academic community 2,16,17 . High-fidelity simulation training not only improves knowledge, but also allows for a refinement of the care protocols 16,17,18 . Besides, a structured simulation setting enables interactive and standardized scenarios 5 , which, along with a debriefing session conducted by experienced physicians after each scenario, promotes an active way of learning, resulting in the consolidation of the content 4,19 .
Our course may be an effective tool for the training and institution of an acute stroke care algorithm in the emergency department. The protocol is presented theoretically at the beginning of it and repeatedly practiced through each scenario.
Training with realistic scenarios can also lead to a decrease in the error rate of these subjects when treating acute stroke 17,20 . Our interventional, controlled design made it possible to correlate the intervention with an increase in healthcare professionals' self-perception of confidence.
However, the study had some limitations. Its small sample size may hamper the generalizability of our findings. However, we analyzed participants in a national congress with different backgrounds, so our results may apply to similar settings. Moreover, the short duration of our observation of the subjects makes it difficult to infer the long-term effects of the intervention; therefore, the course's impact on participants may be over-or underestimated. Nevertheless, because this was a pilot study, its long-term effects were not our primary endpoint. Furthermore, even though we had a control group, which strengthens our findings, the non-randomized design of the study made it prone to confounding factors that may not have been accounted for. Nevertheless, our results were sustained after multivariable analysis adjusting for potential confounders. In addition, we analyzed the self-perception of confidence, but not the improvement of actual skills in acute stroke management.
This realistic simulation training was associated with an increase in confidence in the management of acute stroke patients. The course was highly accepted by the subjects and was relatively easy to follow.

ACKNOWLEDGMENTS
We would like to acknowledge the support of the acute stroke team and research department of Hospital São Rafael in the development of this study. We also would like to thank all members of the Serviço de Assistência, Ensino e Pesquisa em Medicina Intensiva (SAPIENS-MI) for being a part of this research project.