Speech and language therapy students’ perception of learning through the incorporation of clinical simulation workshops: a mixed-methods study

Purpose: this study had two aims: (1) Analyse students’ perceptions of achieved learning in the simulation workshops implemented in multiple areas in a Speech and Language Therapy curricula, and (2) Establish the effect of incorporating simulation workshops in the students’ comprehensive evaluation of the course. Methods: a survey on perceived learning was validated, including quantitative and qualitative sections, and applied to students that participated in the simulation workshops (n=241). Additionally, quantitative information from a survey of systematic application on the comprehensive perception of the courses that implemented workshops was analysed (n=277). Results: the quantitative section showed a positive perception of learning through the simulation workshop and coincided with the qualitative section’s positive opinions. Additionally, the courses that implemented simulation workshops showed a positive evaluation in methodology, feedback, and organization. Conclusion: the students had a positive perception of the incorporation of clinical simulation workshops. The survey’s sections provided complementary information regarding learning through clinical simulation.


INTRODUCTION
Clinical simulation is a teaching-learning method that replaces or amplifies real experiences with guided experiences that evoke or replicate substantial aspects of the real clinical context in an interactive way 1 . Clinical simulation occurs in a safe environment in which the student can make mistakes and repeat a procedure without negative consequences for the patient 1,2 . This methodology has been widely implemented in areas of healthcare, facilitating the development of students' clinical and transversal skills [3][4][5][6] .
There is currently limited but increasing evidence regarding simulation in Speech and Language Therapy (SLT) programs 7 . The research focuses mainly on simulated patients' experiences, allowing technical, non-technical, and clinical decision-making skills 3 . Among the potential benefits of including simulation are the following: (a) Strengthen the abilities to address communicative and speech disorders 4,8 , assess auditory function 9 , evaluate swallowing 10 , manage dysphagia 11,12 and tracheostomy management 13 ; (b) Create a safe learning environment, correcting errors without adverse patient consequences, and allowing teachers to focus more on the students than on the patients 4 , (c) Provide specific and manageable experiences 3 , preparing the student for future clinical experiences 14,15 , and (d) Support safety and confidence to face real and difficult clinical situations 14,15 . Other recent simulation studies in SLT have included simulated parents in the context of early hearing detection 16 , evaluation of preterm infant feeding 17 , and interprofessional education scenarios between nutrition and dietetics, and speech and language therapy students 18 . Also, Howells et al. 19 have described the effect of using simulation-based learning cases on SLT students' confidence and perceptions.
Hill et al. 4 evaluated the use of clinical simulation in SLT curricula. The simulated patients were able to reproduce scenarios that allowed the students to interact with them in a standardized way, maximizing the opportunity to demonstrate comparable clinical skills 4 . In SLT, a patient interview should be performed, diagnostic information should be given, and intervention should be developed -complex tasks for students entering real clinical environments-. Additionally, explaining diagnostic hypotheses and soliciting referrals, in cases of severe communicative disorders, can have emotional implications for the patient's family. The evident symptoms (word-finding deficits, hand tremors, and hemiparalysis) are often portrayed realistically by standardized patients, but covert symptoms, such as comprehension difficulties, are portrayed far less realistically 20 . This restriction could affect simulated patients' ability (SPA) to interpret some language disorders related to the practice of SLT. Moreover, SPA almost exclusively tends to be adults since no studies have reported children's use as SPA 21 .
Simulated patients are effective in increasing confidence in the management of difficult interactions 12 . Despite the reported positive effects, this methodology must be planned appropriately when being incorporated into curricula, due to the fact that the simple existence of simulation workshops does not automatically increase learning 12 . There is evidence to suggest that simulation can partially replace traditional placement time for SLT students without the cost of competency. Hill 22 reported that when a mean of 20% of placement time was replaced with simulation, SPL students achieved an equivalent competency level. Also, educators of SLT programs agree that simulated experiences could account for up to 25% of required direct clinical hours in speech-language pathology and audiology 23 . It bases its value as an alternative model of practice hours in SLT curricula.
The current demands of higher education institutions regarding the guarantee of graduate quality require the search for strategies to develop early clinical competence. The simulation allows the integration of theoretical aspects and practical skills in a controlled environment 24 . However, students' perceptions of simulation and their competency development when engaged in simulation-based learning activities are less reported 25 . Likewise, the effect of implementing simulation-based workshops on multiple areas (e.g., combined effect on students' learning when participating in multiple workshops across the degree program) along a cycle in SLT curricula is unknown. Additionally, to our knowledge, there are no reports regarding the effect that such an implementation could have in other aspects of the execution of the course, such as the organization of the course itself, the materials available for learning, and the delivery of feedback, among other aspects that students perceive as relevant when evaluating their learning experience.
Thus, the objectives of this study were: (1) To analyse students' perceptions of achieved learning in the simulation workshops implemented in multiple areas in a SLT curricula, and (2), To establish the effect of incorporating simulation workshops in the students' evaluation (e.g. methodology, organization, infrastructure and materials, teacher quality, and feedback) of the course in which the workshops are implemented.

METHODS
The Ethics Committee of the Faculty of Medicine at Pontificia Universidad Católica de Chile approved the protocol of the study (ID number 170926007). Each student that participated signed an informed consent form.
This study was mixed with qualitative and quantitative designs, in which the quantitative research was performed together with the qualitative research, and the mixing occurred in the interpretation of the results. For the first objective, third and fourth year SLT students were recruited after attending simulation workshops. Since the activity was part of the course program, all students participated in the simulation workshops.
The students answered a survey that compiled quantitative and qualitative data. Concerning the second objective, we collected quantitative data from a survey delivered across all courses in a program, and across subsequent years (systematic application) regarding the comprehensive evaluation of the course. This evaluation was answered by students belonging to courses that already incorporated the simulations, and by others from previous years when simulation workshops were not incorporated yet.

Simulation workshops
The university where the simulation workshops were implemented is one of the oldest universities in Chile, with more than 20,000 undergraduate students. However, the SLT undergraduate degree is a recent creation at the university in question with three graduates' cohorts. The SLT curricula consist of five years of training in six areas of assessment and treatment of speech, voice, language, auditory, swallowing, and orofacial myofunctional disorders. In the first two years, the courses correspond to basic sciences, and then in the third year, the preclinical and clinical courses begin. In the fifth year, the student completes an internship in the areas mentioned above. Five simulation workshops were implemented in three clinical courses. Two of these courses were in the 8th semester (4th year), and the other was in the 6th semester (3rd year), with 277 students participating overall. In parallel to the simulation, the students were involved in clinical training in hospitals and other clinical centers throughout the course ( Table 1). The simulation workshops' aims were (1) to develop the communicative and clinical skills necessary to perform patient interviews and communicate diagnostic hypotheses effectively, and (2) to develop the procedural skills to perform therapeutic interventions.
All workshops involved simulated patients, who had never presented the health conditions they represented. The course teachers were previously trained in simulation methodology and debriefing. The simulated patients were prepared through a pilot training session, in which each course lecturer provided feedback regarding the health condition represented. A day before workshops began, the activity instructions and relevant information regarding to aetiology, clinical records, and signs presented by patients were sent to the students. Each workshop had three sections: (1) Providing initial instructions: at the beginning, it was indicated the aims of each simulation workshop; (2) Interaction with simulated patients: students had to interact -individually or in pairs-with the simulated patients; and (3) Debriefing: students were guided through a reflective process, identifying the weaknesses and strengths of their performance.

Sample size
The sample size was calculated based on the confirmatory factor analysis's goodness of fit 27 . Root Mean Square Error of Approximation (RMSEA) test was used to obtain the sample size needed to test the close-fit hypothesis of the confirmatory model. Thus, considering an RMSEA = 0.04, with a 80% power, 7 items/one factor, and α = 0.05, the sample size required was 205 participants.

Instrument development
The 'Survey of Perception of Simulation Workshop for Non-surgical Procedures', developed by Villagrán et al. 26 and used with medical students was adapted. A panel of local experts composed of those in charge of clinical courses was constituted. The adapted survey evaluated the perception of learning by the students regarding the simulation workshop. It was in a Likert scale format, with seven items (quantitative) and four open questions (qualitative). The newly

Sample size
The sample was estimated for the comparisons between evaluations applied two years before carrying out the simulation workshops, with an evaluation performed immediately after the workshops' implementation. A Mixed-effects model was performed to make these comparisons. Considering a mean difference of 0.4 points between evaluations, an alpha level of 0.05, a 90% power, with students nested within 3 clusters, and a model with a random intercept, the sample size required was 277 participants.

Quantitative analysis
Students belonged to courses that incorporated the simulations, and from previous years that did not incorporate the simulation workshop performed the evaluation. There were records of evaluations from the previous two years before the implementation of the workshops, and the evaluations given upon completion of the workshops. A mixed-effects model was used to compare adjacent evaluations specifying a random intercept to consider that students' perceptions about the same course would be correlated. Given the non-normal distribution of the model's residuals, the standard error was estimated with bootstrapping.

Validity and reliability
Content and appearance validity of the survey was determined through a panel of experts, and the construct validity through exploratory and confirmatory factor analysis. Since the survey was a Likert-type scale, a matrix of polychoric correlations was created. The determination coefficient (R 2 ) was estimated to quantify the percentage of the variance of the survey's assertions explained by the factor identified. Reliability was assessed through Cronbach's Alpha coefficient.

Quantitative analysis
All assertions were rated with a Likert scale with five response options (i.e. strongly disagree, disagree, neutral, agree and strongly agree). These options were coded from one point for 'strongly disagree' to five points for 'strongly agree'. The Kruskal-Wallis test was used to compare global scores among workshops.

Qualitative analysis
A qualitative analysis allowed the identification of categories, and determined the units of register (text citations regarding the topics broached by students). A categorization system was built, which complied with the characteristics of thoroughness, mutual exclusion, single classification principle, objectivity, and appropriateness 28 . Although previously defined categories were used at the beginning, new subcategories were added.

Data triangulation
The quantitative information was triangulated, taking into consideration the proposal of O'Cathain et al. 29 identifying total agreements, complementary information, and discordance in the qualitative and quantitative sections. Data triangulation was used to compare and consolidate findings across the quantitative and qualitative data sets.

Part 2: effect of incorporating simulation workshops in the students' course evaluation
Regarding the second objective, an instrument delivered across all courses in the SLT program, and across subsequent years was applied. Eight aspects were analysed: methodology, teachers, information sources, feedback, grades, organization, infrastructure and materials, and comprehensive evaluation of the course 30 . Students rated each aspect on a scale from 1 to 7. In Methodology, students were asked to

Instrument development
The expert committee adapted the survey initially developed by Villagrán et al. 18 to be applied in SLT area. The adapted survey maintained the seven Likert-type assertions and four open questions. The phrase 'simulation models' ('Modelos de simulación'), was replaced with 'workshop methodology ('Metodología del taller').

Validity and reliability
The exploratory factor analysis showed that the assertions adequately represented the only construct identified (Perceptions of learning through simulation workshops). This construct explained 87.2% of the variance scores. All of the factor loadings were greater than 0.4 ( Table 2). The R 2 fluctuated between 0.73 and 0.41, and the Cronbach's Alpha coefficient reached a value of 0.8.

Quantitative analysis
The overall score had a median of 34 points (35 points was the highest possible score), with the 25 th percentile scoring 32 points and the 75 th percentile scoring 35. There were no significant differences between the medians of the workshops (X 2 =7.38; p=0.117) ( Table 3).

Qualitative analysis
The

Category three: Difficulties
Three types of subcategories were observed in the answers to the question: Did you have difficulties during the simulation workshop? In the first subcategory, Organization and contextual factors, students made references to problems with the number of students, time, and environmental factors ('I need to optimize the time to perform the patient evaluation'). The Difficulty in the cognitive competence subcategory alluded to a lack of experience and studies ('I had difficulties due to the lack of experience with patients with severe speech disorder'). In the third subcategory called Difficulty in procedural competency, students referred to the delivery of diagnosis, formulation of questions, detailing, and the order of evaluation ('We did not explain the diagnosis to the patient. It was difficult to manage the patient's frustration and tiredness').
In Difficulty in attitudinal competency subcategory, students referred to seem nervous or insecure, and manifested obstacles in adapting to the needs of the patient ('It was difficult for me to adapt to the situation while achieving the objectives of the session').

Category four: Recommendations
Three types of subcategories were observed in the answers to the question: Would you recommend the simulation workshop to other students? The Perception of development of transversal and evaluative skills subcategory referred to the development of these skills ('yes, since it incorporated strategies that made possible an adequate interaction with the patient'). The preparation for clinical visits and their futures as professional, as well as the possibility to make mistakes in a safe environment was mentioned in the subcategory Preparation before real clinical practice ('yes, this allowed me to improve before starting clinical practices'). In the Practical learning opportunity subcategory, students referred to the  One  32  34  35  26  35  Two  30  33  35  28  35  Three  31  34  35  23  35  Four  32  34  35  23  35  Five  33  34  35  29  35  Total  32  34  35  23  35 workshop as a meaningful learning experience ('yes, since it was essential to work on communication skills and the human side, rather than only on technical and theoretical knowledge').

Triangulation of information
There was total agreement among the quantitative assertion number two, related to the usefulness of receiving feedback to recognize strengths and weaknesses, and the qualitative subcategories perception of learning based on reflective practice and identification of strengths and weaknesses. There was also total agreement among the assertion six regarding the opinion that the simulation workshop should be an essential component in teaching, and that 100% of the participants would recommend the workshop in the qualitative category recommendations.
The responses to assertion seven, related with being allowed to prepare oneself before performing clinical procedures, was complemented with the subcategories related to the possibility of learning from mistakes before encountering real patients. Discordance was observed in the category Contextual characteristics of the workshop, which included negative aspects related to the number of students, and that the simulation imitates reality, identifying weaknesses that are not reflected in the quantitative section.

Part 2: effect of incorporating simulation workshops in the students' course evaluation
There were no differences in any of the evaluated aspects when comparing years previous to the implementation of the workshops, except sources of information (p<0.01) and comprehensive grade (p<0.01). There were differences when comparing the year before implementing the workshops and the evaluation performed immediately after their implementation in aspects such as methodology (p<0.05), feedback (p<0.05), organization (p<0.01), infrastructure and materials (p<0.001), and comprehensive grade (p<0.001) (Figure 1).

Figure 1.
Results from the evaluation survey of the systematic application course, including results from the two years since the beginning of the course, and the evaluations given upon completion of the workshops (n=277).

DISCUSSION
Clinical simulation workshops were implemented in three courses of the SLT curricula to develop communicative and procedural skills. The first objective was to analyse students' perceptions of achieved learning in the simulation workshops implemented in multiple areas in SLT curricula. There was a positive perception of the inclusion of simulation workshops reported both in quantitative and qualitative sections of the survey.
In the perception survey, assertions five and seven obtained the highest R 2 , relating strongly to the construct measured. These assertions are linked to an increase in safety and preparation for clinical visits. This finding is in agreement with studies performed in SLT in which students after participating in workshops with simulated patients, reported decreased anxiety and increased confidence 4,7,13 .
A second objective was to establish the effect of simulation workshops in the students' evaluation of the course in which the workshops were implemented. There were differences when comparing before-after simulation workshops implementation in methodology, feedback, organization, and the overall grade. The increase in the score for feedback could be associated in part with the debriefing, which corresponds to the space for reflection regarding clinical practice, allowing students to clarify their knowledge and understand the rationality of the simulation 31 . This improvement is consistent with Clinard & Dudding 7 , where students identified the feedback and communication with the educator as the most significant strengths of the process.
The present study found the strengths and weaknesses of the simulation workshops via the incorporation of qualitative survey questions. Students' difficulties and their recommendations for future workshops were also gathered. Having a high score in the quantitative assertions, with a ceiling effect for some, limits the application of the instruments that only consider quantitative aspects. On the other hand, high scores highlight the fact that the experience was a positive one, and that the difficulties were outweighed by the benefits experienced.

Limitations and projections
One limitation is the evaluation of learning via perception, without including changes in student performance, making this study more of an initial evaluation. Despite this, it was possible to confirm changes in the evaluation that was applied systematically regarding other aspects that students perceive as relevant when evaluating their learning experience. Additionally, the performance obtained in courses that included the workshops is high, which makes it challenging to evaluate possible modifications.
Another limitation is that the improvement in the evaluation of the course in which the workshops were implemented cannot be attributed solely to the simulation workshops since the study was not controlled by other activities such as clinical training. There is a constant increase in the evaluation of courses, which is to be expected given that there is an early evaluation (half of semester), and all professors have completed methodology-based teacher training. The increase in aspects such as organization and methodology is likely attributable to the simulation workshops.
One possible projection is to evaluate the transfer of skills among students that participated in the workshops to students that did not. In this institution where the participants study, it is common that students become teachers' assistants for lower-level courses. Thus, it is not unreasonable to assume that some students model others. Barsuk et al. 32 observed that medical residents that had not yet received simulation training improved their basal performance after observing residents with more experience and who had completed simulation training.

CONCLUSION
Clinical simulation workshops were implemented in the SLT curricula to develop skills before clinical practice. To evaluate these, a learning perception survey that included a quantitative and qualitative section was adapted, and its validity and reliability were confirmed. Elevated scores showed students' positive perceptions regarding the implementation of simulation workshops, which were a learning experience where students highlighted the benefits of putting their knowledge into practice before facing real patients.