Validation of an endoscopic flavectomy training model

ABSTRACT Objective: to validate a lumbar spine endoscopic flavectomy simulator using the construct method and to assess the acceptability of the simulator in medical education. Methods: thirty medical students and ten video-assisted surgery experienced orthopedists performed an endoscopic flavectomy procedure in the simulator. Time, look-downs, lost instruments, respect for the stipulated edge of the ligamentum flavum, regularity of the incision, GOALS checklist (Global Operative Assessment of Laparoscopic Skills), and responses to the Likert Scale adapted for this study were analyzed. Results: all variables differed between groups. Procedure time was shorter in the physician group (p < 0.001). Look-downs and instrument losses were seven times greater among students than physicians. Half of the students respected the designated incision limits, compared to 80% of the physicians. In the student group, about 30% of the incisions were regular, compared to 100% in the physician group (p < 0.001). The physicians performed better in all GOALS checklist domains. All the physicians and more than 96% of the students considered the activity enjoyable, and approximately 90% believed that the model was realistic and could contribute to medical education. Conclusions: the simulator could differentiate the groups’ experience level, indicating construct validity, and both groups reported high acceptability.


INTRODUCTION
S pinal endoscopy emerged in the 1990s as a less invasive method for the surgical treatment of herniated lumbar discs. This technique has functional advantages over traditional methods [1][2][3]  It is expected that experienced surgeons have greater surgical skills than those who do not perform surgery, and the better performance of surgeons in a simulator should be evident [9][10][11][12] .
The use of objective criteria, such as procedure time and the number of instrument losses or lookdowns, gives credibility to the analysis. The more often the operator looks down, the greater the difficulty in understanding the three-dimensional environment of a video-assisted surgery and the less skilled the operator is 13,14 .
Questionnaires such as those involving Likert scales are also used to corroborate the research results 15 .
Using simulators of spinal endoscopy is an interesting option, since endoscopic flavectomy is However, the prototypes currently available on the market are expensive, limiting access to training.
To fill the niche between expensive commercial simulators and low-cost solutions, this study aimed to validate a reproducible simulator of endoscopic lumbar spine flavectomy using the construct method, the GOALS checklist, and Likert scale assessments. The simulator's acceptability in medical education was also assessed.

METHODS
This cross-sectional experimental study was approved by the research ethics committee of a university hospital (reference 1,994,655).
The sample consisted of a group of orthopedists and a control group of senior medical students, who were selected through simple random sampling using a random number generator (UX APPS Random Number®, version 2.1.8.2018). The exclusion criteria were refusal to grant consent and previous contact with the simulator.
A previously developed spinal endoscopy simulator was used to assess the selected individuals 16 ( Figure 1).
The yellow ligament was simulated using an 8 x 11 cm sheet of yellow ethylene-vinyl acetate with a 6.25 cm 2 square drawn on it ( Figure 1).
An SXT-5.0M video camera (KKMOON, Shenzhen, China) coupled to a computer was used to simulate the endoscope, with the images projected on a monitor. The probe-type camera had its own light source and USB port ( Figure 2).
Real endoscopic surgical scissors were used for the flavectomy procedure ( Figure 2).  The participant inserted the instruments into the simulator through the classic dorsal paramedian portal at the L5-S1 level. The participant was asked to find his/her position in the space, identify the surrounding structures, the mark on the yellow ligament, and perform the flavectomy up to designated limit in the model (maximum 6.25 cm 2 ), opening the ligament in a rectilinear manner in the center of the marked square, so that the nerve root of L5-S1 and the herniated disc (represented in red) could be visualized.
All procedures were supervised, and the participants were told to interrupt the activity if they considered the result satisfactory or when the 10-minute time limit was reached. All procedures were performed at the Orthopedic Skills Laboratory of our institution.
The endoscopic images from the flavectomy were transmitted to a personal computer via USB cable and recorded with NCH Debut 5.14.c video capture software (NCH Software, Inc., Greenwood Village, CO, USA) ( Figure 3). The videos and images were analyzed blindly.
The assessed variables were total procedure time, number of look-downs, and number of instrument losses, respect for the established limit for the yellow ligament (exceeding this limit was considered inappropriate; this item was rated as "yes" or "no"), and an appropriate, regular incision shape (i.e., straight and centered without deviations and having delicate edges).

Kulcheski
Validation of an endoscopic flavectomy training model

RESULTS
The sample consisted of 30 students (53% female, 47% male) and ten orthopedists (100% male), who accounted for the total number of physicians with sufficient experience to perform the procedure safely. The 30 students were randomly selected from the population of 88 students enrolled in the last year of the medical curriculum in our institution. Mean age was 23 years for students and 44 years for physicians.
Physicians' mean experience in video-assisted surgery was 13 years. Five orthopedists had experience in knee arthroscopy, three in shoulder arthroscopy, one in knee and shoulder arthroscopy, and one in knee, ankle, and shoulder arthroscopy.
The quickest procedure, less than 2 minutes, was performed by the physician with 9 years of experience in knee and shoulder arthroscopy. Two students used the full 10-minute allowance to complete the task and another five students required more than eight minutes to do so, while none of the doctors required the maximum time. The difference between the mean total time of the students (4 min) and doctors (2 min) was significant (p <0.001).
The Mann-Whitney test was used to compare the groups of variables monitored throughout the procedure. The significance level was set at 5% ( Table 1).
The physician group had lower values for all parameters. All physicians obtained an appropriate contour at the edges, while only one-third of the students did so.
The GOALS score was lower for the student group in every domain, as well as in the consolidated total. We found a statistically significant difference between the groups for all assessed competences ( Table   2).
The distribution (as percentages) of the participants' responses to the Likert Scale is shown in Table 3.   18,27,28 .

DISCUSSION
In addition to manipulation of the simulator itself, it is also important to evaluate task performance.

Milcent et al. assessed the amount of area removed
from the menisci using software that provided precise measurements 18 . In the present study, the regularity of the incision and compliance with the stipulated ligament limit were assessed through visual verification and blindly to minimize assessment bias. Regarding compliance with the ligament limit, it was considered to have been exceeded or not, without quantifying the overshoot. The simplicity of this method distinguished between the two groups without requiring a specialist or specific software.
The participants were also evaluated using the GOALS method 8 . In all analyzed domains, there were significant differences between the physicians and the students, which complemented the construct validity.
The GOALS checklist can be applied in future spinal endoscopy studies to assess performance progress, either in simulated training or the real surgical environment.
A number of studies have shown the advantages of using simulators in medical education 23 .
In the present study, the Likert questionnaire results demonstrated that the simulator was well evaluated by about 94% of the participants, regarding its motivational quality, their interest in undergoing further simulated training in other surgical areas, and the realism of the prototype. More than 90% of the participants felt that the simulator could improve medical education due to its ability to stimulate student involvement, and the physicians recognized its potential for training young surgeons.
At the end of the procedure, only 30% of the physicians considered themselves able to perform a flavectomy using a real endoscopy, which demonstrates that isolated practice does not necessarily lead to proficiency. A study on simulator training with a decompression technique for the posterior cervical spine found that repetitive practice led to improved skill assessment scores for all participants 7 . It is believed that this simulator can demonstrate the acquired proficiency in future prospective studies assessing skill progression.
In the present study, we asked about whether the simulator replaces cadaver training, and resistance. Although this characteristic is more evident in simulations in cadavers, it can also occur in synthetic simulators. Some commercial synthetic models for laparoscopy have hardened tissue consistency, requiring sudden movements to perform dissection maneuvers 23 .
The present study confirmed the difficulty of faithfully reproducing the anatomical structures of soft tissues. The plastic mannequin, despite having a similar shape, has a different density and malleability than human tissue.
This issue was partially resolved by filling the mannequin with foam, which reproduced the musculature and satisfactorily guided the procedure.  13,29 . However, the ethical issues and increasing difficulties involved in cadaver training must be considered.
The simulator proved to be effective due to its low cost, easy reproduction, and portability. Its validation involved parameters that had been previously used in the literature and were well adapted for this study. The procedure was easily understood by the participants and the main hypothesis of the study was proven, i.e. that the model could differentiate experienced and inexperience groups of participants, providing construct validity to the simulator.
The present study has certain limitations.