Validation of an evaluation instrument for responders in tactical casualty care simulations

Objective: to construct and validate a tool for the evaluation of responders in tactical casualty care simulations. Method: three rubrics for the application of a tourniquet, an emergency bandage and haemostatic agents recommended by the Hartford Consensus were developed and validated. Validity and reliability were studied. Validation was performed by 4 experts in the field and 36 nursing participants who were selected through convenience sampling. Three rubrics with 8 items were evaluated (except for the application of an emergency bandage, for which 7 items were evaluated). Each simulation was evaluated by 3 experts. Results: an excellent score was obtained for the correlation index for the 3 simulations and 2 levels that were evaluated (competent and expert). The mean score for the application of a tourniquet was 0.897, the mean score for the application of an emergency bandage was 0.982, and the mean score for the application of topical haemostats was 0.805. Conclusion: this instrument for the evaluation of nurses in tactical casualty care simulations is considered useful, valid and reliable for training in a prehospital setting for both professionals who lack experience in tactical casualty care and those who are considered to be experts.


Introduction
Prehospital care includes a wide range of health interventions, and the environment in which it is performed often complicates care because of the danger of staying on the scene (explosions, traffic accidents, extreme temperatures, etc.); therefore, the assessment must be done quickly (1) .
The rapid assessment of dangerous environments is one of the challenges in emergency care during   (3) .
Tourniquets are recommended for external bleeding from extremities when the control of bleeding is ineffective or impossible through direct pressure.
Haemostatic agents should be used in combination with direct pressure to control major bleeding. Compressive bandages can be used to control external bleeding (5)(6)(7) .
Nurses play fundamental roles in these types of incidents (8) , both as first responders, adapting their care to an environment with special characteristics, and in developing collaborations in medical care and in joint learning with other professionals. Nursing is positioned as a connecting link between medical care personnel (9) .
One of the main points advocated is the development of training strategies for both first responders and professionals (health and non-health) involved in incidents in emergency environments to improve the survival of victims. Caring for victims is a responsibility shared by both the security forces involved and by the medical teams that care for them (3) .
Prehospital providers should be prepared to address situations of extreme urgency, but despite this premise, these situations represent a relatively low percentage of total care provided by these providers (10) . In light of this knowledge, there is a need to provide experience in the care of seriously injured patients through other means, such as simulations (11) .
For health care needed due to terrorist acts, improved training in controlling bleeding for the different responding teams, in providing early and effective triage and in simulations of incident management for multiple victims is necessary (12) .
Well-trained responders involved in the control of external bleeding increases the chances of the survival of victims of a terrorist attack (13) .
that the results obtained are correct and to obtain assessment instruments that adapt to the needs of instructors (14) .
The absence of similar studies and the need for the evaluation of training lead to the objective of this study: to construct and validate a tool for the evaluation of responders in tactical casualty care simulations.

Method
A validation study of an assessment instrument was designed for the following tactical casualty care simulations: application of a tourniquet, application of haemostatic dressings, application of an emergency bandage.
These tasks were chosen because they are basic techniques that should be known by all first responders (7) .
The study consisted of 3 phases. To compile the different items that make up each instrument, information was obtained from the observation of cognitive tasks by 2 experts in the subject and was supported with reference manuals.
Once the rubrics were created, estimation of the construct validity was performed using expert-based judgement and the use of statistical methods derived from the application of the measurement instrument, such as the content validity index (CVI) (15) .  In addition to proposing inclusion for each of the levels, the experts assessed each item individually to determine whether it was appropriate and if it should be retained in the final version. Each item was assigned a score based on 3 possibilities: essential to evaluate the construct, useful but expendable, or unnecessary.
Finally, the CVI was calculated for the instrument as a whole. Because there were 4 experts, for an item to be included, a minimum score of 0.9 was needed (15) .
For those items that did not reach the minimum score, a new review was performed, obtaining a definitive version after reaching total agreement among the instructors who performed the assessment. A Cronbach's alpha value above 0.70 is considered acceptable. CCI values below 0.40 represent low reliability, between 0.40 and 0.75 represent fair to good reliability, and above 0.75 represent excellent reliability (17) .
Agreement was calculated using the CCI during the simulation of 3 tasks, such as hot zone application of a tourniquet and application of an emergency bandage and haemostatic dressings in a non-threatening environment.
The sample was defined for the 2 levels that were to be evaluated: competent (health professionals without experience in prehospital care) and expert (health professionals with experience in prehospital care). For the present study, a sample of 26 nursing students (competent level) from EMISAN were selected by convenience sampling, in addition to 10 students who participated in advanced training courses at the school (expert level). The latter had experience in international missions and provided similar training to these simulations. Each of the students (competent and expert) was evaluated by the 3 instructors, with a grade of YES/NO (depending on whether they completed the evaluated task). If the simulation was completed Rev. Latino-Am. Enfermagem 2020;28:e3251.
correctly, the maximum score was 10. Some of the items are tasks whose performance is considered essential.
The test was developed as follows: For the competent level, the participants were provided with 2 hours of training in which the procedures for correct application of each device were explained.
Two days later and in an EMISAN classroom, the 3 tasks were performed by each student.
Three stations were created, with each station evaluated by 3 observers.
For the application of a tourniquet, a Tourniquet in Emergency (TIE™) model was used; this task is habitually practised at EMISAN. The tourniquet was selfapplied to the arm. A box of gloves was provided.
For the application of an emergency bandage, the participant decided between an arm or leg bandage. A box of gloves was provided.
For the application of a haemostatic dressing, Celox Gauze™ (gauze bandage) was used, in addition to a box of gloves and a low-cost thigh wound simulator.
For the expert level, no prior training was given because they were considered experts in the field. The simulation was conducted in the same way as in the previous group.
To carry out this study, permission was requested from the Central Defense Academy for the collection and analysis of the results. All participants gave their consent to participate in the study.
Calculations were performed using International Business Machines Statistical Package for the Social Sciences (IBM SPSS) 24.

Results
Phase 1 Results: After calculating the CVI for the rubric for the application of a tourniquet, 3 items were extracted that did not reach the minimum desired score (0.9). After a second meeting in which a final drafting was carried out, the rubric to validate was unanimously accepted (Tables 1 and 2). Usero-Pérez C, Jiménez-Rodríguez ML, González-Aguña A, González-Alonso V, Orbañanos-Peiro L, Santamaría-García JM, Gómez-González JL. For the rubric for emergency bandaging, the same procedure was performed: an item was removed that did not achieve the minimum score, and after applying the same test on volunteers, 7 items that constituted the rubric were considered for inclusion.
The evaluation of the application of haemostatic agents led to the removal of 2 items that did not reach the minimum score for inclusion, in addition to an editorial review; therefore, 8 items were used to assess this task.
During Phase 2 of the study, no changes were made to the rubric.

Results of Phase 3:
The sociodemographic data for the participants are provided in Table 3.
The CCI was adequate for most items, as shown in Tables 4 and 5. Rev. Latino-Am. Enfermagem 2020;28:e3251.  The data obtained for items 2, 3 and 8 for the application of haemostats did not reach the minimum recommended score. In a subsequent analysis, combining the competent and expert level groups as a single sample, the score obtained indicated good reliability; the lowest values were obtained for item 3, with a CCI of 0.424, a significance of 0.047 and a Cronbach's alpha of 0.437.
This calculation was performed with the assumption that the items that were assessed and the test that was evaluated were exactly the same for the 2 levels of training; therefore, the calculation was performed with a pooled competent and expert level sample (n = 36).
To calculate the means of the CCI for the competent and expert levels, the competent sample was added to the expert sample (n = 36); the rubrics for the 3 tasks had excellent reliability (0.897 for the application of a tourniquet, 0.982 for the application of an emergency bandage and 0.805 for the application of haemostatic dressings).

Discussion
The purpose of this study was to construct and validate a tool for the evaluation of responders in tactical casualty care simulations. The teaching activities carried out at EMISAN made it necessary to obtain valid instruments for evaluating training. Medical training is provided not only to military health personnel but also to health professionals in prehospital emergency medical services that provide tactical casualty care (18) .  (19)(20) . The novelty of this study is the validation of the 3 tasks, which are fundamental in controlling bleeding both in the civilian and military environments.
It is important to train responders in these tasks because not being familiar with them implies they will not be used in situations where they could be of great utility, decreasing the chances of survival (21) . Valid evaluation instruments are necessary for the education of health professionals. As in our study, two-thirds of validation studies use internal consistency to validate assessment instruments (22) . Rev. Latino-Am. Enfermagem 2020;28:e3251.
The authors continue the work to implement training in the general population. Validation of outcome assessment tools will help develop programmes that are useful and efficient. The Hartford Consensus recognizes the importance of providing this training to both health professionals and to non-health personnel who, because of their profession or activity, can become first responders.

Conclusion
In conclusion, we emphasize that this study presents consistent results that support the use of our rubric for the evaluation of the application of tourniquets during threatening civilian environments for the evaluation of the application emergency bandages and haemostatic dressings under non-threatening conditions.
These assessment instruments are valid and reliable; therefore, their use is recommended to reliably evaluate training results from the simulations described above in both the military and civilian environments.