Validation of an evaluation instrument for responders in tactical casualty care simulations* * The publication of this article in the Thematic Series “Human Resources in Health and Nursing: Training and Practice in the Americas” is part of Activity 2.2 of Reference Term 2 of the PAHO/WHO Collaborating Center for Nursing Research Development, Brazil.

Maria Del Carmen Usero-Pérez Maria Lourdes Jiménez-Rodríguez Alexandra González-Aguña Valentín González-Alonso Luis Orbañanos-Peiro Jose María Santamaría-García Jorge Luís Gómez-González About the authors

Abstracts

Objective:

to construct and validate a tool for the evaluation of responders in tactical casualty care simulations.

Method:

three rubrics for the application of a tourniquet, an emergency bandage and haemostatic agents recommended by the Hartford Consensus were developed and validated. Validity and reliability were studied. Validation was performed by 4 experts in the field and 36 nursing participants who were selected through convenience sampling. Three rubrics with 8 items were evaluated (except for the application of an emergency bandage, for which 7 items were evaluated). Each simulation was evaluated by 3 experts.

Results:

an excellent score was obtained for the correlation index for the 3 simulations and 2 levels that were evaluated (competent and expert). The mean score for the application of a tourniquet was 0.897, the mean score for the application of an emergency bandage was 0.982, and the mean score for the application of topical haemostats was 0.805.

Conclusion:

this instrument for the evaluation of nurses in tactical casualty care simulations is considered useful, valid and reliable for training in a prehospital setting for both professionals who lack experience in tactical casualty care and those who are considered to be experts.

Descriptors:
Rubric; Validation; Emergency Services; Simulation; Terrorism; Training


Objetivo:

construir e validar um instrumento de avaliação da prática, por meio da simulação, nos cuidados de saúde estratégicos.

Método:

três instrumentos para práticas de avaliação de aplicação do torniquete, bandagem de emergência e agente hemostático recomendados pelo Consenso de Hartford foram desenvolvidos e validados. A validade e a confiabilidade foram estudadas. A validação foi realizada por quatro especialistas da área e trinta e seis enfermeiros participantes selecionados por amostragem por conveniência. Três instrumentos de avaliação com 8 itens foram avaliados (com exceção da bandagem de emergência, que tinha 7 itens para avaliar). Cada prática foi avaliada por três especialistas.

Resultados:

uma pontuação excelente foi obtida no cálculo do índice de correlação para as três práticas e nos dois níveis avaliados (competente e especialista). A pontuação média para a aplicação do torniquete foi de 0,897, para o curativo de emergência foi de 0,982 e para a aplicação de agentes hemostáticos tópicos foi de 0,805.

Conclusão:

este instrumento de avaliação da prática por meio de simulação nos cuidados de saúde estratégicos é considerado útil, válido e confiável para o treinamento no contexto pré-hospitalar tanto dos profissionais que não possuem experiência nos cuidados estratégicos quanto nos considerados peritos.

Descritores:
Rubrica; Validação; Serviços de Emergência; Simulação; Terrorismo; Treinamento


Objetivo:

construir y validar un instrumento de evaluación de la práctica, mediante simulación, en la atención sanitaria táctica.

Método:

se construyeron y validaron tres rúbricas de las prácticas de aplicación del torniquete, vendaje de emergencia y agente hemostático recomendadas por el Consenso Hartford. Se estudió la validez y fiabilidad. La validación se realizó por cuatro expertos en la materia y treinta y seis participantes enfermeros que fueron la muestra de conveniencia. Se evaluaron tres rúbricas con 8 ítems (excepto para el vendaje de emergencia que fueron 7 ítems a evaluar). Cada práctica fue evaluada por tres expertos.

Resultados:

se ha obtenido una excelente puntuación en el cálculo del índice de correlación para las tres prácticas y en los dos niveles evaluados (competente y experto). La puntuación media para la rúbrica de aplicación del torniquete fue de 0.897, la del vendaje de emergencia 0.982 y para la aplicación de hemostáticos tópicos 0.805.

Conclusión:

este instrumento de evaluación de la práctica mediante simulación en la atención sanitaria se considera útil, válido y fiable para la formación en el entorno prehospitalario tanto de profesionales que carecen de experiencia en atención táctica como de aquellos considerados como expertos.

Descriptores:
Rúbrica; Validación; Servicios de Emergencias; Simulación; Terrorismo; Formación


Introduction

Prehospital care includes a wide range of health interventions, and the environment in which it is performed often complicates care because of the danger of staying on the scene (explosions, traffic accidents, extreme temperatures, etc.); therefore, the assessment must be done quickly(11 Abelsson A, Rystedt I, Suserud B-O, Lindwall L. Learning by simulation in prehospital emergency care - an integrative literature review. Scand J Caring Sci. [Internet] 2016 [cited 2018 Jun 15];30(2):234-40. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/scs.12252/ Subscription required
https://onlinelibrary.wiley.com/doi/abs/...
).

The rapid assessment of dangerous environments is one of the challenges in emergency care during terrorist attacks. Recently, terrorist attacks in different parts of the world have involved the use of explosives, vehicle collisions, and active shooters, demonstrating the need to develop medical care systems in “unsafe” environments. An incident with an active shooter is defined as “an individual actively engaged in killing or attempting to kill people in a confined and populated area; in most cases, active shooters use firearms(s), and there is no pattern or method to their selection of victims”(22 U.S. Departmanet of Homeland Security. Active shooter how to respond. [Internet]. Washington 2019 May 24 [cited 2018 Jun 10]. Available from: dhs.gov/publication/active-shooter-how-to-respond
dhs.gov/publication/active-shooter-how-t...
).

Prehospital health care must adapt its procedures so that interventions in this environment are performed with maximum safety, both for the patient and for the responding personnel. Based on this adequacy requirement, in 2013, recommendations of the Hartford Consensus(33 Pons PT, Jerome J, McMullen J, Manson J, Robinson J, Chapleau W. The Hartford Consensus on Active Shooters: implementing the continuum of prehospital trauma response. J Emerg Med. [Internet] 2015[cited 2018 Jun 15];49(6):878-85. Available from: https://www.jem-journal.com/article/S0736-4679(15)00943-9/fulltext
https://www.jem-journal.com/article/S073...
)were published. These procedures originated from the experience gained by military casualty care in recent years through the care of those wounded by firearms or explosives in military conflicts in areas such as Iraq or Afghanistan. During combat, caring for the wounded is carried out following the Tactical Combat Casualty Care (TCCC) recommendations, in which special importance is given to the treatment of external bleeding, airway obstruction and tension pneumothorax. Supported by this experience, the recommendations of the Hartford Consensus are applied to civilian prehospital care in risky environments such as terrorist attacks, emphasizing the importance of the rapid control of external bleeding of victims by the first responder (security forces, health personnel) as well as the rapid transfer of the wounded to a safe area and evacuation by medical services to a centre used to receive definitive treatment. The recommendations provide guidance to security forces for basic tactical casualty care(33 Pons PT, Jerome J, McMullen J, Manson J, Robinson J, Chapleau W. The Hartford Consensus on Active Shooters: implementing the continuum of prehospital trauma response. J Emerg Med. [Internet] 2015[cited 2018 Jun 15];49(6):878-85. Available from: https://www.jem-journal.com/article/S0736-4679(15)00943-9/fulltext
https://www.jem-journal.com/article/S073...
) (for our study, we define tactical casualty care as that carried out in risky environments with scarce resources, such as care provided during terrorist attacks). The recommendations of the Consensus establish 3 action zones as a function of environmental threat (hot, warm and cold zones), and tactical considerations are emphasized as determinants in prehospital care(44 Jacobs LM. Strategies to enhance survival in active shooter and intentional mass casualty events. Bull Am Coll Surg. [Internet] 2015 [cited 2018 Jun 20];100(1 Suppl):16-7. Available from: http://www.facsbulletin.com/acsbulletin/hartford-consensus?article_id=976592&pg=18#pg18
http://www.facsbulletin.com/acsbulletin/...
).

The recommendations of the Hartford Consensus should be adapted to the different health systems of each country, especially training in the application of tourniquets, haemostatic dressings and compressive bandages(33 Pons PT, Jerome J, McMullen J, Manson J, Robinson J, Chapleau W. The Hartford Consensus on Active Shooters: implementing the continuum of prehospital trauma response. J Emerg Med. [Internet] 2015[cited 2018 Jun 15];49(6):878-85. Available from: https://www.jem-journal.com/article/S0736-4679(15)00943-9/fulltext
https://www.jem-journal.com/article/S073...
).

Tourniquets are recommended for external bleeding from extremities when the control of bleeding is ineffective or impossible through direct pressure. Haemostatic agents should be used in combination with direct pressure to control major bleeding. Compressive bandages can be used to control external bleeding(55 Holcomb JB, Butler FK, Rhee P. Hemorrhage control devices: Tourniquets and hemostatic dressings. Bull Am Coll Surg. [Internet] 2015 [cited 2018 May 20];100(1 Suppl):66-70. Available from: http://www.facsbulletin.com/acsbulletin/hartford-consensus?article_id=976592&pg=68#pg68
http://www.facsbulletin.com/acsbulletin/...

6 Bennett BL, Littlejohn L. Review of new topical hemostatic dressings for combat casualty care. Military medicine. [Internet] 2014 [cited 2018 May 20];179(5):497-514. Available from: https://academic.oup.com/milmed/article/179/5/497/4160708/
https://academic.oup.com/milmed/article/...
-77 Jacobs LM. Bulletin reprint: The Hartford Consensus III: Implementation of Bleeding Control. Bull Am Coll Surg. [Internet] 2015[cited 2018 Jun 20];100(1 Suppl):40. Available from: http://www.facsbulletin.com/acsbulletin/hartford-consensus?article_id=976592&pg=42#pg42
http://www.facsbulletin.com/acsbulletin/...
). Nurses play fundamental roles in these types of incidents(88 Glasofer BA, Laskowski-jones L. Mass shootings?: A call for nursing awareness and action. [Internet] 2018 [cited 2018 Jul 10];13(5):14-20. Available from: https://journals.lww.com/nursingcriticalcare/Fulltext/2018/09000/Mass_shootings__A_call_for_nursing_awareness_and.4.aspx
https://journals.lww.com/nursingcritical...
), both as first responders, adapting their care to an environment with special characteristics, and in developing collaborations in medical care and in joint learning with other professionals. Nursing is positioned as a connecting link between medical care personnel(99 Mendes IAC, Ventura CAA, Trevizan MA, Marchi-Alves LM, de Souza-Junior VD. Educación, liderazgo y colaboraciones: potencialidades que la enfermería puede dar a la Cobertura Universal de Salud. Rev. Latino-Am. Enfermagem. [Internet] 2016 [cited 2019 Oct 4]; 24:e2673. Available from: http://www.scielo.br/pdf/rlae/v24/es_0104-1169-rlae-02673.pdf
http://www.scielo.br/pdf/rlae/v24/es_010...
).

One of the main points advocated is the development of training strategies for both first responders and professionals (health and non-health) involved in incidents in emergency environments to improve the survival of victims. Caring for victims is a responsibility shared by both the security forces involved and by the medical teams that care for them(33 Pons PT, Jerome J, McMullen J, Manson J, Robinson J, Chapleau W. The Hartford Consensus on Active Shooters: implementing the continuum of prehospital trauma response. J Emerg Med. [Internet] 2015[cited 2018 Jun 15];49(6):878-85. Available from: https://www.jem-journal.com/article/S0736-4679(15)00943-9/fulltext
https://www.jem-journal.com/article/S073...
).

Prehospital providers should be prepared to address situations of extreme urgency, but despite this premise, these situations represent a relatively low percentage of total care provided by these providers(1010 Alberdi F, García I, Atutxa L, Zabarte M. Epidemiología del trauma grave. Med Intensiva. [Internet] 2014 [cited 2018 Jul 10];38(9):580-8. Available from: http://www.medintensiva.org/es-epidemiologia-del-trauma-grave-articulo-S0210569114001806
http://www.medintensiva.org/es-epidemiol...
). In light of this knowledge, there is a need to provide experience in the care of seriously injured patients through other means, such as simulations(1111 Von Der Heyden M, Meissner K. Simulation in preclinical emergency medicine. Best Pract Res Clin Anaesthesiol. [Internet] 2015[cited 2018 Jun 20] ;29(1):61-8. Available from: https://www.sciencedirect.com/science/article/pii/S1521689615000026
https://www.sciencedirect.com/science/ar...
). For health care needed due to terrorist acts, improved training in controlling bleeding for the different responding teams, in providing early and effective triage and in simulations of incident management for multiple victims is necessary(1212 Turner CDA, Lockey DJ, Rehn M. Pre-hospital management of mass casualty civilian shootings: a systematic literature review. Crit Care. [Internet] 2016 [cited 2018 Jul 20];20(1):362. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5101656/
https://www.ncbi.nlm.nih.gov/pmc/article...
).

Well-trained responders involved in the control of external bleeding increases the chances of the survival of victims of a terrorist attack(1313 Hirsch M, Carli P, Nizard R, Riou B, Baroudjian B, Baubet T, et al. The medical response to multisite terrorist attacks in Paris. Vol. 386, The Lancet. [Internet] 2015[cited 2018 Jun 12];19;386(10012):2535-8. Available from: https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(15)01063-6/fulltext
https://www.thelancet.com/journals/lance...
).

In the same way that clinical staff perform evidence-based care, educators should follow the same pattern. Too much interest in learning from training results can lead to neglecting the development of valid instruments for obtaining the results. If the information for medical care training in emergency environments is obtained through reliable and valid tests, policies and programmes can be developed to improve the training. Validation is necessary to ensure that the results obtained are correct and to obtain assessment instruments that adapt to the needs of instructors(1414 Cook DA, Hatala R. Validation of educational assessments?: a primer for simulation and beyond. Adv Simul. [Internet] 2016 [cited 2018 Jun 12];1-12. Available from: https://advancesinsimulation.biomedcentral.com/articles/10.1186/s41077-016-0033-y
https://advancesinsimulation.biomedcentr...
).

The absence of similar studies and the need for the evaluation of training lead to the objective of this study: to construct and validate a tool for the evaluation of responders in tactical casualty care simulations.

Method

A validation study of an assessment instrument was designed for the following tactical casualty care simulations: application of a tourniquet, application of haemostatic dressings, application of an emergency bandage.

These tasks were chosen because they are basic techniques that should be known by all first responders(77 Jacobs LM. Bulletin reprint: The Hartford Consensus III: Implementation of Bleeding Control. Bull Am Coll Surg. [Internet] 2015[cited 2018 Jun 20];100(1 Suppl):40. Available from: http://www.facsbulletin.com/acsbulletin/hartford-consensus?article_id=976592&pg=42#pg42
http://www.facsbulletin.com/acsbulletin/...
).

The study consisted of 3 phases.

  1. Phase 1 consisted of the development of an assessment instrument for each of the 3 tasks. These instruments were obtained in 3 steps.

To compile the different items that make up each instrument, information was obtained from the observation of cognitive tasks by 2 experts in the subject and was supported with reference manuals.

Once the rubrics were created, estimation of the construct validity was performed using expert-based judgement and the use of statistical methods derived from the application of the measurement instrument, such as the content validity index (CVI)(1515 Pedrosa I, Suárez-Álvarez J, García-Cueto E. Evidencias sobre la validez de contenido: avances teóricos y métodos para su estimación. Acción Psicol. [Internet] 2013 [cited 2018 Jun 12];10(2):3-18. Available from: http://scielo.isciii.es/pdf/acp/v10n2/02monografico2.pdf
http://scielo.isciii.es/pdf/acp/v10n2/02...
).

The experts who were selected to establish construct validity were 4 instructors from the Military Medical Care School (Escuela Militar de Sanidad, EMISAN) with recognized experience in the field (more than 3 years of teaching experience and conducting advanced training courses, with publications and teacher accreditations, in the subject in question). The judges were selected by convenience sampling.

Based on the levels of expertise in the Benner model(1616 Algarra-Carrillo, Julia A, Serrano G, Orjuela LC, Marcela C, Sánchez D, et al. La filosofía de Patricia Benner y la práctica clínica. Enfermería Glob. [Internet]. 2013 Oct [cited 2019 Jun 20]; 12(32): 346-61. Available from http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1695-61412013000400021&lng=es.
http://scielo.isciii.es/scielo.php?scrip...
) (beginner, advanced beginner, competent, efficient and expert), 3 levels of training were proposed that corresponded to first responders (beginner), medical personnel with no experience in emergencies (competent) and medical staff with experience in prehospital care (expert). Thus, these 3 levels include practically all professionals who must be trained for interventions in this emergency environment. For this study, due to access to the sample, only validation was carried out at the competent and expert levels. The experts (selected by convenience) had to assess the inclusion of each item in the instruments for the 3 levels of knowledge proposed (beginner, competent and expert).

In addition to proposing inclusion for each of the levels, the experts assessed each item individually to determine whether it was appropriate and if it should be retained in the final version. Each item was assigned a score based on 3 possibilities: essential to evaluate the construct, useful but expendable, or unnecessary.

Finally, the CVI was calculated for the instrument as a whole. Because there were 4 experts, for an item to be included, a minimum score of 0.9 was needed(1515 Pedrosa I, Suárez-Álvarez J, García-Cueto E. Evidencias sobre la validez de contenido: avances teóricos y métodos para su estimación. Acción Psicol. [Internet] 2013 [cited 2018 Jun 12];10(2):3-18. Available from: http://scielo.isciii.es/pdf/acp/v10n2/02monografico2.pdf
http://scielo.isciii.es/pdf/acp/v10n2/02...
).

For those items that did not reach the minimum score, a new review was performed, obtaining a definitive version after reaching total agreement among the instructors who performed the assessment.

  1. Phase 2: To prepare the evaluation, a preliminary meeting was held to resolve item disagreements between evaluators. A pilot study was conducted prior to the evaluation study using 15 nursing students from EMISAN (professionals with a bachelor’s degree or diploma in nursing carrying out technical training prior to committing to a military career) considered as a “competent” group. During simulations, each item was evaluated independently by the 3 instructors. This test served to familiarize instructors with the evaluation instrument and to modify items if necessary.

  2. Phase 3: To calculate reliability, Cronbach’s alpha and the correlation coefficient index (CCI) were determined.

A Cronbach’s alpha value above 0.70 is considered acceptable. CCI values below 0.40 represent low reliability, between 0.40 and 0.75 represent fair to good reliability, and above 0.75 represent excellent reliability(1717 Cortés-Reyes ÉT, Andrés Rubio-Romero J, Gaitán-Duarte H. Statistical methods for evaluating diagnostic test agreement and reproducibility. Rev Colomb Obstet Ginecol. [Internet] 2010 [cited 2018 Jun 12];61(3):247-55. Available from: http://www.scielo.org.co/scielo.php?script=sci_arttext&pid=S0034-74342010000300.
http://www.scielo.org.co/scielo.php?scri...
).

Agreement was calculated using the CCI during the simulation of 3 tasks, such as hot zone application of a tourniquet and application of an emergency bandage and haemostatic dressings in a non-threatening environment.

The sample was defined for the 2 levels that were to be evaluated: competent (health professionals without experience in prehospital care) and expert (health professionals with experience in prehospital care). For the present study, a sample of 26 nursing students (competent level) from EMISAN were selected by convenience sampling, in addition to 10 students who participated in advanced training courses at the school (expert level). The latter had experience in international missions and provided similar training to these simulations. Each of the students (competent and expert) was evaluated by the 3 instructors, with a grade of YES/NO (depending on whether they completed the evaluated task). If the simulation was completed correctly, the maximum score was 10. Some of the items are tasks whose performance is considered essential.

The test was developed as follows:

For the competent level, the participants were provided with 2 hours of training in which the procedures for correct application of each device were explained. Two days later and in an EMISAN classroom, the 3 tasks were performed by each student.

Three stations were created, with each station evaluated by 3 observers.

For the application of a tourniquet, a Tourniquet in Emergency (TIE™) model was used; this task is habitually practised at EMISAN. The tourniquet was self-applied to the arm. A box of gloves was provided.

For the application of an emergency bandage, the participant decided between an arm or leg bandage. A box of gloves was provided.

For the application of a haemostatic dressing, Celox Gauze™ (gauze bandage) was used, in addition to a box of gloves and a low-cost thigh wound simulator.

For the expert level, no prior training was given because they were considered experts in the field. The simulation was conducted in the same way as in the previous group.

To carry out this study, permission was requested from the Central Defense Academy for the collection and analysis of the results. All participants gave their consent to participate in the study.

Calculations were performed using International Business Machines Statistical Package for the Social Sciences (IBM SPSS) 24.

Results

  1. Phase 1 Results: After calculating the CVI for the rubric for the application of a tourniquet, 3 items were extracted that did not reach the minimum desired score (0.9). After a second meeting in which a final drafting was carried out, the rubric to validate was unanimously accepted (Tables 1 and 2).

    Table 1
    Results of Phase 1 of the study in which the validity of the evaluation instrument for the application of a tourniquet was evaluated. Madrid, CM, Spain, 2018
    Table 2
    Results of Phase 1 in which the validity of the evaluation instrument for the application of haemostats and emergency bandage was evaluated. Madrid, CM, Spain, 2018

For the rubric for emergency bandaging, the same procedure was performed: an item was removed that did not achieve the minimum score, and after applying the same test on volunteers, 7 items that constituted the rubric were considered for inclusion.

The evaluation of the application of haemostatic agents led to the removal of 2 items that did not reach the minimum score for inclusion, in addition to an editorial review; therefore, 8 items were used to assess this task.

  1. During Phase 2 of the study, no changes were made to the rubric.

  2. Results of Phase 3: The sociodemographic data for the participants are provided in Table 3.

    Table 3
    Sociodemographic data for the participants. Madrid, CM, Spain, 2018

The CCI was adequate for most items, as shown in Tables 4 and 5.

Table 4
Correlation coefficient index (CCI*) results for the application of a tourniquet (competent and expert levels). Madrid, CM, Spain, 2018
Table 5
Correlation coefficient index (CCI*) results for the application of a haemostatic agent and an emergency bandage (competent and expert levels). Madrid, CM, Spain, 2018

The data obtained for items 2, 3 and 8 for the application of haemostats did not reach the minimum recommended score. In a subsequent analysis, combining the competent and expert level groups as a single sample, the score obtained indicated good reliability; the lowest values were obtained for item 3, with a CCI of 0.424, a significance of 0.047 and a Cronbach’s alpha of 0.437. This calculation was performed with the assumption that the items that were assessed and the test that was evaluated were exactly the same for the 2 levels of training; therefore, the calculation was performed with a pooled competent and expert level sample (n = 36).

To calculate the means of the CCI for the competent and expert levels, the competent sample was added to the expert sample (n = 36); the rubrics for the 3 tasks had excellent reliability (0.897 for the application of a tourniquet, 0.982 for the application of an emergency bandage and 0.805 for the application of haemostatic dressings).

Discussion

The purpose of this study was to construct and validate a tool for the evaluation of responders in tactical casualty care simulations. The teaching activities carried out at EMISAN made it necessary to obtain valid instruments for evaluating training. Medical training is provided not only to military health personnel but also to health professionals in prehospital emergency medical services that provide tactical casualty care(1818 Usero-Pérez C, González Alonso V, Orbañanos Peiro L, Gómez Crespo JM, Hossain López, S. Implementación de las recomendaciones del Consenso Hartford y TECC en los servicios de emergencia: revisión bibliográfica. Emergencias. [Internet].2017;20:29(6) [cited 2018 Jun 15];20:29(6). Available from: http://emergencias.portalsemes.org/descargar/implementacin-de-las-recomendaciones-del-consenso-de-hartford-y-tactical-emergency-casualty-care-tecc-en-los-servicios-de-emergencia-revisin-bibliogrfica/
http://emergencias.portalsemes.org/desca...
).

No studies of these characteristics have been found for application in civilian prehospital care. Other published studies have validated evaluation instruments, for example, for simulations of external bleeding control by tourniquets in the military environment; however, this was done during the care under fire phase (threat to the responder and the patient), which differs from our study in that the simulations were performed during the threat-free phase (warm or cold zone according to the Hartford Consensus), which was not considered in the above studies(1919 Schmitz CC, Chipman JG, Yoshida K, Vogel RI, Sainfort F, Beilman G, et al. Reliability and Validity of a Test Designed to Assess Combat Medics' Readiness to Perform Life-Saving Procedures. Mil Med. [Internet] 2014[cited 2018 Jun 18];179(1):42-8. Available from: https://academic.oup.com/milmed/article/179/1/42/4160703/
https://academic.oup.com/milmed/article/...
-2020 Hart D, Clinton J, Anders S, Reihsen T, McNeil MA, Rule G, et al. Validation of an Assessment Tool for Field Endotracheal Intubation. Mil Med. [Internet] 2016 [cited 2018 Jun 15];181(11):e1484-90. Available from: https://academic.oup.com/milmed/article/181/11-12/e1484/4158575/
https://academic.oup.com/milmed/article/...
). The novelty of this study is the validation of the 3 tasks, which are fundamental in controlling bleeding both in the civilian and military environments.

It is important to train responders in these tasks because not being familiar with them implies they will not be used in situations where they could be of great utility, decreasing the chances of survival(2121 Beaucreux C, Vivien B, Miles E, Ausset S, Pasquier P. Application of tourniquet in civilian trauma: Systematic review of the literature. Anaesth Crit Care Pain Med. [Internet] 2018 [cited 2018 Jun 15]1;37(6):597-606. Available from: https://www.sciencedirect.com/science/article/abs/pii/S2352556817302655
https://www.sciencedirect.com/science/ar...
). Valid evaluation instruments are necessary for the education of health professionals. As in our study, two-thirds of validation studies use internal consistency to validate assessment instruments(2222 Cook DA, Zendejas B, Hamstra SJ, Hatala R, Brydges R. What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment. Adv Heal Sci Educ. [Internet]2014 [cited 2018 Jun 18];19(2):233-50. Available from: https://link.springer.com/article/10.1007%2Fs10459-013-9458-4/ Subscription required
https://link.springer.com/article/10.100...
).

The results of our study show that the instrument presented is valid and reliable in the evaluation of simulations necessary for training regarding external bleeding control in incidents involving multiple victims. The internal consistency values were excellent (value of 1). Likewise, the reliability supports the use of the proposed rubric as an evaluation tool: the CCI results for each item are above the value considered good (0.4), except one (item 3 in the application of topical haemostats). These value can result from the items being developed for integration into simulations of different situations in which it is necessary to consider whether a tourniquet is necessary. The results for this item improved after increasing the sample and considering the competent and expert levels as a single sample. Likewise, the results for 2 other items (2 and 8) improved, as well as correlation index scores, when combining the samples. For 37% of the items, a correlation of 100% was obtained, most of which were found in the application of an emergency bandage simulation. This good result could be explained by the simplicity of the technique and the low degree of difficulty in its implementation.

We highlight the excellent values obtained for the average CCI for the 3 tasks (0.897, 0.982 and 0.805), which is why we consider that the evaluation instruments proposed are reliable for evaluating students being trained in the aforementioned tasks.

This work contributes to the development and validation of a rubric for the evaluation of tactical training in new scenarios, such as mass-casualty incidents, for prehospital care providers.

The results obtained are in accordance with other proposals(1111 Von Der Heyden M, Meissner K. Simulation in preclinical emergency medicine. Best Pract Res Clin Anaesthesiol. [Internet] 2015[cited 2018 Jun 20] ;29(1):61-8. Available from: https://www.sciencedirect.com/science/article/pii/S1521689615000026
https://www.sciencedirect.com/science/ar...
) that suggest that checklists during simulations can help to prioritize the sequence of the participant’s performance in simulated scenarios in which several tasks must be developed at the same time. The scenarios that are part of the simulation should include a number of components such as learning objectives, assessment tools, pre-screening, and debriefing, which are essential parts of the validation of assessment tools through peer review, clinical experts, evidence review and evaluation to establish usefulness of the simulation in the process of student training(2323 Waxman KT. The Development of Evidence-Based Clinical Simulation Scenarios: Guidelines for Nurse Educators. J Nurs Educ. [Internet]. [Internet] 2010 [cited 2018 Jun 21];49(1):29-35. Available from: https://www.healio.com/nursing/journals/jne/2010-1-49-1/%7B4c2d5f4c-18b9-455f-89cf-2e8d6797adc5%7D/the-development-of-evidence-based-clinical-simulation-scenarios-guidelines-for-nurse-educators
https://www.healio.com/nursing/journals/...
).

The limitations of this study is sample size. The difficulty in obtaining an adequate sample size lies in the fact that, to date, the professionals who have used these instruments to control bleeding are in the Army. The geographical dispersion of these professionals after their completion of the courses in EMISAN provides limited opportunities to include these individuals in these studies. On the other hand, the unique experience of these professionals allows us to obtain reliable data for the study of an instrument for the evaluation of simulations in a threatening environment.

Future research should aim to develop evaluation instruments for simulations in which the environment exerts a great influence on the type of care: assessing safety, assessing the type of danger (explosions, firearms, etc.), determining necessary equipment, etc. Simulation training for this type of incident should recreate environments similar to those in which prehospital tactical care will be developed.

Validation is considered useful for both civilian and military training because these 3 techniques are used in the same way in both environments.

The authors continue the work to implement training in the general population. Validation of outcome assessment tools will help develop programmes that are useful and efficient. The Hartford Consensus recognizes the importance of providing this training to both health professionals and to non-health personnel who, because of their profession or activity, can become first responders.

Conclusion

In conclusion, we emphasize that this study presents consistent results that support the use of our rubric for the evaluation of the application of tourniquets during threatening civilian environments for the evaluation of the application emergency bandages and haemostatic dressings under non-threatening conditions.

These assessment instruments are valid and reliable; therefore, their use is recommended to reliably evaluate training results from the simulations described above in both the military and civilian environments.

  • *
    The publication of this article in the Thematic Series “Human Resources in Health and Nursing: Training and Practice in the Americas” is part of Activity 2.2 of Reference Term 2 of the PAHO/WHO Collaborating Center for Nursing Research Development, Brazil.

References

Publication Dates

  • Publication in this collection
    17 Apr 2020
  • Date of issue
    2020

History

  • Received
    13 Dec 2018
  • Accepted
    23 Oct 2019
Escola de Enfermagem de Ribeirão Preto / Universidade de São Paulo Av. Bandeirantes, 3900, 14040-902 Ribeirão Preto SP Brazil, Tel.: +55 (16) 3315-3451 / 3315-4407 - Ribeirão Preto - SP - Brazil
E-mail: rlae@eerp.usp.br