Validation of a specific checklist for service cardiopulmonary resuscitation: shockable rhythmsABSTRACT
Introduction: The use of checklists in simulation training aims to support, improve or change the operator’s cognitive processes.
Objective: to develop a checklist for cardiopulmonary arrest (CPA) care using a Likert scale aiming to evaluate performance in CPA care.
Method: The checklist was created based on items selected and adapted by experts from the Advanced Cardiovascular Life Support (ACLS) training guide. It was tested based on the training video that involved CPA in the ACLS Megacode format based on a pre-established clinical case. After a new test with the same training video, the final version of the checklist was approved by the experts.
Results: Of the 37 evaluated questions, 30 (81.1%) had the maximum score assigned by the three evaluators, 6 (16.2%) had at least one score of three and only one (2.7%) had a score of one. The final percentage of maximum scores assigned by the evaluators was 91.9%, exceeding the target of 85%.
Conclusion: This study assesses a checklist developed based on the 2015 AHA guidelines and proved to be reliable, since the agreement between the evaluators was high.
Keywords:
Simulation Training; High Fidelity Simulation Training; Advanced Cardiovascular Life Support
RESUMO
Introdução: O uso do checklist no treinamento de simulação visa apoiar, melhorar ou alterar os processos cognitivos do operador.
Objetivo: Este estudo teve como objetivo elaborar um checklist para o atendimento de parada cardiorrespiratória (PCR) utilizando uma escala de Likert a fim de avaliar o desempenho no atendimento de PCR.
Método: A criação do checklist foi baseada em itens selecionados e adaptados por experts do guia de treinamento de Suporte Avançado de Vida (SAVC). Para testar o checklist, utilizou-se o vídeo de treinamento de atendimento que envolvia a PCR nos moldes do SAVC Megacode baseado em um caso clínico preestabelecido. Após um novo teste no mesmo vídeo de treinamento, a versão final do checklist foi aprovada pelos experts.
Resultado: Das 37 questões avaliadas, 30 (81,1%) tiveram a nota máxima atribuída pelos três avaliadores, seis (16,2%) tiveram pelo menos uma nota três e apenas uma (2,7%) apresentou nota um. O percentual final de notas máximas atribuídas pelos avaliadores foi de 91,9%, superando a meta de 85%.
Conclusão: Este estudo se refere a um checklist elaborado com base nas diretrizes de 2015 da American Heart Association (AHA) e se mostrou confiável, uma vez que a concordância entre os avaliadores foi elevada.
Palavras-chave:
Simulação de Alta Fidelidade; Suporte Avançado de Vida Cardiovascular; Educação Médica; Emergências Cardiovasculares; Checklists
INTRODUCTION
Checklists are used when the order of steps during task performance is important and work as an aid in evaluating simulation scenarios and supporting the operator’s potential memory. They need to be performed in a sequential order, which generally does not allow the substitution of one step by another1. Checklists act as memory aids to ensure that all relevant criteria are considered, and also as decision aids when making objective judgments. As such, it reduces the influence of favorable overall impression affecting the judgment of specific characteristics1)-(3.
Simulation has been used to support teaching, promote critical thinking among students, and improve the quality of patient care4. When appropriately conducted, simulation creates an ideal educational environment, because learning activities are predictable, consistent, standardized, safe, and reproducible5.
The definition of valid, reliable and generalizable standards for student performance assessment is important in the evaluation of basic competence and mastery of medical education. Regarding advanced cardiovascular life support (ACLS), the performance standards are those recommended by the published guidelines for patient management6.
The American Heart Association (AHA) ACLS training program was first conducted in 1974 and is currently a training course used to teach the skills and knowledge needed to respond to cardiopulmonary emergencies7. The ACLS course arose from the need to train nurses, physicians, and respiratory physiotherapists for the initial care of a cardiovascular emergency8.
Previous studies have described the process by which ACLS checklists can be validated and suggest that minimum passing scores for team leader performance can be determined for the ACLS test 9.
The objective of this study was to develop a sequential, linear checklist using a 4-point Likert scale and to use it for a detailed evaluation of the cardiorespiratory arrest (CPA) care team performance, considering the team and team members.
METHOD
The checklist used in this study was constructed following the steps described in studies on the development of checklists, as well as studies by experts in the design of checklists10)-(13.
Aiming to evaluate the teaching-learning process of clinical skills in simulation training, a checklist was created, which represented the scenario prepared by the researchers for CPA care programmed in the LLEAP (Laerdal Learning Application) software. The checklist content was based on the International Guidelines on Cardiovascular Emergencies and Cardiopulmonary Resuscitation, established by the International Alliance of Resuscitation Committees (ILCOR) and the 2015 AHA Guidelines for Cardiorespiratory Resuscitation and Emergency Cardiovascular Care13. The content of the 2015 AHA guidelines was evaluated by a group of simulation experts, who could propose the inclusion of new items or the modification of previously selected items. The checklist was created and based on the standard checklist used by the AHA 2015 for CPA care for shockable rhythms (pulseless VF/VT) and the checklist developed by McEVoy et al (2012) in addition to experts’ suggestions. Specific items were selected for the selected CPA scenario. In order to allow the participants’ evaluation, both individually and collectively, the checklist was divided according to the position occupied by the team member: 1 - Team Leader, 2 - Compression rescuer, 3 - Ventilation rescuer, 4 - Medication and time rescuer, and 5 - Team. The items were ordered in the logical order of evaluation of CPA care. A Likert scale with 4 grading points was used for detailed evaluation of the performance of the skills needed in the scenario: 1-Did not perform the step; 2-Performed the step, but in an incomplete and insecure way; 3-Performed the step completely and insecurely; 4-Performed the step completely and safely. The checklist had 37 items: Leader (17), Compression rescuer (9), Ventilation rescuer (4), Medication and time rescuer (4) and team (3). The maximum score for the Team Leader is 68 points, for the Compression Rescuer (36 points), Ventilation Rescuer (16 points), Medication and Time Rescuer (16 points) and Team (12 points). The checklist was translated into Brazilian Portuguese and the version was presented to three faculty experts from the School of Medical Sciences-MG (FCM-MG) involved with simulation education and training/research for at least 3 years, and who teach crisis management simulation courses, including ACLS, at least 5 times a year. The experts evaluated the checklist regarding the content and order of the items and were able to make suggestions to change the order of items in addition to adding or excluding items. The selected items, the translated version, and the changes proposed by the experts are shown in Table 1.
The care training video was prepared by the researchers and involved CPA along the lines of the ACLS Megacode to contemplate a clinical case and proposed care. The scenario was programmed using the LLEAP software and performed using the SimMan 3G (Laerdal) manikin and performed in the simulation laboratory of the School of Medical Sciences of Minas Gerais (FCM-MG) in a patient care room that simulates a hospital admission room, with a cardiorespiratory arrest cart and standardized manual defibrillator. The participants in the training video are members of the institution’s faculty and certified by ACLS and were not the same experts who participated in the evaluation of the scenario using the checklist. The video lasted 14 minutes and 54 seconds. The scenario management video was recorded using the SIMVIEW system.
A pre-test was carried out, used to evaluate the clinical ability in the care training video that involved CPA along the lines of the ACLS Megacode with the clinical case presented. The experts performed the evaluation separately to allow a standard performance rating and were informed about the order in which to evaluate the simulation video after the scenario was presented.
To evaluate the construct validity of the checklist, which refers to the degree to which an instrument measures what it intends to measure, the agreement in the distribution of the scores assigned by item by the three evaluators was verified, as well as the final distribution of all evaluations. A goal of 85% agreement between the evaluators of score four (equivalent to the best evaluation) was established, considering that the participants in the training video are simulation experts and ACLS certified. Frequency tables were used for categorical variables. To analyze the assessment instrument test, a table was constructed with the distribution of the scores assigned by the three evaluators per item, as well as the final distribution of all the evaluations. A goal of 85% of scores four (equivalent to the best evaluation) was established to consider the instrument appropriate for evaluation.
This project was approved by the Research Ethics Committee of the institution CPG - Postgraduate Center of the School of Medical Sciences of Minas Gerais-FCM-MG in accordance with the Declaration of Helsinki, under CAAE number 91179218.8.0000.5134, approved on August 18, 2018.
RESULTS
Checklist preparation
The creation of the checklist went through the following steps: (1) selection of items from the AHA 2015 Checklist (Chart 1); (2) Selection of items from the McEVoy 2012 Checklist (Chart 2); (3) translation of the items of the McEVoy 2012 Checklist into Brazilian Portuguese (Chart 2); (4) Evaluation of the items by the experts and suggestion of changes to the previously selected items, in addition to the addition of new items based on the concepts of the AHA 2015 guideline (Chart 1 and Chart 2); (5) Pre-validation of the Checklist using the care training video; (6) Suggestion by the experts to include new items and modify pre-existing items (7) and final validation of the Checklist using the care training video again. Finally, in the final validation, the experts approved the selected items and the sequence presented in the final version of the Checklist (Chart 3). Some suggestions and modifications were also made, namely: In the Team Leader dimension, in the item “Requests monitoring, peripheral venous accesses and oxygen therapy if necessary”, it was suggested to include a minimum range to start O2, and the item was changed to “Requests monitoring, peripheral venous accesses and oxygen therapy, in case of SatO2<94%”. It was suggested in the Team Leader dimension to include “Positions the paddles in an appropriate place”; to the item “Uses a manual defibrillator safely and clearly and identifies a rhythm of coarse VF/fine VF” and the item was changed to “Places the paddles in an appropriate place and uses a manual defibrillator safely and clearly and identifies a rhythm of coarse VF/fine VF. In the compression rescuer dimension, it was suggested to include the item “Interrupts chest compressions for a maximum of 10 seconds”. In the Medication and Time rescuer dimension, it was suggested to include “Administer medication as ordered by the team leader” in the item “Administers medication as ordered by the team leader.” and the item was changed to “Administers medication as ordered by the team leader with saline solution flush and limb elevation for 10 to 15 seconds”.
Use of the checklist in the service training video that involved CPA along the lines of ACLS Megacode.
Of the 37 evaluated questions, 30 (81.1%) had the maximum score (four) assigned by the three evaluators, six (16.2%) had at least one score three and only one (2.7%) had a score of one. The final percentage of scores four assigned by the evaluators was 91.9%, exceeding the target of 85%. In the Checklist analysis, the dimension “Team leader” obtained 96.1% of scores four, “Compression rescuers” (81.5%), “Ventilation rescuers” (91.7%), “Medication and time rescuers” (91.7%) and “Team” (100%). Evaluator 1 scored 34 four scores (91.9%), evaluator 2 scored 37 four scores (100%) and evaluator 3, 32 four scores (86.5%) (Table 1).
DISCUSSION
In situations of high stress and emotional load, cognitive information is less likely to be processed effectively14),(15. Checklists allow providing instructions and considerations that help ensure that all vital steps of the procedure are completed16. Fewer failures in the completion of vital steps were observed in simulation scenarios that used checklists17.
Studies have shown that the use of ACLS checklists allows valid judgments to be obtained to verify minimum criteria for approval in competencies8),(9),(10),(18. However, it was observed that several questions remain unanswered when checklists are used for general use in the ACLS training scenario instead of specific checklists.
This study was conducted with the objective of developing a checklist to evaluate the simulation training of a specific scenario of CPA care based on the AHA training molds in its last 2015 update. To assess content validity, the checklist items were selected, other items were selected and translated into Portuguese, and the experts verified the adequacy of the items regarding the representation of the proposed scenario and proposed adjustments. The checklist was divided by functions according to the position occupied by the team member, and the order of the items in the dimensions was also verified by the experts. To evaluate construct validity, experts used the checklist to evaluate a care training video that involved CPA along the lines of the ACLS Megacode, in order to contemplate the proposed clinical case. Inter-rater agreement was high. Only five items showed disagreement between evaluators. The item in the Rescuers dimension: “Positions rigid board before starting compressions” was considered the item with the greatest range of variation, with extreme scores.
Studies have shown that non-adherence to life-saving care processes was lower in simulations in which checklists were present17),(19. Studies have observed that training of team members in the simulation increases familiarity with the scenario, and also that an increase in the clinical use of checklists has been observed(19.
Studies have demonstrated the efficacy of using checklists to evaluate simulation scenarios, especially when used in training for CPA care14),(20)-(23.
Other studies have indicated the ability of checklists to discriminate different levels of performance, especially in situations where clearly accepted “best actions” are present18),(24)-(28. Checklists can help teams work more efficiently during critical events, and they help to recall appropriate actions in emergency situations when stress is high. Simulation-based training can also increase acceptance as an important teaching-learning strategy, skill acquisition, and compliance with new processes29.
Simulation training is used as an important teaching-learning strategy. The results of this study point to the possibility of using specific checklists as an evaluation parameter for training and evaluating students during an ACLS simulation. The results also indicate that the use of the Likert scale facilitated the evaluation criteria for each item. Moreover, the use of the checklist can allow you to verify training steps that have not been clear to the students, or that have not been trained properly, aiming to reduce doubts and difficulties in the performance of the processes.
Additional studies using the Likert scale checklist are required to confirm the ability to use the checklist for use in ACLS certifications, and for the training of other ACLS skills and training scenarios. The use of checklists with rigorous and systematic methods to establish standards for the assessment of clinical skills during training allows the assessment standardization and addresses areas in a standardized way in the debriefing, preventing the neglect of important topics.
One possibility to improve score differences between examiners would be to provide instructional material to examiners, guiding them on the standards, and to develop a set of standardized video assessments to use as a measure of variability and training tool30)-(32.
Checklists have been used for years by clinicians as personal aids, and health service administrators have encouraged the adoption of checklists with the aim of minimizing risk, increasing patient safety, and decreasing the cost of litigation33. However, the benefits of checklists are only achieved when the clinical team is involved and are used to change the dynamics of a team’s culture34. Checklists should therefore not be considered “magic bulletins”. However, they can help to minimize variation and standardize care, as well as to maintain consistency and ensure quality of care, resulting in reduced complication rates and lower mortality35),(36. Many health professionals, however, fear that checklists may limit their clinical judgment, autonomy and hinder professional relationships37. These concerns will require significant changes in organizational culture and will take time to be appreciated and overcome38)-(40. Furthermore, investments in training will be necessary to incorporate checklists into clinical operations19).
The main study limitation is related to the fact that a single observer recorded the video of care training involving CPA, and the reliability of this recording process is not yet fully known. Another limitation refers to the evaluation of the scenario by experts in a single recording and the fact that a larger number of experts could have been used. Moreover, in the absence of a clear guideline for preparing the checklist. Previous studies suggest several measures to ensure the effective implementation of the checklists, starting with the evaluation of other services or studies that have successfully implemented emergency checklists or manuals, in addition to also conducting a literature review of checklists and simulation-based training. Finally, our checklists were not able to identify all the possible steps or details in a simulation training in the face of the several variables involved.
CONCLUSION
The use of the checklist in the evaluation of clinical ability, in the video of care training that involved CPA along the lines of the ACLS Megacode, showed high agreement between the evaluators. The use of rigorous and systematic methods to establish standards for the evaluation of the clinical skills required in ACLS allows a more detailed description of the team’s failures and successes and allows the individual and collective evaluation of care, so that the evaluator and instructor can direct the training and correct failures in the debriefing of each case, aiming to improve the teaching method and optimize the participants’ learning. This evaluation model allows one to decentralize the obligation of only one participant, the leader, and evaluating the team globally and individually.
References
-
1 Kramer HS, Drews FA. Checking the lists: a systematic review of electronic checklist use in health care. Biomed Inform. 2017;71S:S6-S12. doi: http://dx.doi.org/10.1016/j.jbi.2016.09.006.
» https://doi.org/http://dx.doi.org/10.1016/j.jbi.2016.09.006 -
2 Reason, J. (1990). Human Error. Cambridge: Cambridge University Press. doi:10.1017/CBO9781139062367
» https://doi.org/10.1017/CBO9781139062367 -
3 McFetrich J, Price C. Simulators and scenarios: training nurses in emergency care. Med Educ. 2006;40:1139. doi: http://dx.doi.org/10.1111/j.1365-2929.2006.02591.x.
» https://doi.org/http://dx.doi.org/10.1111/j.1365-2929.2006.02591.x -
4 Cannon-Diehl MR. Simulation in healthcare and nursing: state of the science. Critical Care Nurs Q. 2009;32:128-36. doi: http://dx.doi.org/10.1097/cnq.0b013e3181a27e0f.
» https://doi.org/http://dx.doi.org/10.1097/cnq.0b013e3181a27e0f -
5 McEvoy MD, Smalley JC, Nietert PJ, Field LC, Furse CM, Blenko JW, et. al. 3rd. Validation of a detailed scoring checklist for use during advanced cardiac life support certification. Simul Healthc. 2012 Aug;7(4):222-35. doi: 10.1097/SIH.0b013e3182590b07. PMID: 22863996; PMCID: PMC3467004.
» https://doi.org/10.1097/SIH.0b013e3182590b07 -
6 Rodgers DL, Securro Jr S, Pauley RD. The effect of high-fidelity simulation on educational outcomes in an advanced cardiovascular life support course. Simul Healthc . 2009;4:200-6. doi: http://dx.doi.org/10.1097/SIH.0b013e3181b1b877.
» https://doi.org/http://dx.doi.org/10.1097/SIH.0b013e3181b1b877 - 7 Collicott PE, Hughes I. Training in advanced trauma life support. JAMA. 1980;243:1156-9.
-
8 Wayne DB, Siddall VJ, Butter J, Fudala MJ, Wade LD, Feinglass J,et al. A longitudinal study of internal medicine residents’ retention of advanced cardiac life support skills. Acad Med. 2006;81:S9-S12. doi: http://dx.doi.org/10.1097/00001888-200610001-00004.
» https://doi.org/http://dx.doi.org/10.1097/00001888-200610001-00004 -
9 Wayne DB, Butter J, Siddall VJ, Fudala MJ, Wade LD, Feinglass J, et al. Graduating internal medicine residents’ self-assessment and performance of advanced cardiac life support skills. Med Teach. 2006;28:365-9. doi: http://dx.doi.org/10.1080/01421590600627821.
» https://doi.org/http://dx.doi.org/10.1080/01421590600627821 -
10 Wayne DB, Butter J, Siddall VJ, Fudala MJ, Wade LD, Feinglass J,et al. Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice. J Gen Intern Med. 2006;21:251-6. doi: http://dx.doi.org/10.1111/j.1525-1497.2006.00341.x.
» https://doi.org/http://dx.doi.org/10.1111/j.1525-1497.2006.00341.x -
11 Mukaino M, Prodinger B, Okouchi Y, Mizutani K, Senju Y, Suzuki M, et al. Development and assessment of a home environment checklist to evaluate mismatch between patients’ ability and home environment. Ann Phys Rehabil Med. 2020;63:288-95. doi: http://dx.doi.org/10.1016/j.rehab.2019.09.004.
» https://doi.org/http://dx.doi.org/10.1016/j.rehab.2019.09.004 -
12 Stufflebeam DL. The methodology of metaevaluation as reflected in metaevaluations by the Western Michigan University Evaluation Center. Journal of Personnel Evaluation in Education. 2000;14:95-125. doi: http://dx.doi.org/10.1023/a:1008198315521.
» https://doi.org/http://dx.doi.org/10.1023/a:1008198315521 -
13 ECC Committee, Subcommittees and Task Forces of the American Heart Association. American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation. 2006;112:. doi: http://dx.doi.org/10.1161/CIRCULATIONAHA.105.166550.
» https://doi.org/http://dx.doi.org/10.1161/CIRCULATIONAHA.105.166550 -
14 Arriaga AF, Bader AM, Wong JM, Lipsitz SR, Berry WR, Ziewacz JE, et al. Simulation-based trial of surgical-crisis checklists. New Engl J Med. 2013;368:246-53. doi: http://dx.doi.org/10.1056/NEJMsa1204720.
» https://doi.org/http://dx.doi.org/10.1056/NEJMsa1204720 -
15 Kindermann NK, Werner NS. The impact of cardiac perception on emotion experience and cognitive performance under mental stress. J Behav Med. 2014;37:1145-54. doi: http://dx.doi.org/10.1007/s10865-014-9564-7.
» https://doi.org/http://dx.doi.org/10.1007/s10865-014-9564-7 -
16 Cole E, Crichton N. The culture of a trauma team in relation to human factors. J Clin Nurs. 2006;15:1257-66. doi: http://dx.doi.org/10.1111/j.1365-2702.2006.01566.x.
» https://doi.org/http://dx.doi.org/10.1111/j.1365-2702.2006.01566.x -
17 Harvey A, Nathens AB, Bandiera G, Leblanc VR. Threat and challenge: cognitive appraisal and stress responses in simulated trauma resuscitations. Med Educ . 2010;44:587-94. doi: http://dx.doi.org/ 10.1111/j.1365-2923.2010.03634.x.
» https://doi.org/http://dx.doi.org/ 10.1111/j.1365-2923.2010.03634.x -
18 Wayne DB, Fudala MJ, Butter J, Siddall VJ, Feinglass J, Wade LD, et al. Comparison of two standard-setting methods for advanced cardiac life support training. Acad Med . 2005;80:S63-S66. doi: http://dx.doi.org/10.1097/00001888-200510001-00018.
» https://doi.org/http://dx.doi.org/10.1097/00001888-200510001-00018 -
19 Goldhaber-Fiebert SN, Lei V, Nandagopal K, Bereknyei S. . emergency manual implementation: can brief simulation-based OR staff trainings increase familiarity and planned clinical use? Jt Comm J Qual Patient Saf. 2015;41:212-20. doi: http://dx.doi.org/ 10.1016/s1553-7250(15)41028-1.
» https://doi.org/http://dx.doi.org/ 10.1016/s1553-7250(15)41028-1 -
20 Byrne AJ, Greaves JD. Assessment instruments used during anaesthetic simulation: review of published studies. Br J Anaesth. 2001;86:445-50. doi: http://dx.doi.org/10.1093/bja/86.3.445.
» https://doi.org/http://dx.doi.org/10.1093/bja/86.3.445 -
21 Devitt JH, Kurrek MM, Cohen MM, Fish K, Fish P, Murphy PM, , et al. Testing the raters: inter-rater reliability of standardized anaesthesia simulator performance. Can J Anaesth. 1997;44:924-8. doi: http://dx.doi.org/10.1007/BF03011962.
» https://doi.org/http://dx.doi.org/10.1007/BF03011962 -
22 Devitt JH, Kurrek MM, Cohen MM, Fish K, Fish P, Noel AG, et al. Testing internal consistency and construct validity during evaluation of performance in a patient simulator. Anesth Analg. 1998;86:1160-4. doi: http://dx.doi.org/10.1097/00000539-199806000-00004.
» https://doi.org/http://dx.doi.org/10.1097/00000539-199806000-00004 -
23 Field LC , McEvoy MD , Smalley JC , Clark CA, McEvoy MB, Rieke H, et al. Use of an electronic decision support tool improves management of simulated in-hospital cardiac arrest. Resuscitation. 2014;85:13842. doi: http://dx.doi.org/10.1016/j.resuscitation.2013.09.013.
» https://doi.org/http://dx.doi.org/10.1016/j.resuscitation.2013.09.013 -
24 Kurrek MM, Devitt JH, Cohen M. Cardiac arrest in the OR: how are our ACLS skills? Can J Anaesth . 1998;45:130-2. doi: http://dx.doi.org/10.1007/BF03013250.
» https://doi.org/http://dx.doi.org/10.1007/BF03013250 -
25 Morgan PJ, Cleave-Hogg D. Evaluation of medical students’ performances using the anesthesia simulator. Acad Med . 1999;74:202. doi: http://dx.doi.org/10.1097/00001888-199902000-00022.
» https://doi.org/http://dx.doi.org/10.1097/00001888-199902000-00022 -
26 Morgan PJ, Cleave-Hogg D, DeSousa S, Tarshis J. . High-fidelity patient simulation: validation of performance checklists. Br J Anaesth . 2004;92:388-92. doi: http://dx.doi.org/10.1093/bja/aeh081.
» https://doi.org/http://dx.doi.org/10.1093/bja/aeh081 - 27 Adler M, Ziglio E. Gazing into the oracle: The Delphi Method and its application to social policy and public health. Jessica Kingsley Publishers; Londres,1996.
-
28 Downing SM. Validity: on the meaningful interpretation of assessment data. Med Educ . 2003; 37:830-7. doi: http://dx.doi.org/10.1046/j.1365-2923.2003.01594.x.
» https://doi.org/http://dx.doi.org/10.1046/j.1365-2923.2003.01594.x -
29 Dagey D. Using simulation to implement an OR cardiac arrest crisis checklist. AORN J. 2017; 105:67-72. doi: http://dx.doi.org/10.1016/j.aorn.2016.11.002.
» https://doi.org/http://dx.doi.org/10.1016/j.aorn.2016.11.002 -
30 McManus IC, Thompson M, Mollon J. Assessment of examiner leniency and stringency (“hawk-dove effect”) in the MRCP(UK) clinical examination (PACES) using multi-facet Rasch modelling. BMCMed Educ . 2006;18:6-42. doi: http://dx.doi.org/10.1186/1472-6920-6-42.
» https://doi.org/http://dx.doi.org/10.1186/1472-6920-6-42 -
31 Weller JM, Robinson BJ, Jolly B, Watterson LM, Joseph M, Bajenov S, et al. Psychometric characteristics of simulation-based assessment in anaesthesia and accuracy of self-assessed scores. Anaesthesia. 2005;60:245-50. doi: http://dx.doi.org/ 10.1111/j.1365-2044.2004.04073.x.
» https://doi.org/http://dx.doi.org/ 10.1111/j.1365-2044.2004.04073.x -
32 Gray JD. Global rating scales in residency education. Acad Med . 1996;71:S55-63. doi: http://dx.doi.org/10.1097/00001888-199601000-00043.
» https://doi.org/http://dx.doi.org/10.1097/00001888-199601000-00043 -
33 Morris JA, Carrillo Y, Jenkins JM, Smith PW, Bledsoe S, Pichert J, et al. Surgical adverse events, risk management, and malpractice outcome: morbidity and mortality review is not enough. Ann Surg. 2003;237:844-52. doi: http://dx.doi.org/10.1097/01.SLA.0000072267.19263.26.
» https://doi.org/http://dx.doi.org/10.1097/01.SLA.0000072267.19263.26 -
34 McConnell DJ, Fargen KM, Mocco J. Surgical checklists: a detailed review of their emergence, development, and relevance to neurosurgical practice. Surg Neurol Int. 2012;3:2. doi: 10.4103/2152-7806.92163.
» https://doi.org/10.4103/2152-7806.92163 -
35 Berenholtz SM, Pronovost PJ, Lipsett PA, Hobson D, Earsing K, Farley JE, et al. Eliminating catheter-related bloodstream infections in the intensive care unit. Crit Care Med. 2004;32:2014-20. doi: http://dx.doi.org/ 10.1097/01.ccm.0000142399.70913.2f.
» https://doi.org/http://dx.doi.org/ 10.1097/01.ccm.0000142399.70913.2f -
36 Resar R, Pronovost P, Haraden C, Simmonds T, Rainey T, Nolan T. . Using a bundle approach to improve ventilator care processes and reduce ventilator-associated pneumonia. Jt Comm J Qual and Patient Saf. 2005;31: 243-8. doi: http://dx.doi.org/10.1016/s1553-7250(05)31031-2.
» https://doi.org/http://dx.doi.org/10.1016/s1553-7250(05)31031-2 -
37 Catchpole K, Russ S. The problem with checklists. BMJ Qual Saf. 2015;24:545-9. doi: http://dx.doi.org/ 10.1136/bmjqs-2015-004431.
» https://doi.org/http://dx.doi.org/ 10.1136/bmjqs-2015-004431 -
38 Borchard A, Schwappach DLB, Barbir A, Bezzola P . A systematic review of the effectiveness, compliance, and critical factors for implementation of safety checklists in surgery. Ann Surg . 2012; 256:925-33. doi: http://dx.doi.org/ 10.1097/SLA.0b013e3182682f27.
» https://doi.org/http://dx.doi.org/ 10.1097/SLA.0b013e3182682f27 -
39 Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, et al. Changes in safety attitude and relationship to decreased postoperative morbidity and mortality following implementation of a checklist-based surgical safety intervention. BMJ Qual Saf . 2011;20:102-7. doi: http://dx.doi.org/ 10.1136/bmjqs.2009.040022.
» https://doi.org/http://dx.doi.org/ 10.1136/bmjqs.2009.040022 -
40 Parand A, Burnett S, Benn J, Iskander S, Pinto A, Vincent C. Medical engagement in organisation-wide safety and quality-improvement programmes: experience in the UK Safer Patients Initiative. BMJ Qual Saf . 2010;19:e44. doi: http://dx.doi.org/10.1136/qshc.2009.036368.
» https://doi.org/http://dx.doi.org/10.1136/qshc.2009.036368
Research data is available in the body of the document.
