Neonatal patient safety assessment: construction and validation of a protocol and a checklist

Abstract

Objective

To build and validate the content and appearance of a graph protocol and a checklist for patient safety assessment in the Neonatal Intensive Care Unit.

Methods

This is a methodological research, developed from March to September 2018, under construction of a protocol and a checklist and content and appearance validation. The Delphi technique was used to assess the instruments, and agreement among judges was measured by the Content Validity Coefficient. The item with more than 80% agreement was considered valid.

Results

The instruments presented a content validity coefficient of 0.97 in the second Delphi round, for content validity. The general estimate of the appearance validation instruments was 0.99 in Delphi II. After inclusion of suggested changes, all judges recommended the use of the protocol and the checklist.

Conclusion

The protocol and the checklist were considered valid and its use constitutes an important means to verify the conditions that compromise a safe newborn care.

Patient safety; Intensive care units, neonatal; Protocols; Checklist

Resumo

Objetivo

Construir e validar conteúdo e aparência de um protocolo gráfico e checklist para a avaliação da segurança do paciente em unidade de terapia intensiva neonatal.

Métodos

Pesquisa metodológica, desenvolvida no período de março a setembro de 2018, em duas etapas: construção do protocolo e checklist , e validação de conteúdo e aparência. Utilizou-se a técnica Delphi para avaliação das ferramentas, e o consenso entre os juízes foi mensurado pelo Coeficiente de Validade de Conteúdo. Considerou-se válido o item com mais de 80% de concordância.

Resultados

Os instrumentos apresentaram Coeficiente de validade de conteúdo de 0,97 na segunda rodada Delphi, para validade de conteúdo. A estimativa geral dos instrumentos para validação de aparência foi de 0,99 na Delphi II. Após inclusão de alterações sugeridas 100% dos juízes recomendaram o uso do protocolo e do checklist .

Conclusão

O protocolo e o checklist foram considerados válidos e sua utilização constitui importante meio para verificar as condições que comprometem o cuidado seguro ao neonato.

Segurança do paciente; Unidades de Terapia Intensiva neonatal; Protocolos; Checklist

Resumen

Objetivo

Construir y validar contenido y la apariencia de un protocolo gráfico y de una checklist para la evaluación de la seguridad del paciente en una unidad de cuidados intensivos neonatal.

Métodos

Investigación metodológica, desarrollada en el período de marzo a septiembre de 2018, en dos etapas: construcción del protocolo y de una checklist , y validación de contenido y apariencia. Se utilizó la técnica Delphi para la evaluación de las herramientas y el consenso entre los jueces medido por medio del Coeficiente de Validez de Contenido. Se consideró válido el ítem con más del 80 % de consenso.

Resultados

Los instrumentos presentaron un Coeficiente de validez de contenido del 0,97 en la segunda ronda Delphi, para validez de contenido. La estimación general de los instrumentos para validación de la apariencia fue del 0,99 en Delphi II. Después de la inclusión de las alteraciones sugeridas, el 100 % de los jueces recomendaron el uso del protocolo y de la checklist .

Conclusión

El protocolo y la checklist fueron considerados válidos y su utilización constituye un medio importante para que se verifiquen las condiciones que comprometen el cuidado seguro con el neonato.

Seguridad del paciente; Unidades de cuidado intensivo neonatal; Protocolos; Checklist

Introduction

The incorporation of quality in health and patient safety (PS) to care practices in health services brings the need to develop performance monitoring strategies that help management decision-making.( ¹ )

One of the control mechanisms for compliance with these requirements is the assessment of health quality, defined as a continuous investigation to detect and correct deviations from the standards found early, allowing the improvement of assessed processes.( ² )

Among the health quality assessment approaches, the most used model in the world is the Donabedian model, which comprises a systemic structure and establishes the triad “structure-process-result” as a conceptual model that favors the extraction of data by categories. The “structure” highlights the context of infrastructure analysis and organizational characteristics, the “process” involves the delivery of care itself, and the “result” represents the effect of care on the health status of patients or populations.( ² )

This model can be applied in different health contexts to systematize the analysis of assessed requirements and generate inferences regarding quality of care.( ² )In this sense, it is possible to use specific PS requirements, as a fundamental and inseparable dimension of quality in health, due to its global priority and important impact on the reduction of failures related to care.( ³ )

One way to systematize the assessment of quality of care and PS is using technological instruments such as protocols and checklists, which increase assessment reliability and reliability, as they are based on valid scientific evidence. The development and implementation of these instruments have been recurrent in order to provide safer care and reduce harm to patients.( ¹ , ⁴ )

The use of these instruments represents a resource of hard light technology that favors the planning of interventions and provides security to professionals for decision-making. The construction of technologies for the assessment of care processes are in evidence and support the implementation of safe practices that contribute to the quality of individual and collective care.( ¹ , ⁴ , ⁵ )

Under this logic, it is plausible to use protocols and checklists that support the assessment of specific quality criteria in health and PS in critical care units, such as the Neonatal Intensive Care Unit (NICU). This environment demands specialized attention, complex technologies and an agile and assertive practice on the part of the team, due to the profile of admitted patients (mostly premature, with low birth weight, who present complications of the gestational period or childbirth).( ⁶ )

Newborns admitted to the NICU are at high risk for adverse events, with the potential for significant harm. In some cases, the damage rate can range from 10 to 15%, with a higher proportion for those with low birth weight. Among the main causes are inadequate medication management, healthcare-associated infections (HAI), invasive procedures, misidentifications, skin lesions due to skin immaturity, nasal lesions caused by non-invasive ventilation, among others.( ⁷ , ⁸ )These conditions can be reduced with the adherence of preventive measures incorporated into daily life.( ⁸ , ⁹ )

In this context, it is considered that neonatal PS should be a priority for health systems, due to the serious family, social, individual and economic implications related to preterm birth.( ⁸ , ⁹ )

In light of this perspective, it is considered that the assessment of health services, especially in the NICU, through standardized instruments, can substantially contribute to harm-free care, and thus support the implementation of interventions focused on continuous improvement and in strengthening the safety culture.( ⁴ , ¹⁰ , ¹¹ )We believe in the importance of assessing health services guided by these technologies to raise opportunities for improvement that must be worked on to achieve quality and safe care.

That said, this study was guided by the following research question: what content and appearance are essential for the construction of an instrument to assess PS in the NICU?

To answer this question, the objective was to build and validate the content and appearance of a graph protocol and a checklist for PS assessment in the NICU.

Methods

This is a methodological and quantitative research developed according to Pasquali’s psychometrics methodological framework.( ¹² )This type of study is appropriate to verify the methods of obtaining, organizing and analyzing data, with a view to developing, validating and assessing instruments and techniques for the context of the research.( ¹² )

The study took place from March to September 2018 in two stages: 1) construction of a graph protocol and checklist; and 2) content and appearance validation.

To build the instruments, the assessment model proposed by Donabedian was used as a theoretical framework, based on three basic elements: structure, process and result.( ² )These correspond to the essential premises for assessing quality and PS, as they enable the identification of strengths and weaknesses that may guide actions to improve health institutions.( ² , ¹⁰ )

In this study, care quality assessment indicators were adapted to neonatal care. In this way, the “structure” is related to the necessary means to provide assistance, such as the infrastructure conditions; the “process” consists of diversity of praxis performed by health professionals such as the execution of care practices and PS protocols; and the “result” reflects the consequences of care provided to patients.( ² )

The protocol construction was also guided by the results of a literature review entitled “Segurança do paciente em unidades de terapia intensiva neonatal: uma Scoping review ”, which aimed to identify the elements of safe care in the NICU environment.

Other sources were consulted, such as the legislation that supports the minimum functioning of a neonatal unit, the national recommendations of the Ministry of Health and the Brazilian National Regulatory Surveillance Agency (ANVISA - Agência Nacional de Vigilância Sanitária ), and the international recommendations of the World Health Organization and guidelines on the subject.

The protocol was structured on the elements of the structure-process-result triad. We opted for the graphical representation, as it allows aggregating a set of actions and decisions focused on results, in a clear and concise way.

In addition to the protocol, a structured observation script of the checklist type was developed, entitled “Patient Safety Assessment in the Neonatal Intensive Care Unit” ( Avaliação da Segurança do Paciente na Unidade de Terapia Intensiva Neonatal ), consisting of ten dimensions related to the elements of the triad (infrastructure; materials and equipment; human resources in nursing; patient identification; effective communication between professionals and family; prevention of infection related to health care; safe use of medication; prevention of falls; prevention of skin damage; patient safety indicators) with their respective items.

Moreover, a guide for implementing the graph protocol and checklist for assessing PS in the NICU was created to guide professionals in conducting these instruments.

The validation process took place electronically and was conducted using the Delphi technique, in two steps. The population consisted of 356 judges/experts on the subject. The sample was selected through searching resumes on the Plataforma Curriculum Lattes of the Brazilian National Council for Scientific and Technological Development (CNPq - Conselho Nacional de Desenvolvimento Científico e Tecnológico ) (http://lattes.cnpq.br/).

For the selection of judges, resumes were analyzed using the expert scoring system of Fehring’s content validation model,( ¹³ )adapted to guide the choice of professionals with a profile focused on PS and neonatology. A minimum score of five was considered according to the following criteria: master’s and doctoral degrees in nursing or related fields; thesis in PS or neonatal nursing (four points); research in PS or neonatal nursing (three points); article published in PS or neonatal nursing in Qualis B1 to A1 (three points) journals; experience of at least one year in the PS center or in neonatal care (three points); specialization in PS or neonatal nursing.

The initial Delphi I sample consisted of the first 30 judges who met the pre-established criteria.( ¹³ )According to the framework used, it is necessary to have six to 20 judges; however, it is necessary to consider the possible losses, as evidenced in the literature.( ¹² )Those who did not respond or did not accept to participate in the survey were automatically excluded from the study.

For the 30 selected experts, an invitation to participate in the research was sent electronically and their objectives were presented. Initially, seven judges agreed to participate in the validation and received the Informed Consent Form (ICF) in order to guarantee the ethical principles of the study. Subsequently, an email was sent with instructions for the validation process.

The data collection instrument was built using Google Forms, consisting of four parts: 1) characterization of judges, with guarantee of their anonymity; 2) relevance analysis of the protocol and checklist content, based on the following Pasquali criteria: behavior; objectivity/desirability, simplicity, clarity, relevance/relevance, precision, typicality and range.( ¹² )To assess the items, the options Inadequate, Partially adequate or Adequate were considered, with space for “comments or suggestions for inadequacies”; 3) judgment of protocol and checklist appearance, using the criteria adapted from the Suitability Assessment of Materials (SAM), to assess requirements for effective communication in the instruments developed: content; language; illustrations; layout; motivation; culture.( ¹⁴ )To assess the items, the options Inadequate, Partially adequate or Adequate with open space for “comments or suggestions for inadequacies” were also considered; 4) graph protocol and checklist assessment as a whole (it meets the objectives for which it was proposed and its use/application in the NICU is recommended), with the alternatives yes or no.

Data were analyzed using descriptive statistics (absolute and relative frequencies, minimum, maximum, mean, median and standard deviation).

For the validation of protocol and checklist content and appearance, the scores attributed to each item of judges’ assessments were verified, considering Pasquali’s criteria for content validation and SAM’s adapted criteria for appearance validation.( ¹² , ¹⁴ )

Item relevance was obtained by applying the Content Validation Coefficient (CVC), proposed by Hernandez-Nieto.( ¹⁵ )CVC values were calculated from the formula suggested by Hernandez-Nieto, in which CVC was determined for each adapted Pasquali criterion, for each adapted SAM criterion, and the protocol’s total CVC and checklist. The item that presented more than 80% of agreement among judges (assessed as adequate) and a CVC > 0.80 was considered valid.

It is noteworthy that for appearance criteria validation, CVC was also used as per the recommendation for analysis of the SAM instrument.( ¹⁴ )

Agreement among judges and CVC scores were reached using the Delphi technique in two assessment rounds.

Ethical aspects were respected and the study was approved by an Institutional Review Board, as stated in embodied Opinion 2,007,317 and CAAE ( Certificado de Apresentação para Apreciação Ética - Certificate of Presentation for Ethical Consideration) 64879717.4.0000.5537.

Results

The “Guide for implementing the graph protocol and patient safety assessment in the Neonatal Intensive Care Unit checklist”, together with the final version of the instruments “Graph protocol for Patient Safety Assessment in the Neonatal Intensive Care Unit” and “Checklist for Patient Safety Assessment in the Neonatal Intensive Care Unit”, are available respectively in appendices 1, 2 and 3.

The protocol (^{Appendix 2}) was divided into “Safe care structure”, “Safe care processes” and “Safe care results”. The graphical representation brought together possible actions, decisions and results in a way that the reader understands.

To use the protocol, the checklist (^{Appendix 3}) must be consulted in order to assist in the integral analysis of listed requirements. Thus, initially each dimension of the checklist will be assessed for the presence or absence of the requirements contained therein. The dimension is considered adequate if 100% of its items reach “yes” answers; or partially adequate if the “yes” answers correspond to the range of 50 to 99.9% of the items; or inadequate when compliance is below 50%.

Therefore, each dimension is assigned a score regarding item adequacy, which can vary from zero to two, according to the judgment of each dimension: Adequate=2; Partially Adequate=1; Inappropriate=0. The sum of the scores of all dimensions allows for a global assessment of whether the care offered in the NICU is safe (14 to 20 points), partially safe (7 to 13 points) or unsafe for patients (0 to 6 points).

After the protocol elaboration, content and appearance validation was started. Seven judges whose ages ranged between 29 and 53 years (mean=42.5; standard deviation=9.6) participated in the first Delphi round, with a predominance of females (n=6; 85.8%), doctors (n= 6; 85.8%), with experience in teaching (n=5; 71.3%) and in neonatal care (n=6; 85.8). In the second Delphi round there was only one loss, which totaled a final sample of six judges, with characteristics similar to those of the Delphi I stage (mean age 42.5 years, 83.3% female, 83.3% PhD holders, 66.7% with teaching experience and 83.3% with NICU care experience).

After the first Delphi, judges’ considerations about the protocol and the checklist were analyzed. The suggestions indicated modifications regarding the use of these instruments, which are summarized in Chart 1 . After the changes, feedback was given to judges and the graph protocol and checklist were sent with such changes for a new assessment (Delphi II).

Chart 1
Summary of judges’ suggestions included in the protocol

Elements	Aspects included from judges’ suggestions
Structure	Infrastructure: Separate the temperature and lighting elements. Clarify programming pattern of equipment alarms in the NICU. Replace “area for care and sanitation” by “procedure room”. Specify inputs and determine if there are identified storage locations. Replace the term “service room”. Include “area must be free from interruptions in medicine preparation”. Include an appropriate place to store high surveillance medicines.
	Materials and equipment: Add in materials and equipment: oxy-hood; cuffs of sizes adequate to hospitalized newborns’ profile; presence of alcohol in gel for each bed; material for peripherally inserted central venous catheter, central venous catheter and umbilical catheter.
	Human resources in nursing. Add a record of a continuing education program for all professionals working in the unit.
Process	Patient identification Include a protocol describing the patient identification routine with a check before each intervention.
	Communication between professionals and family: Include reporting of alarming results and care transition. Add “bedside shift change”. Include guidance to the family regarding the rules and routines for PS. Insert “one record per nursing technician shift”. Include electronic medical prescription.
	Healthcare-associated infection prevention: Include process surveillance measuring hand hygiene compliance by professional category. Include guidance for parents/visitors in a systematic way. Include control in antimicrobial use.
	Safety in medicine use: Insert the double check performed by different professionals.
	Fall prevention: Consider the signage at the entrance of the unit for risk of falls.
	Pressure injury prevention: Replace “pressure injury” by “skin injury”.
Result	Patient safety indicators: Include other care indicators such as accidental extubation, loss of central and peripheral catheter.

In the first stage of content validation only “Typicality” did not reach CVC above 0.80 (CVC=0.76). Nevertheless, the final agreement of the instruments in the initial stage was 92.8% ( Table 1 ).

Table 1
Agreement among judges about the content of the graph protocol and checklist dimensions and items, in Delphi I and II. Natal/RN, 2020.

Assessed Items	CVC		Agreement (%)
	Delphi 1	Delphi 2	Delphi 1	Delphi 2
Behavior	0.90	1.0	100	100
Objectivity	0.85	1.0	100	100
Simplicity	0.85	1.0	100	100
Clarity	0.80	0.94	85.7	100
Relevance	0.95	1.0	100	100
Precision	0.90	0.94	85.7	100
Typicality	0.76	0.94	85.7	100
Range	0.85	1.0	85.7	100
Mean	0.85	0.97	92.8	100

• CVC - Content Validity Coefficient

As for appearance validity, in the first round it was possible to reach an agreement level of 96.5%, although “language” did not reach a minimum CVC of 0.80 (CVC=0.76). After adjustments, the final CVC reached 0.94 for appearance validity. Thus, the instrument’s general estimate was CVC=0.90 in Delphi I and CVC=0.99 in Delphi II ( Table 2 ).

Table 2
Agreement among judges in Delphi I and II stages for the items to assess the appearance of the graph protocol and checklist according to the instrument “Suitability Assessment of Materials”, adapted

Assessed Items	CVC		Agreement (%)
	D1	D2	D1	D2
Content
Meets protocol objectives	0.85	1	100	100
Pages are split consistently	0.90	1	100	100
Meets the target audience’s needs	0.90	1	100	100
There is logic in the sequence of pages	0.95	1	100	100
It is relevant to be informed to the target audience	0.95	1	100	100
It is correct from a scientific point of view	0.90	1	100	100
Language
Writing is compatible with the target audience	0.90	1	100	100
Sentences are attractive and not tiring	0.76	0.94	85.7	100
There are clarity and objectivity in the text	0.76	0.94	100	100
Illustrations
Graphics match content	0.90	1	100	100
Graphic elements are understandable	0.90	1	100	100
The number is enough to address the content	0.80	1	85.7	100
Layout
Letter size and font favor reading	0.85	0.94	85.7	100
The colors used enable reading	0.85	1	85.7	100
Item and page layout is organized	0.90	0.94	85.7	100
Coherent number of pages and material size	0.95	1	100	100
Motivation
Readers are encouraged to continue reading	0.85	1	100	100
The protocol is enlightening	0.85	1	100	100
Culture
Meets the various profiles of professionals	0.85	1	100	100
Culturally suited to the target audience	0.95	1	100	100
Culturally appropriate images and examples	0.90	0.94	100	100

• CVC - Content Validity Coefficient; D1 - Delphi1; D2 - Delphi2

In Delphi I 57.1% of the judges considered that the protocol and the checklist met the proposed objectives and recommended its use in the NICU (28.6% recommended modifications and 14.3% did not recommend it). After adjustments, at the end of the assessments, all judges recommended the use of the instruments in NICU assessment.

Discussion

The construction and validation of a protocol and a checklist for PS assessment in the neonatal context becomes essential as it identifies the necessary requirements for safe care for this patient, based on legal regulations and scientific evidence that encourage continuous improvement. The importance of assessing indicators related to this assistance is justified due to the criticality of care provided by the health team, in addition to patients’ vulnerability with specific demands of structure and processes and high risk of adverse events.( ⁶ , ¹⁶ , ¹⁷ )

A study points out that the next challenge in PS is the development and implementation of instruments and strategies that allow organizations to measure and reduce damage inside and outside the hospital, continuously and routinely.( ¹⁸ )In this sense, the use of valid instruments to assess PS is an important means of verifying the existence of sources of problems responsible for the occurrence of failures, in order to recognize the problem and seek solutions, and has been successfully implemented in different contexts.( ¹⁸ - ²³ )

In such a way, in the international context, it is also evident that the measurement of results in quality improvement in the NICU generates a greater effect on quality of care for preterm infants than the introduction of new research approaches or new therapies.( ²⁴ )

Thus, content and appearance validation processes become essential for the reliability of these instruments, in order to make them safe for use in the services for which they are intended.( ²³ , ²⁵ )Therefore, the use of Fehring’s criteria adapted for this study favored the selection of judges with experience in teaching, research and care, which contributed decisively in the validation process, and ensured instrument reliability in such a specific area.( ¹³ )Studies highlight the role of masters and doctors in the development of research that can promote an impact on care practices and, consequently, on the advancement of science.( ²⁶ , ²⁷ )

It is noteworthy that, despite these instruments having been developed and validated by nursing, the responsibility for damage-free care belongs to the entire multidisciplinary team. Nursing is highlighted as an ally in reducing incidents, as provision and coordination of care occur continuously and the concern with safe care is inherent to their activities. This allows nurses and their team to identify early flaws in processes and behaviors, and help to minimize possible damage.( ⁹ , ²⁸ )

Regarding the content of the checklist and graph protocol items, it was found that the division according to PS goals can guide compliance with nationally required requirements, in addition to guiding the development of risk prevention protocols inherent to neonatal care.( ⁴ )

Thus, the instruments addressed the verification of the local infrastructure of care, the equipment and materials used in care, the nursing records in the medical record, the staff dimensioning and continuing education, the priority PS protocols and their indicators. It should be noted that the set of items must cover the entire magnitude of the attribute.( ¹² )Thus, the validation of these criteria demonstrates the importance of a detailed analysis of a protocol built to assess PS in the NICU.

Considering the initial version, the judges recommended changes relevant to neonatal care that resulted in the inclusion of items mentioned in Chart 1 . Thus, for “Structure”, all questions were guided by current legislation and relevant literature for the assessment of NICU’s minimum operating requirements. The standardization of alarms in the NICU stands out, as technology-related security should be an organizational priority and it is an international recommendation.( ²⁹ )

For “Process”, expansions were suggested in the verification of safety actions in the NICU, for instance, in the patient identification process. The verification of identifiers before the performance of care and the involvement of the family in these actions with explanation of the purpose of identification. These measures are essential for adherence to safe practices and involve changing habits and strengthen the culture of safety among NICU professionals. Furthermore, it is understood that the family is an essential element in the co-participation of care and needs to be involved in this process.( ³⁰ )

For “Result”, accidental extubation and loss of central and peripheral catheter were included. The inclusion is justified because they are incidents already reported in the literature resulting from important secondary adverse events, such as HAI and other complications inherent to the use of invasive devices.( ³¹ )

As for “Validation”, the level of agreement regarding content and appearance at the end of the assessments was adequate for all criteria, remaining above 80%. This data indicates a consonance of opinion about the protocol being relevant and contributing as an assessment instrument for the services. Thus, it can be inferred that there was agreement among participants in judging protocol and checklist validity, and that they effectively explore the requirements for measuring safe care.

In Delphi I, for content validation, only one criterion did not reach satisfactory agreement, “Typicity”, which indicates the formation of sentences with expressions consistent with the attribute. This implied the need for its adequacy in order to be revalidated and classify it as applicable.( ¹² )

Pertaining to appearance validity, in the first assessment there was a level of agreement below the recommended for “language”, related to “attractive and not tiring sentences” and “text clarity and objectivity”. This result may represent weaknesses in the understanding of the instrument, which directly influences the interpretation and consequent use of the protocol. In this sense, the recommendations for changes proposed by the judges were adopted to reach a valid agreement.

Finally, the percentage of judges who indicated the protocol and the checklist as useful instruments for assessing PS in the NICU was considered satisfactory, whereas in the first Delphi round, only one judge disagreed, and two suggested the recommendation after the adjustments raised. In the second round there was no objection to the use of the protocol, and 100% responded that after adjustments were made, it met the proposed objectives.

As a study limitation, the small number of participating judges is highlighted, although it is considered adequate by the methodological framework used. Furthermore, the specificity of the theme that considered only factors related to care in the NICU environment. Therefore, it is also recommended to carry out research for the construction and validation of instruments aimed at other scenarios involving newborn care, including the multidisciplinary dimension.

Conclusion

The construction of a graph protocol and a checklist for PS assessment in the NICU was based on the structure-process-result triad and on scientific evidence dealing with PS. In this context, the content and appearance validation process enrolled a group of judges who considered the instruments valid and indicated their use after adjustments made. Thus, using these instruments, specific to the neonatal context, was an important means to verify the conditions that compromise a safe newborn care, related to active and latent problems, in order to seek continuous improvement.

Appendix 1 Graph protocol implementation guide and patient safety assessment checklist in the Neonatal Intensive Care Unit

This script aims to guide the use of a graph protocol Implementation Guide and a Checklist for Patient Safety Assessment in the Neonatal Intensive Care Unit (NICU). These instruments aim to identify care risks, in a global way, to support professionals’ decision-making in the NICU, in order to corroborate the prevention of harm to neonates.

For the purpose of explaining how the protocol and the checklist will work:

The graph protocol is based on a checklist entitled “Patient Safety Assessment in the Neonatal Intensive Care Unit”. This checklist consists of ten dimensions (1. Infrastructure; 2. Materials and equipment; 3. Human resources in nursing; 4. Patient identification; 5. Effective communication between professionals and family; 6. Healthcare-associated infection prevention; 7. Safety in medicine use; 8. Fall prevention; 9. Skin injury prevention; 10. Patient safety indicators) that are inserted in “structure”, “process” and “result”.

Each checklist dimension has specific items that correspond to the requirements of obtaining compliance for each dimension and were drawn up based on national regulations on PS, on legislation that addresses the minimum operating conditions for a NICU, and on scientific evidence on the safe care of newborns.

It is suggested that at least two previously trained evaluators perform a non-participant observation in the NICU with the protocol and checklist in hand. The higher the number of observations made, the greater the possibility of verifying the requirements to be evaluated. It is important that there is no communication among them at the time of observations so as not to influence the pattern of responses.

The protocol and the checklist should be used together throughout the assessment. Each evaluator will initially fill out the checklist by punctuating the presence (Y), or absence (N) of checked items. At the end, the sum of items will be performed and a classification by dimension according to the percentage of conformities achieved.

Thus, it will be considered “Adequate” if 100% of its items present “yes” answers; or “Partially adequate” if the “yes” answers are between 50 and 99.9% of the items; or “Inappropriate” when compliance is below 50%. This classification will represent a value to be considered in the graph protocol, as shown in the table below:

Compliance percentage of checklist items	Dimension classification according to the percentage of conformities	Score assigned in the graph protocol
Below 50%	Inadequate	0
From 50 to 99.9%	Partially adequate	1
100%	Adequate	2

With this, we move on to the application of the graph protocol:

The graph protocol is subdivided into three parts, namely:

The dimensions assessed in the checklist are inserted in the graph protocol in the form of a rand, which indicates decision-making with three response possibilities (Adequate, Partially adequate and Inadequate). Next to each rand there is a dashed line that connects it to an explanatory box, which indicates which checklist items should be checked, as shown in the following figure:

Thus, each dimension is assigned a score referring to adequacy of assessed items. This score can range from zero to two, according to the judgment of each dimension: Adequate = 2 points; Partially adequate = 1 point; Inadequate = 0 point. The sum of the scores for all dimensions allows us to assess whether the care offered in the NICU is safe, partially safe or unsafe for patients.

Sum of the scoring of the graph protocol dimensions	Care Classification
0 to 6 points	Unsafe Care
7 to 13 points	Partially Safe Care
14 to 20 points	Safe Care

It is suggested that these instruments be used periodically in order to promote a continuous PS assessment in the Neonatal Unit to identify and treat opportunities for improvement in the unit and strengthen the safety culture in the organization.

Appendix 2 Graph protocol for Patient Safety Assessment in the Neonatal Intensive Care Unit

NICU - Neonatal Intensive Care Unit

T.N.: As this protocol was not officially translated into English, a free translation was carried out.

NICU - Neonatal Intensive Care Unit

NICU - Neonatal Intensive Care Unit

Appendix 3 Checklist for Patient Safety Assessment in the Neonatal Intensive Care Unit

STRUCTURE ELEMENT		Y	N
1. Infrastructure	1.1) Does it have an air-conditioned environment?
	1.2) Does the environment have natural lighting?
	1.3) Does the area per bed correspond to approximately 6m2?
	1.4) Is there a spacing of 1m between the cribs?
	1.5) Does it have oxygen, compressed air and vacuum points per bed?
	1.6) Does it have a power grid point for portable x-ray equipment for each environment?
	1.7) Does it have an emergency electricity grid?
	1.8) Does it have a hand wash sink for every 5 beds?
	1.9) Does it have an alcoholic preparation dispenser per bed?
	1.10) Is there a procedure room for care and hygiene of newborns?
	1.11) Are there areas of purge, washing and preparation of material for further sterilization?
	1.12) Does the unit have a single isolation room?
	1.13) Is there an identified location for storage of general-purpose supplies?
	1.14) Does it have an exclusive noise-free area for preparing medicines, with good lighting and no sources of distraction and interruption in medicine preparation?
	1.15) Is there a specific place for the custody of high-surveillance medicinal products?
	1.16) Does it have a human milk collection room?
	1.17) Does it have a multidisciplinary prescription room?
2. Materials and equipment	2.1) Does the unit have neonatal resuscitation equipment, 1 for every 5 beds?
	2.2) Does it have 1 bedside monitor for continuous heart rate monitoring, cardioscopy, pulse oximetry and noninvasive pressure, respiratory rate and temperature per bed?
	2.3) Does it have cuffs of sizes appropriate to the profile of hospitalized newborns to measure non-invasive blood pressure?
	2.4) Is there a standard programming of alarms and control over the noise of electromedical equipment in the NICU?
	2.5) Are teams trained to respond promptly to alarms?
	2.6) Does it have a microprocessed mechanical pulmonary ventilator: 1 for every 2 beds, with an operational reserve of 1 equipment for every 5 beds?
	2.7) Does it have a specific pulmonary ventilator for transport, with battery: 1 for every 10 beds or fraction?
	2.8) Is there noninvasive pulmonary ventilation equipment: 1 for every 5 beds, when the microprocessed pulmonary ventilator does not have the resources to perform the noninvasive ventilation modality?
	2.9) For each bed, are there facial interface materials for noninvasive pulmonary ventilation (mask or prong)? The NICU must have all sizes (00, 0, 1, 2, 3, and 4).
	2.10) Does it have an inflatable self-balloon with a reservoir and bed mask?
	2.11) Does it have tracheal aspiration materials in open and closed systems?
	2.12) Are there 3 continuous and controlled fluid infusion equipment (“infusion pump”) per bed?
	2.13) Is there an operational reserve of equipment for continuous and controlled infusion of fluids (“infusion pump”): 1 for every 3 beds?
	2.14) Does it have double-walled incubators with humidification system (1 per NICU patient)?
	2.15) Does it have heated intensive care cribs for at least 10% (ten percent) of the beds?
	2.16) Does it have incubator for complete transport, with continuous monitoring, support for controlled infusion equipment of fluids, with battery, support for oxygen cylinder, transportable oxygen cylinder and kit (“briefcase”) to monitor the transport of severe patients, containing medicines and materials for care to emergencies: 1 for every 10 beds or fraction?
	2.17) Does it have trays containing materials for deep venous access procedures, including PICC and umbilical catheter?
	2.18) Does it have trays containing materials for lumbar puncture procedures, chest drainage, bladder catheterization, dressings in general?
	2.19) For every 5 beds, is there a capillary glucose measurement equipment specific for hospital use?
	2.20) For each bed, is there individual kit with digital thermometer, measuring tape and stethoscope?
	2.21) Are personal and collective protective equipment available for use?
	2.22) Is there 1 phototherapy device for every 2 beds?
	2.23) Is there a portable electronic scale?
	2.24) Does it have a capnograph for every 10 beds?
	2.25) Is there 1 negatoscope for every 10 beds?
	2.26) Does it have 1 otoscope for every 10 beds?
	2.27) Is there 1 ophthalmoscope for every 10 beds?
	2.28) Does it have refrigerator with internal temperature of 2 to 8°C, for exclusive use for medicine keeping, with conference and temperature record at maximum intervals of 24 hours: 1 (one) per NICU?
	2.29) Is there a wall clock visible to the beds?
	2.30) Is there a removable and comfortable chair for carrying out the Kangaroo Method?
3. Human resources in nursing	3.1) Is there a nurse coordinator exclusive to the unit?
	3.2) Is there 1 exclusive care nurse in the unit for every 8 beds or fraction per work shift?
	3.3) Does the team present 1 nursing technician for every two beds or fraction per shift?
	3.4) Does it have one (1) nursing technician for care support services per shift?
	3.5) Does the technical officer implement and maintain permanent education program records for all professionals working in the unit?
PROCESS ELEMENT		Y	N
4. Patient identification	Is there a protocol describing the patient identification routine in the NICU?
	Does the service use wristbands to identify newborns with at least two identifiers (mother’s full name and date of birth)?
	Is the conference of patient identification by professionals before the performance of care been held?
	Is the companion/family/caregiver involved in the correct identification process?
	Is it explained to the companion/family/caregiver the purpose of the 2 bracelet identifiers and that the identification conference is mandatory before care?
	Are there identification plates in newborn beds with at least two identifiers (mother’s full name and date of birth)?
5. Effective communication between professionals and family	5.1) Does the shift pass occur at the patients’ bedside?
	5.2) Is there standardization of information that should be passed on among professionals during care transition?
	5.3) Is the family oriented about the unit’s rules and routines for newborn safety?
	5.4) Are parents informed that they have the right to free access and stay in the unit?
	5.5) Are information on the evolution of patients communicated to family members by the team at least once a day?
	5.6) Is there at least 1 nurse record in medical records every 24 hours?
	5.7) Is there at least 1 record per work shift of nursing technicians in patients’ medical records every 24 hours?
	5.8) Are professionals trained to communicate bad news to families?
	5.9) Do they have an electronic prescription?
	5.10) Are prescriptions for medicinal products legible without abbreviations and/or erasures?
6. Healthcare-associated infection prevention	6.1) Do they have a hand hygiene protocol?
	6.2) Are there warnings for the correct hygiene of hands near patients’ sinks or beds?
	6.3) Are hand hygiene guidelines systematically passed on to parents/visitors?
	6.4) Is there process surveillance or auditing to measure hand hygiene compliance by professional category?
	6.5) Is there an implanted protocol to prevent bloodstream infection associated with catheter use?
	6.6) Are continuous parenteral infusion equipment exchanged every 96 hours?
	6.7) Is there a protocol for exchanging coverage of central venous accesses?
	6.8) Do they have a protocol for the prevention of ventilator-related pneumonia?
	6.9) Is there a protocol to prevent urinary tract infection related to catheter use?
	6.10) Do they have a control protocol for antimicrobial use?
	6.11) Do they have a protocol for Total Parenteral Nutrition use?
7. Safety in medicine use	7.1) Are there protocols for medicine use in newborns?
	7.2) Do they use identification tags on parenteral extenders?
	7.3) Are medicines dispensed by the pharmacy according to the medical prescription?
	7.4) Is the pharmacy control system used for medicine traceability and validity?
	7.5) Are medicines dispensed ready for use, i.e. in unit doses?
	7.6) Do the medicines stored in the refrigerator identify the date of opening of the bottle and the time of stability?
	7.7) Is a list of high surveillance medicines available at the unit with guidance on preventive actions for incidents related to these medicines?
	7.8) Do they adopt alerts, such as specific labels, for high surveillance medicines used in the unit?
	7.9) Do they have allergy warning systems?
	7.10) Is standardization available regarding medicine incompatibility?
	7.11) Are already prepared medicinal products identified with printed or manual labels?
	7.12) Are parenteral solutions infused in infusion pump or syringe pump identified with labels?
	7.13) Is double checking performed by the nursing team before medicine administration?
8. Fall prevention	8.1) Is there a protocol to prevent falls in newborns?
	8.2) Are newborns assessed for risk of falls?
	8.3) In case of high risk, are there any visual signs of risk of falls in the unit or in patients’ bed?
	8.4) Are guidance on the prevention of falls provided for mothers/companions?
	8.5) Is inter- and in-hospital transport of newborns carried out in a transport incubator with a seat belt?
	8.6) Are professionals trained to maintain fall prevention measures in the NICU?
9. Skin Injury prevention	9.1) Is there a protocol to prevent skin injury?
	9.2) Is rotation performed for pulse oximeter sensor use?
	9.3) Is rotation performed for thermal sensor use?
	9.4) Does the team change decubitus every 2 hours in clinically stable newborns?
	9.5) Is there a protocol to prevent nasal septum injury?
	9.6) Is there standardization to exchange fixations without damage to newborns’ skin?
RESULT ELEMENT		Y	N
10. Patient safety indicators	10.1) Does the service monitor healthcare-associated infection indicators in the NICU on a monthly basis?
	10.2) Does the service monitor the occurrence of skin lesions in the NICU monthly?
	10.3) Are medicine errors notified and analyzed monthly?
	10.4) Are falls in the NICU notified and analyzed monthly?
	10.5) Does the unit monitor the occurrence of accidental extubation?
	10.6) Are central and peripheral catheter losses measured?
	10.7) Is a user/family satisfaction survey conducted?

NICU: Neonatal Intensive Care Unit Y: YES N: NO T.N.: As this checklist was not officially translated into English, a free translation was carried out.

Referências

• ¹
  Hallam BD, Kuza CC, Rak K, Fleck JC, Heuston MM, Saha D, et al. Perceptions of rounding checklists in the intensive care unit: a qualitative study. BMJ Qual Saf. 2018;27(10):836-43.
• ²
  Donabedian A. Evaluating the quality of medical care. 1966. Milbank Q. 2005;83(4):691-729.
• ³
  Agência Nacional de Vigilância Sanitária (Anvisa). Gerência de Vigilância e Monitoramento em Serviços de Saúde (GVIMS). Gerência Geral de Tecnologia em Serviços de Saúde (GGTES). Assistência segura: uma reflexão teórica aplicada à prática. [Série: Segurança do Paciente e Qualidade em Serviços de Saúde]. Brasília (DF): Anvisa; 2017. Cap.3. 168 p [citado 2021 Jul 31]. Disponível em: https://www.saude.go.gov.br/images/imagens_migradas/upload/arquivos/2017-09/2017-anvisa---caderno-1---assistencia-segura---uma-reflexao-teorica-aplicada-a-pratica.pdf
  » https://www.saude.go.gov.br/images/imagens_migradas/upload/arquivos/2017-09/2017-anvisa---caderno-1---assistencia-segura---uma-reflexao-teorica-aplicada-a-pratica.pdf
• ⁴
  Mourão Netto JJ, Dias MA, Goyanna NF. Uso de instrumentos enquanto tecnologia para a saúde. Saúde Redes. 2016;2(1):65-72.
• ⁵
  Krauzer IM, Dall’Agnoll CM, Gelbcke FL, Lorenzini E, Ferraz L. A construção de protocolos assistenciais no trabalho em Enfermagem. Rev Min Enferm. 2018;22:e-1087.
• ⁶
  Gaíva MA, Rondon JN, Jesus LN. Segurança do paciente em unidade de terapia intensiva neonatal: percepção da equipe de enfermagem. Rev Soc Bras Enferm Ped. 2017;17(1):14-20.
• ⁷
  Fabretti SC, Brassica SC, Cianciarullo MA, Romano-Lieber NS. Rastreadores para a busca ativa de eventos adversos a medicamentos em recém-nascidos. Cad Saude Publica. 2018;34(P):e00069817.
• ⁸
  Caeymaex L, Lebeaux C, Roze JC, Danan C, Reynaud A, Jung C, et al. Study on preventing adverse events in neonates (SEPREVEN): a stepped-wedge randomised controlled trial to reduce adverse event rates in the NICU. Medicine (Baltimore). 2020;99(31):e20912.
• ⁹
  Tomazoni A, Rocha PK, Ribeiro MB, Serapião LS, Souza S, Manzo BF. Segurança do paciente na percepção da enfermagem e medicina em unidades de terapia intensiva neonatal. Rev Gaúcha Enferm. 2017;38(1):e64996.
• ¹⁰
  Gomes AT, Alves KY, Bezerril MS, Rodrigues CC, Ferreira Júnior MA, Santos VE. Validation of graphic protocols to evaluate the safety of polytrauma patients. Acta Paul Enferm. 2018;31(5):504-17.
• ¹¹
  Costa DB, Ramos D, Gabriel CS, Bernardes A. Cultura de segurança do paciente: avaliação pelos profissionais de enfermagem. Texto Contexto Enferm. 2018;27(3):e2670016.
• ¹²
  Pasquali L. Instrumentação psicológica: fundamentos e práticas. Porto Alegre: Artmed; 2010. 560 p.
• ¹³
  Fehring RJ. The fehring model. In: Carrol-Johnson RM, Paquete M, editors. Classification of nursing diagnoses: proceeding of the tenth conference. Philadelphia: Lippincott Company; 1994. 592 p.
• ¹⁴
  Sousa CS, Turrini RN, Poveda VB. Translation and adaptation of the instrument “Suitability Assessment of Materials” (SAM) into Portuguese. Rev Enferm UFPE On line. 2015;9(5):7854-61.
• ¹⁵
  Hernández-Nieto RA. Contributions to statistical analysis. Mérida: Universidad de Los Andes; 2002. 228 p.
• ¹⁶
  Carr H, Crotto J, Demirel S, Fisher S, Logue L, Marcott M, et al. A system-wide approach to prevention of in-hospital newborn falls. MCN Am J Matern Child Nurs. 2019;44(2):100-7.
• ¹⁷
  Krzyżaniak N, Pawłowska I, Bajorek B. Review of drug utilization patterns in NICUs worldwide. J Clin Pharm Ther. 2016;41(6):612-20. Review.
• ¹⁸
  Bates DW, Singh H. Two decades since to err is human: an assessment of progress and emerging priorities in patient safety. Health Aff (Millwood). 2018;37(11):1736-43.
• ¹⁹
  Costa DB, Ramos Daniele G, Carmen S, Bernardes A. Cultura de segurança do paciente: avaliação pelos profissionais de enfermagem. Texto Contexto Enferm. 2018;27(3):e2670016.
• ²⁰
  Carvalho RE, Arruda LP, Nascimento NK, Sampaio RL, Cavalcante ML, Costa AC. Assessment of the culture of safety in public hospitals in Brazil. Rev Lat Am Enfermagem. 2017;25:e2849.
• ²¹
  De Souza TL, Mota RO, Brito EA, Farias LM, Matias EO, Lima FE. Patient safety in the administration of intramuscular medication in pediatrics: assessment of the nursing practice. Rev Gaúcha Enferm. 2018;39:e2017-0002.
• ²²
  Ronsoni RM, Pereira CC, Stein AT, Osanai MH, Machado C. Avaliação de oito Protocolos Clínicos e Diretrizes Terapêuticas (PCDT) do Ministério da Saúde por meio do instrumento AGREE II: um estudo piloto. Cad Saude Publica. 2015;31(6):1157-62.
• ²³
  Marinho PM, Campos MP, Rodrigues EO, Gois CF, Barreto ID. Construction and validation of a tool to Assess the Use of Light Technologies at Intensive Care Units. Rev Lat Am Enfermagem. 2016;24:e2816.
• ²⁴
  Spitzer AR. Has quality improvement really improved outcomes for babies in the neonatal intensive care unit? Clin Perinatol. 2017;44(3):469-83.
• ²⁵
  Leite SS, Áfio AC, Carvalho LV, Silva JM, Almeida PC, Pagliuca LM. Construction and validation of an educational content validation instrument in health. Rev Bras Enferm. 2018;71(4):1635-41.
• ²⁶
  Gomes DC, Prado ML, Canever BP, Jesus BH, Sebold LF, Backes VM. Doctor of nursing: capacity for building a professional and scientific career project. Texto Contexto Enferm. 2016;25(3):e1260015.
• ²⁷
  Zamprogna KM, Backes VM, Menegaz JC, Francisco BS. Characterization of didactic and pedagogical training in Brazilian stricto sensu Postgraduate Programs in Nursing. Rev Esc Enferm USP. 2019;53:e03430.
• ²⁸
  Reis CE. Protocolo de segurança do paciente na Unidade de Terapia Intensiva: a importância da equipe de enfermagem. Rev Cient Multid Núcleo Conhecimento. 2019;9(3):104-13.
• ²⁹
  Oliveira AC, Pinto SA. Patient participation in hand hygiene among health professionals. Rev Bras Enferm. 2018;71(2):259-64.
• ³⁰
  Gomes AP, Querido DL, Silva GR, Almeida LF, Rocha RG. The importance of newborn identification to the delivery of safe patient care. Cogitare Enferm. 2017;(22)3:e49501.
• ³¹
  Lanzillotti LS, Andrade CL, Mendes W, Seta MH. Eventos adversos e incidentes sem dano em recém-nascidos notificados no Brasil, nos anos 2007 a 2013. Cad Saude Publica. 2016;32(9):1-13.

Edited by

Associate Editor (Peer review process): Ariane Ferreira Machado Avelar (https://orcid.org/0000-0001-7479-8121) Escola Paulista de Enfermagem, Universidade Federal de São Paulo, SP, Brazil

Publication Dates

• Publication in this collection
  06 June 2022
• Date of issue
  2022

History

• Received
  16 Apr 2020
• Accepted
  25 Aug 2021