Validation of the process criteria for assessment of a hospital nursing service ¹

Abstracts

OBJECTIVE:

to validate an instrument containing process criteria for assessment of a hospital nursing service based on the National Accreditation Organization program.

METHOD:

a descriptive, quantitative methodological study performed in stages. An instrument constructed with 69 process criteria was assessed by 49 nurses from accredited hospitals in 2009, according to a Likert scale, and validated by 16 judges through Delphi rounds in 2010.

RESULT:

the original instrument assessed by nurses with 69 process criteria was judged by the degree of importance, and changed to 39 criteria. In the first Delphi round, the 39 criteria reached consensus among the 19 judges, with a medium reliability by Cronbach's alpha. In the second round, 40 converging criteria were validated by 16 judges, with high reliability. The criteria addressed management, costs, teaching, education, indicators, protocols, human resources, communication, among others.

CONCLUSION:

the 40 process criteria formed a validated instrument to assess the hospital nursing service which, when measured, can better direct interventions by nurses in reaching and strengthening outcomes.

Nursing Evaluation Research; Quality Assurance, Health Care; Outcome and Process Assessment (Health Care); Evaluation Studies; Accreditation; Safety Management

OBJETIVO:

validar um instrumento com critérios de processo para avaliação do serviço de enfermagem, com base na Organização Nacional de Acreditação.

MÉTODO:

estudo descritivo, quantitativo, de abordagem metodológica, realizado em etapas, onde, após elaboração do instrumento com 69 critérios de processo, esse foi avaliado, conforme escala tipo Likert, por 49 enfermeiros de hospitais acreditados em 2009, e validado por 16 juízes, por meio do ciclo de Delphi, em 2010.

RESULTADO:

o instrumento inicial, analisado pelos enfermeiros com 69 critérios de processo, foi julgado pelo grau de importância, e passou para 39 critérios. No 1º ciclo Delphi, os 39 critérios atingiram o consenso entre os 19 juízes, com média de confiabilidade pelo alfa Cronbach. No 2º ciclo, foram validados 40 critérios convergentes, por 16 juízes, com alta confiabilidade. Os critérios abordam a gestão, custo, ensino, educação, indicadores, protocolos, recursos humanos, comunicação, entre outros.

CONCLUSÃO:

os 40 critérios de processo validados formam um instrumento para avaliação do serviço de enfermagem hospitalar que, ao ser mensurado, pode melhor direcionar as intervenções pelo enfermeiro para o alcance e fortalecimento dos resultados.

Pesquisa em Avaliação de Enfermagem; Garantia da Qualidade dos Cuidados de Saúde; Avaliação de Processos e Resultados (Cuidados de Saúde); Estudos de Avaliação; Acreditação; Gerenciamento de Segurança

OBJETIVO:

validar un instrumento que contiene los criterios del proceso de evaluación de los servicios de enfermería de hospitales basado en el programa de la Organización Nacional de Acreditación.

MÉTODO:

estudio metodológico descriptivo, cuantitativo realizado en etapas. Un instrumento construido con 69 criterios del proceso fue evaluado por 49 enfermeras de los hospitales acreditados en 2009, utilizando una escala Likert, y validado por 16 jueces a través de rondas Delphi en 2010.

RESULTADO:

el instrumento original evaluado por enfermeras tenía 69 criterios del proceso que fueron juzgados por el grado de importancia, y se modificaron 39 criterios. En la primera ronda Delphi, los 19 jueces llegaron a un consenso acerca de los 39 criterios, con una confiabilidad media obtenida a través de alfa de Cronbach. En la segunda ronda, 40 criterios convergentes fueron validados por 16 jueces, con una alta confiabilidad. Los criterios abordados fueron gestión, costos, enseñanza, educación, indicadores, protocolos, recursos humanos, comunicación, entre otros.

CONCLUSIÓN:

los 40 criterios del proceso formaron un instrumento validado para evaluar el servicio de enfermería hospitalario, que, cuando se mide, puede dirigir mejor las intervenciones de enfermería para alcanzar y fortalecer los resultados.

Investigación en Evaluación de Enfermería; Garantía de la Calidad de Atención de Salud; Evaluación de Proceso y Resultados (Atención de Salud); Estudios de Evaluación; Acreditación; Administración de la Seguridad

Introduction

Assessment is a management function which aims to assist the administrative process of decision-making, in order to make it as rational and effective as possible^{( 1 )}. It has been a constant activity in professional practice, especially in the health area by nurses.

While measurement is basically a descriptive process, as it consists in quantitatively describing a phenomenon, assessment is an interpretative process, as it consists in a judgment based on standards, criteria, instruments, purposes and others^{( 2 )}.

Therefore, what makes an assessment scientific is the effort to verify observations and validate their unique or diverse meaning^{( 3 )}, revealing the causal and compatibility relationship among the service actions, specificity and results. Therefore, measuring quality and quantity of programs, services and health systems is vital for planning, organizing and controlling activities, the target of the assessment being the structure, process and outcomes, in addition to the influences and consequences of the environment^{( 4 - 5 )}.

The model based on the analysis of structure, process and outcome has been widespread, although some authors criticize the limitation of this triad. They claim that, when it comes to analyzing health policies with specific features and configurations, e.g. aspects of direct care service or clinical treatment, those might not be sustained by using a single rationality. To do so, they propose the composition of more than one method of assessment^{( 1 , 6 )}. In that case, a theoretical model for assessment of clinical services or user satisfaction would be used along with the quality assessment of the health service suggested by Donabedian^{( 7 )}.

In that sense, assessment by accreditation stands out because it is associated with the possibility and need for interventions capable of modifying certain sanitary situations such as verifying difficulties and facilities, identifying vulnerabilities, seeking better solutions, changing care and political processes to meet the health/population needs. It refers to the discussion about the characteristics of assessment and its effects^{( 8 )}, in order to establish higher standards of quality and safety.

Thus, assessment by accreditation, both in the United States since the mid-1950s, and in Brazil in the 1990s, among other countries, has become a universal phenomenon^{( 5 )}, whose essence is to ensure the survival of healthcare organizations, considering the financial burden resulting from inadequate management, professional errors, the differences among services provided, as well as advances in informatics, production, dissemination of technical and scientific knowledge and the search for innovative strategies in order to improve quality, satisfaction and patient safety at higher levels^{( 9 - 10 )}.

Therefore, assessment for accreditation of quality in Brazil has grown, and thus, consolidated safer and more effective practices in patient care. In the universe of around 7,500 hospitals, 304 (4.05%) had achieved accreditation by 2012 according to the model proposed by the National Accreditation Organization-ONA^{( 8 )}.

It is a desire to contribute so that all aspects, simple and complex, are included in an instrument for assessment of the nursing service, such as the issue of security, competence, risk management, academia and practice, costs, among others; that which is essential, since only some aspects have been mentioned in the textbooks in use, such as whether the information service has a responsible technician, whether there are records in the chart, whether there are updated routines, whether it has a care model, and evidence of improvement cycles.

The aim was to validate an instrument containing process criteria for assessment of the hospital nursing service (NS), based on the ONA Accreditation program.

Method

This was a descriptive, quantitative, methodological approach, developed in the following stages: 1. Construction of the instrument based on Avedis Donabedian^{( 11 )}, 2. Assessment of the instrument by nurses from accredited hospitals in Brazil, and 3. Validation of the instrument with judges.

The construction of the instrument for assessment was based on the guidelines of the Professional Organization Section and, specifically, on the Nursing subsection of the ONA Accreditation program^{( 12 )}, version 2006, and on the attribute process of Avedis Donabedian^{( 7 )}. The criteria were added and amplified by the current literature and experience of the researchers.

In the second stage, the population consisted of 113 nursing managers from all of the hospitals accredited by March of 2009, in Brazil. Telephone contacts were made to explain the research, the instrument for assessment was sent with an explanatory script and an Informed Consent Form (ICF). A five-point Likert scale was used for judgment of the importance of each criterion^{( 13 )}.

Forty-nine instruments assessed by nurse managers, containing 69 process criteria, were returned. These managers were recruited as practical judges, since they were the ones using the criteria applied in the hospitals to achieve accreditation. A 75% cutoff for the assessment of a criteria as "important" and "very important" by nurses was established.

In the third stage, 27 nurse judges were consulted as experts in order to validate the criteria according to the Delphi Technique^{( 14 )}. The panel of judges was composed considering the theoretical and practical domains, or as indicated by their peers. The ICF, explanatory script and instrument were sent via email, mail or personally delivered. Out of these, 19 (70.3%) judges comprised the first Delphi round.

In the second Delphi round, 16 instruments were returned by January of 2010, when the process was completed due to achievement of consensus.

Data were tabulated in Excel^(r) spreadsheets, analyzed and interpreted^{( 15 )}. Cronbach's alpha was used for reliability analysis with a significance level of 5%^{( 15 )}.

The study respected the guidelines of Resolution 196/96 by the National Health Council. The project was approved by UNIFESP Research Ethics Committee #1195/06.

Results

The instruments and responses were coded in order of receipt.

Out of the 49 nurse managers of the NS, female gender prevailed (45=91.8%), aged 41-50 years (36.7%), most (18=36.7%) graduated between 1990 and 1997. The level of specialization was 91.8% (45), predominantly in the hospital administration area (30=66.6%), seven (14.2%) completed the master's degree and two (4.0%) completed the doctorate degree. Out of the 49 accredited hospitals, private ones predominated (34=69.3%), followed by public ones (9=18.3%), 18 (36.7%) of which had 201-300 beds and only seven (14.3%) of which were classified as specialized.

The suggestions of the nurses managers regarding the modification of criteria proposed, such as phrase changes, inclusion, exclusion or modification of the criteria were incorporated. Table 1 shows the process criteria and the importance attributed by nurses.

Table 1
Distribution of the importance of process criteria for assessment of the nursing service judged by nurses. São Paulo, SP, Brazil, 2010

	Not important			A little important			Relative importance			Important			Very important
	n	%		n	%		n	%		n	%		n	%
1 - The Standard Operating Procedures (SOPs) and/or Normative Instructions (NIs) are updated all over the institution										10	20.4		39	79.6
2 - Nursing SOPs and/or NIs are distributed among workers										10	20.4		39	79.6
3 - There is a formal description of SOPs, NIs and/or nursing protocols in the Nursing Manual										6	12.2		43	87.8
4 - There is a Nursing Manual available and easily accessible for consultation by the team.										8	16.3		41	83.7
5 - Nursing SOPs and/or NIs are validated by the quality area							4	8.3		15	31.3		29	60.4
6 - Nursing SOPs and/or NIs are periodically reviewed										13	26.5		36	73.5
7 - The assessment phase of the Nursing Process is applied										11	22.4		38	77.6
8 - The diagnostic phase of the Nursing Process is applied							2	4.2		17	35.4		29	60.4
9 - The care planning phase of the Nursing Process is applied							1	2.0		10	20.4		38	77.6
10 - The evaluation phase of the Nursing Process is applied										10	20.4		39	79.6
11 - The phases or model of the Nursing Process are applied in critical areas such as the ICU										7	14.3		42	85.7
12 - The phases or model of the Nursing Process are applied all over the institution							4	8.2		16	32.7		29	59.2
13 - Information from the Nursing Process is used by the multidisciplinary team							4	8.2		12	24.5		33	67.3
14 - There is evidence of the utilization of information from the Nursing Process in the prescription of other professionals							7	14.3		21	42.9		21	42.9
15 - The use of the Nursing Process is articulated as a form of continuity of care							2	4.3		10	21.3		35	74.5
16 - Nurses are responsible for their own practice and care coordination							2	4.2		10	20.8		36	75.0
17 - Nursing administration and management is visible in the institution							1	2.0		8	16.3		40	81.6
18 - The nurse specialist participates in the selection and acquisition of hospital technology (equipment) for the workplace				1	2.0		2	4.1		13	26.5		33	67.3
19 - There is evidence of assessment by the nurse manager for the technology to be acquired in the institution				2	4.1		2	4.1		16	32.7		29	59.2
20 - There is at least one nursing work group to improve processes and institutional interaction.							2	4.1		14	28.6		33	67.3
21 - Nurses participate in working groups (committees or commissions) in general in the institution							1	2.0		10	20.4		38	77.6
22 - Nursing administration or management is available							1	2.1		6	12.5		41	85.4
23 - There is active participation (involvement and commitment) of the nurse manager and nurse leaders in nursing administration							4	8.2		4	8.2		41	83.7
24 - The nurse manager is autonomous to make decisions about the work processes							3	6.1		6	12.2		40	81.6
25 - The care processes that involve nursing are often analyzed and improved							2	4.1		4	8.2		43	87.8
26 - The critical analysis and improvement of processes (e.g. non-compliance and risk) identified by nursing are treated with a deadline							3	6.1		10	20.4		36	73.5
27 - Formal periodic meetings are held for the analysis of nursing work processes							2	4.3		10	21.7		34	73.9
28 - Nurses use indicators to measure the quality of care							2	4.1		6	12.2		41	83.7
29 - The indicators of quality of nursing care are used to outline action plans and improvement of processes							2	4.2		6	12.5		40	83.3
30 - The participation of the nursing team is encouraged to establish actions and assess the outcomes.							4	8.2		12	24.5		33	67.3
31 - The nursing team is aware of who its customers and suppliers are							2	4.2		18	37.5		28	58.3
32 - There is a systematic and periodic update and/or improvement program for the nursing team							1	2.0		9	18.4		39	79.6
33 - The nurse manager has the autonomy to hire nursing workers	1	2.1					5	10.6		9	19.1		32	68.1
34 - Nurses have the autonomy to fire nursing workers	1	2.1					6	12.8		7	14.9		33	70.2
35 - There is a formal program to integrate the new employee (nursing professional) into the institutional service				1	2.0		1	2.0		12	24.5		35	71.4
36 - Patient's rights and privacy are respected in all hospital environments by nursing, according to law										7	14.3		42	85.7
37 - Nurses participate or act in the inpatient information system with other professionals										15	31.3		33	68.8
38 - The day/night cycle of inpatients is preserved by nursing							2	4.2		21	43.8		25	52.1
39 - There is evidence that communication is effective in nursing							2	4.2		15	31.3		31	64.6
40 - Nurses know the most prevalent nosocomial profile in the institution							5	10.4		21	43.8		22	45.8
41 - The institution has (multidisciplinary) clinical protocols used by nursing based on evidence and on the nosocomial profile				1	2.1		3	6.3		13	27.1		31	64.6
42 - Newborns are distributed according to severity by nursing.							2	4.3		10	21.7		34	73.9
43 - The nurse who is member of the hospital risk management committee or group participates in decision making							1	2.1		15	31.9		31	66.0
44 - There is a system implemented for the management of adverse events	1	2.1					1	2.1		6	12.8		39	83.0
45 - Nursing implements actions against adverse events							2	4.2		6	12.5		40	83.3
46 - Nursing counts on safety and/or controlled access and/or continuous surveillance in Pediatrics, Neonatal and Psychiatry areas				1	2.1		4	8.3		14	29.2		29	60.4
47 - There is a system implemented for the management of infection risks							1	2.1		8	16.7		39	81.3
48 - Nursing acts in partnership with the Commission for Infection Control in epidemiological surveillance of infections										9	18.4		40	81.6
49 - Nursing implements action against infection-related adverse events							1	2.0		11	22.4		37	75.5
50 - Nursing implements action against adverse events with blood components							2	4.1		13	26.5		34	69.4
51 - Nursing implements action against adverse events with equipment							5	10.2		15	30.6		29	59.2
52 - Nurses control psychotropic drugs in the nursing unit							3	6.3		11	22.9		34	70.8
53 - Nursing implements action against adverse events with drugs							3	6.1		8	16.3		38	77.6
54 - There is validation by nursing related to sterilization processes in the Material and Central Sterilization										5	10.2		44	89.8
55 - There is a thermometer for temperature and humidity measurement by nursing in the arsenal of the Material and Central Sterilization by nursing							2	4.1		8	16.3		39	79.6
56 - Nursing implements action in the absence of any instrument after surgery in the operating room							1	2.0		15	30.6		33	67.3
57 - There is a system implemented for the management of environmental risks in the institution				1	2.1		4	8.5		23	48.9		19	40.4
58 - The pieces in the institution are comprehensively identified and its outflow is registered and controlled by nursing							1	2.0		9	18.4		39	79.6
59 - Nursing follows the destiny of body parts to the environment, with other areas	1	2.1		2	4.2		14	29.2		19	39.6		12	25.0
60 - Nursing implements actions against adverse events with waste							3	6.1		6	12.2		40	81.6
61 - There is an institutional system implemented for the management of civil nursing responsibility risks	1	2.1					2	4.3		16	34.0		28	59.6
62 - There is a contingency plan for unforeseen or unexpected events in Nursing	1	2.1					6	12.8		18	38.3		22	46.8
63 - Nursing implements action when there are contingent events	2	4.1					1	2.0		17	34.7		29	59.2
64 - In the refrigerators of the nursing unit, there is complete identification of what is stored	1	2.1					1	2.1		9	18.8		37	77.1
65 - There is systematic control of the refrigerator temperature in the nursing units							1	2.1		10	20.8		37	77.1
66 - There is a systematic routine of cleaning and hygiene of refrigerators used by nursing							4	8.7		15	32.6		27	58.7
67 - Nursing management receives and analyzes customer assessments							1	2.0		14	28.6		34	69.4
68 - Nursing management uses customer assessments to propose improvements in processes							1	2.1		14	29.2		33	68.8
69 - There is articulation between the Hospital and the Nursing School or College evidenced in care	2	4.3		1	2.1		9	19.1		22	46.8		13	27.7

In the process attribute, out of the 69 criteria, 55 were judged as being of "relative importance" to "very important", with criteria 3, 11, 22, 25, 36 and 54 standing out with a percentage higher than 85%. Criterion 57 obtained a higher percent for "important" (48.9%). Only criteria 59 and 69 were judged within across all five alternatives.

There were divergent responses from some nurses when they did not consider criteria 57, 59 and 69 "very important". In addition, they proposed criteria changes, suggesting to move them to the structure or outcome scale. It was also suggested to group the risk management criteria and to exclude some. Even so, the value of Cronbach's alpha was high (0.971) indicating a quite consistent instrument with medium variability.

After this phase, the instrument for assessment of the nursing service assessed by the nurses decreased from 69 to 39 process criteria.

In the first Delphi round, 19 judges assessed the instrument composed by 39 criteria.

In the group of judges the female gender (94.1%) stood out, 51-60 years old (47%), with experience in service management (82.3%), experience in care services (78.6% ), work focus on nursing administration (76.5%) and specialization or master's degree (35.3%). Figure 1 shows the process criteria as judged in the first and second Delphi rounds.

Figure 1
Process Criteria assessed by judges in the first and second Delphi rounds for assessment of the nursing service. São Paulo, SP, Brazil, 2010

Out of the 39 (100%) criteria, 23 (58.97%) were judged between "not important" and "relative importance", that is, the judges' opinions on the importance of criteria 1, 2, 5, 7, 8, 14, 15, 19, 22, 23, 24, 25, 27, 28, 29, 30, 31, 32, 33, 35, 36, 38 and 39 were diverse, whereas the others reached 100% agreement in the degree of importance of "important" and "very important".

Criterion 19 reached an importance of 73.4% (n=14) between "important" and "very important" in the judges' opinion in the first round. All other criteria achieved at least 78.9%, to a maximum of 100%. Thus, the value of Cronbach's alpha was relatively high (α=0.630), with little variability.

In the second Delphi round, 18 (100%) criteria were judged: 3, 10, 11, 13, 14, 16, 18, 20, 21, 22, 23, 24, 25, 26, 29, 32, 38 and 40. Out of those, 16 (88.8%) received 100% consensus by 16 judges. Only criteria 14 (93.8%) and 23 (87.5%) achieved the lowest agreement. The inclusion of a criterion on costs was proposed: "The cost of nursing care, or indicator of care cost is measured and has institutional impact" and, when judged, had 100% convergence in the opinion "important" and "very important".

The judges' consensus of the 40 criteria in the second Delphi round for assessment of the NS reached at least 87.5%, with high Cronbach's alpha (α=0.970) and minimal variability.

Discussion

The 49 nurse managers from accredited hospitals assessed the degree of importance of 69 process criteria for assessment of the NS. Out of those, 55 (79.7%) stood out with percentage of higher than 85%, criteria 3, 11, 22, 25, 36 and 54 on the matters: description of the operational procedures, implementation of the nursing process, manager accessibility, frequent improvement of care processes, respect of the rights and privacy of the patient, and nurse-validated material sterilization; possibly because they are daily practices of nurses and staff, and thus relevant to the process of assessment of the service.

The responses of some nurses, when they did not consider criteria 33, 34, 57, 59 and 69 "very important", may be explained in some institutions because these tasks are shared in the process of multidisciplinary work, independent of the direct management of nursing. The suggestions to "make this item clearer", "unify criteria 47, 48, 49, 50, 51, 53, 60 and 62" on risk management, and also that some criteria were not relevant to process but to structure were accepted.

In the first Delphi round, while most judges (total=19) assessed the items as "important" or "very important", criterion 19 had the lowest importance, 73.4% (n=14), and dealt with the "participation, contribution and/or nurse's role in the control of psychotropic drugs in the unit, with the pharmacist", possibly because the nurse has been commonly the controller of the psychotropic drugs in service units, while considering it a shared attribution.

In the second Delphi round, 16 (84.2%) judges assessed 40 process criteria until January of 2010, when it was finished. Most reached consensus (100%) in importance, and criteria 14 (93.8%) and 23 (87.5%), despite reaching the lowest agreement among the judges, exceeded the minimum cutoff of 75% in the initially established consensus.

Criterion 40, which was added, reached unanimity, with the degree of importance being "important" and "very important" (16=100%). This result may indicate that the cost approach is relevant, which may be related to the characteristics of the competitiveness era, when the financial issue, profits and investments highlight companies, including hospitals. As experts say, despite the high prices of health, over the economic demand in general, the pressure for the use of technologies, fee-for-service, that is remuneration for service performed, is what stimulates consumption, among other factors^{( 16 )}.

The diversity of nurses and judges provided an opportunity to review the criteria under various aspects and points of view regarding the scope and complexity of the hospital environment. The long instrument, initially composed of 69 criteria, required a time expenditure, writing ability of the researcher^{( 17 )}, as well as effort, attentive and committed participation of the judges, in order to consider the criteria that impacted the demand for care. In this aspect, the complex and multifaceted work of nurses stands out, providing them with relevant skills for performance and critical analysis of professional practice^{( 18 )}.

The criteria of safety and risk management received several criticisms and proposals for unification. The problematization and contextualization of safety were not the focus of this study, but the responsibility for safe care is relevant to all of those involved in patient care and needs further research^{( 19 - 20 )}. This factor has driven new research in recent years^{( 19 - 22 )} through the use of nationally validated instruments and scales, which has enabled nurses to measure and advance the criteria of excellent quality, and maximize safety practices when delivering care.

The instrument should be applied in practice for adequacy of the criteria to the institutional needs in the light of safety, with the reliable measurement of the nursing service^{( 5 , 18 , 23 )} highlighted, as education is also a bridge for the quality gap ^{( 24 )}.

Conclusion

This research allowed the presentation of 40 process criteria for assessment of the hospital nursing service, based on the accreditation program in Brazil. To do so, 49 nurse managers from accredited hospitals judged 69 criteria and suggested modifications. Then 16 judges validated the final instrument, composed of 40 criteria, through the Delphi Technique and reliability obtained by the Cronbach's Alpha test.

The incorporation of assessment as a systematic practice in health, and the use of an instrument composed of process criteria, can provide effective information to the nurse in the definition of intervention strategies and mold the management outcomes of the nursing care.

References

• ¹
  Uchimura KY, Bosi MLM. Qualidade e subjetividade na avaliação de programas e serviços de saúde. Cad Saúde Pública. nov-dez 2002;18(6):1561-9. DOI:10.1590/S0102-311X2002000600009
  » https://doi.org/10.1590/S0102-311X2002000600009
• ²
  Bloom BS, Hastings T, Madaus G. Manual de avaliação formativa e somativa do aprendizado escolar. São Paulo: Pioneira; 1983.
• ³
  Acurcio FA, Cherchiglia ML, Santos MA. Avaliação de qualidade de serviços de saúde. Saúde Debate. 1991;33:50-3.
• ⁴
  D'Innocenzo M, Adami NP, Cunha ICKO. O movimento pela qualidade nos serviços de saúde e Enfermagem. Rev Bras Enferm. jan-fev 2006;59(1):84-8.
• ⁵
  Feldman LB, D'Innocenzo M, Cunha ICKO. Evaluating nursing practices beyond our borders. Nurs Manage. 2013;44(3):28-36.
• ⁶
  Akerman M, Nadanovsky P. Avaliação dos serviços de saúde: avaliar o quê? Cad Saúde Publ. out-dez 1992; [acesso 3 julho 2012];8(4):361-5. Disponível em: <http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0102311X1992000400002&lng=en&nrm=iso>.
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0102311X1992000400002&lng=en&nrm=iso
• ⁷
  D'Innocenzo M, Feldman LB, Fazenda NRR, Helito RAB, Ruthes RM. Indicadores, Auditorias, Certificações. Ferramentas de Qualidade para Gestão em saúde. São Paulo: Martinari; 2006.
• ⁸
  Organização Nacional de Acreditação-ONA [Internet]. Brasília (DF): ONA. [acesso 18 de abril 2012]. Disponível em: http://www.ona.org.br/.
  » Disponível em: http://www.ona.org.br/
• ⁹
  Harada MJCS, Pedreira MLG, Peterlini MAS, Pereira SR. O erro humano e a segurança do paciente. São Paulo: Editora Atheneu; 2006.
• ¹⁰
  Feldman LB, organizadora. Gestão de Risco e segurança hospitalar. Prevenção de danos ao paciente, notificação, auditoria de risco, aplicabilidade de ferramentas, monitoramento. São Paulo: Martinari; 2008.
• ¹¹
  Donabedian A. The definition of quality and approaches to its assessment. Ann Arbor (MI): Health Administration Press; 1999.
• ¹²
  Ministério da Saúde (BR). Secretaria de Políticas da Saúde. Departamento de avaliação de políticas de saúde. Manual Brasileiro de Acreditação Hospitalar. Brasília (DF): Ministério da Saúde; 2006.
• ¹³
  Likert R. A technique for the measurement of attitudes. Arch Psychol. 1932;22(140):1-55.
• ¹⁴
  Williams PL, Webb C. The Delphi technique: a methodological discussion. J Adv Nurs. 1994 Jan;19(1):180-6.
• ¹⁵
  Marconi MA, Lakatos EM. Técnicas de pesquisa: planejamento e execução de pesquisas, amostragem e técnicas de pesquisa, elaboração, análise e interpretação de dados. 7ª ed. São Paulo: Atlas; 2008.
• ¹⁶
  Vecina Neto G, Malik AM. Tendências na assistência hospitalar. Ciência & Saúde Coletiva 2007;12(4):825-39.
• ¹⁷
  Wright JTC, Giovinazzo RA. Delphi: uma ferramenta de apoio ao planejamento prospectivo. Cad Pesq Adm. 2000;1(12):54-65.
• ¹⁸
  Sanna MC. Os processos de trabalho em Enfermagem. Rev Bras Enferm. [periódico na Internet]. abr 2007 [acesso 13 dez 2012]; 60(2):221-4. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-71672007000200018& lng=en. http://dx.doi.org/10.1590/S0034-71672007000200018.
  » Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-71672007000200018& lng=en. http://dx.doi.org/10.1590/S0034-71672007000200018
• ¹⁹
  Bohomol E, Ramos LH, D'Innocenzo M. Medication errors in intensive care unit. J Adv Nurs. 2009;65(6):1259-67.
• ²⁰
  Schatkoski AM, Wegner W, Algeri S, Pedro ANR. Safety and protection for hospitalized children: literature review. Rev. Latino-Am. Enfermagem. 2009;17(3):410-6.
• ²¹
  Lobão WM, Menezes IG. Construction and content validation of the scale of predisposition to the occurrence of adverse events. Rev. Latino-Am. Enfermagem. 2012 Ago; 20(4):796-803.
• ²²
  Kuwabara CCT, Évora YDM, Oliveira MMB. Risk Management in technovigilance: construction and Validation of a Medical-Hospital Product Evaluation Instrument. Rev. Latino-Am. Enfermagem. 2010 Oct;18(5):943-51.
• ²³
  Lobiondo-Wood G, Haber J. Pesquisa em Enfermagem: métodos, avaliação crítica e utilização. 4. ed. Rio de Janeiro: Guanabara-Koogan; 2001.
• ²⁴
  Durham CF, Sherwood GD. Education to bridge the quality gap: a case study approach. Urol Nurs. 2008 Dec;28(6):431-8, 453.

• 1
  Paper extracted from doctoral dissertation “Validação dos critérios de avaliação do serviço de enfermagem no programa de acreditação”, presented to Escola Paulista de Enfermagem, Universidade Federal de São Paulo, SP, Brazil.

Publication Dates

• Publication in this collection
  Jul-Aug 2013

History

• Received
  05 Sept 2012
• Accepted
  02 May 2013