Validation of the process criteria for assessment of a hospital nursing service 1

1 Paper extracted from doctoral dissertation “Validação dos critérios de avaliação do serviço de enfermagem no programa de acreditação”, presented to Escola Paulista de Enfermagem, Universidade Federal de São Paulo, SP, Brazil. 2 Doctoral student, Escola Paulista de Enfermagem, Universidade Federal de São Paulo, SP, Brazil. Scholarship holder from Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES). 3 PhD, Associate Professor, Escola Paulista de Enfermagem, Universidade Federal de São Paulo, SP, Brazil. Objective: to validate an instrument containing process criteria for assessment of a hospital nursing service based on the National Accreditation Organization program. Method: a descriptive, quantitative methodological study performed in stages. An instrument constructed with 69 process criteria was assessed by 49 nurses from accredited hospitals in 2009, according to a Likert scale, and validated by 16 judges through Delphi rounds in 2010. Result: the original instrument assessed by nurses with 69 process criteria was judged by the degree of importance, and changed to 39 criteria. In the first Delphi round, the 39 criteria reached consensus among the 19 judges, with a medium reliability by Cronbach’s alpha. In the second round, 40 converging criteria were validated by 16 judges, with high reliability. The criteria addressed management, costs, teaching, education, indicators, protocols, human resources, communication, among others. Conclusion: the 40 process criteria formed a validated instrument to assess the hospital nursing service which, when measured, can better direct interventions by nurses in reaching and strengthening outcomes.


Introduction
Assessment is a management function which aims to assist the administrative process of decision-making, in order to make it as rational and effective as possible (1) .
It has been a constant activity in professional practice, especially in the health area by nurses.
While measurement is basically a descriptive process, as it consists in quantitatively describing a phenomenon, assessment is an interpretative process, as it consists in a judgment based on standards, criteria, instruments, purposes and others (2) .
Therefore, what makes an assessment scientific is the effort to verify observations and validate their unique or diverse meaning (3) , revealing the causal and compatibility relationship among the service actions, specificity and results.Therefore, measuring quality and quantity of programs, services and health systems is vital for planning, organizing and controlling activities, the target of the assessment being the structure, process and outcomes, in addition to the influences and consequences of the environment (4)(5) .
The model based on the analysis of structure, process and outcome has been widespread, although some authors criticize the limitation of this triad.They claim that, when it comes to analyzing health policies with specific features and configurations, e.g.aspects of direct care service or clinical treatment, those might not be sustained by using a single rationality.To do so, they propose the composition of more than one method of assessment (1,6) .In that case, a theoretical model for assessment of clinical services or user satisfaction would be used along with the quality assessment of the health service suggested by Donabedian (7) .
In that sense, assessment by accreditation stands out because it is associated with the possibility and need for interventions capable of modifying certain sanitary situations such as verifying difficulties and facilities, identifying vulnerabilities, seeking better solutions, changing care and political processes to meet the health/population needs.It refers to the discussion about the characteristics of assessment and its effects (8) , in order to establish higher standards of quality and safety.Thus, assessment by accreditation, both in the United States since the mid-1950s, and in Brazil in the 1990s, among other countries, has become a universal phenomenon (5) , whose essence is to ensure the survival of healthcare organizations, considering the financial burden resulting from inadequate management, professional errors, the differences among services provided, as well as advances in informatics, production, dissemination of technical and scientific knowledge and the search for innovative strategies in order to improve quality, satisfaction and patient safety at higher levels (9)(10) .Therefore, assessment for accreditation of quality in Brazil has grown, and thus, consolidated safer and more effective practices in patient care.In the universe of around 7,500 hospitals, 304 (4.05%) had achieved accreditation by 2012 according to the model proposed by the National Accreditation Organization-ONA (8) .
It is a desire to contribute so that all aspects, simple and complex, are included in an instrument for assessment of the nursing service, such as the issue of security, competence, risk management, academia and practice, costs, among others; that which is essential, since only some aspects have been mentioned in the textbooks in use, such as whether the information service has a responsible technician, whether there are records in the chart, whether there are updated routines, whether it has a care model, and evidence of improvement cycles.
The aim was to validate an instrument containing process criteria for assessment of the hospital nursing service (NS), based on the ONA Accreditation program.

Method
This was a descriptive, quantitative, methodological approach, developed in the following stages: 1.

Construction of the instrument based on Avedis
Donabedian (11) , 2. Assessment of the instrument by nurses from accredited hospitals in Brazil, and 3.

Validation of the instrument with judges.
The construction of the instrument for assessment was based on the guidelines of the Professional Organization Section and, specifically, on the Nursing subsection of the ONA Accreditation program (12) , version 2006, and on the attribute process of Avedis Donabedian (7) .The criteria were added and amplified by the current literature and experience of the researchers.
In the second stage, the population consisted of 113 nursing managers from all of the hospitals accredited by March of 2009, in Brazil.Telephone contacts were made to explain the research, the instrument for assessment was sent with an explanatory script and an Informed Consent Form (ICF).A five-point Likert scale was used for judgment of the importance of each criterion (13) .
Forty-nine instruments assessed by nurse managers, containing 69 process criteria, were returned.
These managers were recruited as practical judges, since they were the ones using the criteria applied in the hospitals to achieve accreditation.A 75% cutoff for the assessment of a criteria as "important" and "very important" by nurses was established.
In the third stage, 27 nurse judges were consulted as experts in order to validate the criteria according to the Delphi Technique (14) .The panel of judges was composed considering the theoretical and practical domains, or as indicated by their peers.The ICF, explanatory script and instrument were sent via email, mail or personally delivered.Out of these, 19 (70.3%) judges comprised the first Delphi round.
In the second Delphi round, 16 instruments were returned by January of 2010, when the process was completed due to achievement of consensus.
Data were tabulated in Excel ® spreadsheets, analyzed and interpreted (15) .Cronbach's alpha was used for reliability analysis with a significance level of 5% (15) . The

Results
The instruments and responses were coded in order of receipt.
The suggestions of the nurses managers regarding the modification of criteria proposed, such as phrase changes, inclusion, exclusion or modification of the criteria were incorporated.Table 1 shows the process criteria and the importance attributed by nurses.In the process attribute, out of the 69 criteria, 55 were judged as being of "relative importance" to "very important", with criteria 3, 11, 22, 25, 36 and 54 standing out with a percentage higher than 85%.
Criterion 57 obtained a higher percent for "important" (48.9%).Only criteria 59 and 69 were judged within across all five alternatives.
There were divergent responses from some nurses when they did not consider criteria 57, 59 and 69 "very important".In addition, they proposed criteria changes, suggesting to move them to the structure or outcome scale.It was also suggested to group the risk management criteria and to exclude some.Even so, the value of Cronbach's alpha was high (0.971) indicating a quite consistent instrument with medium variability.
After this phase, the instrument for assessment of the nursing service assessed by the nurses decreased from 69 to 39 process criteria.
In the first Delphi round, 19 judges assessed the instrument composed by 39 criteria.
Figure 1 shows the process criteria as judged in the first and second Delphi rounds.

Judges Process Criteria for Assessment of the Nursing Service
The nurse manager is autonomous to make decisions about the work processes 2 X Nursing administration and management is visible in the institution 3 X There is active participation (involvement and commitment) of the nurse manager and nurse leaders in nursing administration (e.g.shared leadership) 4 Nursing administration or management is available.Available "to what is accessible, easy to reach, of reasonable value, sociable, communicative".

X
There is a program/policy to contemplate proactive actions of the workers in quality and safety processes with awards/ incentives/other benefits.

6
The institution has (multidisciplinary/interdisciplinary) clinical protocols used by nursing based on evidence and on the nosocomial profile.

X
There is at least one nursing work group to improve processes and institutional interaction.

X
Nurses participate in interdisciplinary commissions/working groups/ committees in the institution.The care processes that involve nursing are often analyzed and improved, and/or when there is procedure change.
(The Figure 1 continue in the next page...) Feldman LB, Cunha ICKO, D'Innocenzo M. 12 There is a Nursing Manual (printed or electronic with the formal description of the procedures) available and easily accessible for consultation by the team.

X
The Nursing Process phases of assessment, diagnosis, care planning and evaluation are applied and/or a model, checklist, standardized protocol is adopted.
14 X X There is evidence of the utilization of information from the Nursing Process/protocols/checklist in the prescription of other professionals.

X
The nursing team is aware of who its customers and suppliers are.The indicators of quality of nursing care are used to outline action plans and improvement of processes.

X
The critical analysis and improvement of processes (e.g.noncompliance, notification of adverse events and failure prevention) identified by nursing are treated in formal meetings with the multiprofessional group, with a deadline.

X
Nurses contribute to, participate in and/or act in control of psychotropic drugs in the unit, with the pharmacist.

X
Nurses contribute to and participate in the multiprofessional hospital commission/group/committee for risk management/safety by acting in decision-making, control, assessment and risk monitoring, adverse, contingency and sentinel events.

X
Nursing contributes to, participates in and/or acts in environmental risk and waste management in a multidisciplinary way (e.g., building reform, oxygen leak, accident with a glass ampoule) 22 X X Nurses contribute to, participate in and/or act in civil responsibility for risk management, in a multidisciplinary way with the legal area.

X X
There is a contingency plan for unforeseen or unexpected events (e.g., a patient is shot (gunfire) in the bed by a visitor in the Emergency Room).

X X
Newborns, children and/or adults are divided/separated/grouped according to the severity/specificity of the case, and/or need of the patient, verified by nursing during the care process.

X X
There is validation and control by nursing related to sterilization processes in Materials and Sterilization.

X
Nursing comprehensively identifies the body parts (organs, biopsies, pathologic anatomy, amputated limbs), controls and delivers the material, registers output, and knows the flow for referral with other areas involved in the process.

X
Nursing knows, participates in and/or controls the flow and destination of the body parts into the environment (landfill, incineration and morgue), with other areas involved in this process.

X
There is a protocol/procedure to comprehensively identify what is stored in refrigerators with multidisciplinary control of this process.29 X X There is systematic and continuous control, registering and monitoring of refrigerator cleaning/hygiene and temperature, and its functionality is verified by areas sharing this process.

X
There is articulation between the Hospital and the Nursing School or College evidenced in care by students.

X
The participation of the nursing team is encouraged to be proactive and to assess their results.32 X X There is a systematic and periodic program for update and improvement for the nursing team.

X
The nurse manager has the autonomy to hire and/or fire the nursing worker.

34
There is a formal program to integrate the new worker (nursing professional) into the institutional service.

X
Nurses contribute to, participate in and/or act in the information/communication system on inpatient data (through institutional procedure/ protocol) with other professionals.

X
There is evidence that written and verbal communication is effective in nursing.

37
Nursing receives information regarding institutional conduct and/or orientations from other sectors, such as Commission for Infection Control, supplies, nutrition and other applicable areas.

X X
The assessments of clients/users/patients (sent by the Consumer Support Service/internal affairs/other) follow the institutional standards and are used by management to improve nursing care.

X
The client receives feedback on his assessment and/or manifestations.

X
The cost of nursing care (or care cost indicator) is measured and has institutional impacts.
enabled nurses to measure and advance the criteria of excellent quality, and maximize safety practices when delivering care.
The instrument should be applied in practice for adequacy of the criteria to the institutional needs in the light of safety, with the reliable measurement of the nursing service (5,18,23) highlighted, as education is also a bridge for the quality gap (24) .

Conclusion
This research allowed the presentation of 40 process criteria for assessment of the hospital nursing service, based on the accreditation program in Brazil.
To do so, 49 nurse managers from accredited hospitals judged 69 criteria and suggested modifications.Then

Figure 1 -
Figure 1 -Process Criteria assessed by judges in the first and second Delphi rounds for assessment of the nursing service.São Paulo, SP, Brazil, 2010

16 X
Nurses use indicators to measure the quality of care (e.g. the monthly index of hospital infection is utilized by nursing as an indicator of quality) 17 16 judges validated the final instrument, composed of 40 criteria, through the Delphi Technique and reliability obtained by the Cronbach's Alpha test.The incorporation of assessment as a systematic practice in health, and the use of an instrument composed of process criteria, can provide effective information to the nurse in the definition of intervention strategies and mold the management outcomes of the nursing care.

Table 1 -
study respected the guidelines of Resolution 196/96 by the National Health Council.The project Distribution of the importance of process criteria for assessment of the nursing service judged by nurses.