Assessment of the nursing care product (APROCENF): a reliability and construct validity study

ABSTRACT Objectives: to verify the reliability and construct validity estimates of the "Assessment of nursing care product" scale (APROCENF) and its applicability. Methods: this validation study included a sample of 40 (inter-rater reliability) and 172 (construct validity) assessments performed by nurses at the end of the work shift at nine inpatient services of a teaching hospital in the Brazilian Southeast. The data were collected between February and September/2014 with interruptions. Cronbach's alpha and Spearman's correlation coefficients were calculated, as well as the intraclass correlation and the weighted kappa index (inter-rater reliability). Exploratory factor analysis was used with principal component extraction and varimax rotation (construct validity). Results: the internal consistency revealed an alpha coefficient of 0.85, item-item correlation ranging between 0.13 and 0.61 and item-total correlation between 0.43 and 0.69. Inter-rater equivalence was obtained and all items evidenced significant factor loadings. Conclusion: this research evidenced the reliability and construct validity of the scale to assess the nursing care product. Its application in nursing practice permits identifying improvements needed in the production process, contributing to management and care decisions.


Introduction
Nursing work takes place in production subsystems, adapting and interacting constantly with other subsystems (1) .This dynamics produces emergent behaviors, besides unexpected situations related to health practice, which demand self-organization and (re) prioritization of the activities, which are characteristics of a complex adaptive system (CAS) (2) . This demands flexibility from nursing to respond to the needs required (3) .
Its organization is not linear, but influenced by factors related to individual needs, health technologies and care production (1) .The product that results -care -is intangible, and delivered and consumed at the moment the care is provided (4) .
The authors consider the construct investigated in this study, the nursing care product, to be the interaction between structural factors (human capital and support services) and work organization methods (care planning, patient/family care and multidisciplinary interaction) that intervene in the care process. The more favorably aligned these factors and methods are, the better the product will be assessed.
As a result of the productive transformation, the professionals need instruments to measure the means needed for nursing's main activity from the perspective of nursing care (2) , contributing to management and care Measuring scales support professional activities and are available to measure a range of phenomena.
Nevertheless, many of these do not properly describe their psychometric properties (5) .The reliability and validity measures determine the accuracy and quality of the instruments, helping the nurses in their evidencebased practice (6) .
The reliability is related to the precision of a scale, that is, if its measures reflect the true values of the attributes accurately (7) .One of the aspects of the reliability, the internal consistency, assesses whether the subparts of a scale measure the same characteristic or attribute. Another, the inter-rater reliability, verifies the agreement among two or more independent raters on the score obtained (8) . proposed in the theoretical perspective (8) .Thus, an imprecise, that is, an unreliable measure cannot measure the attribute in a valid manner. On the other hand, the reliability does not guarantee the validity (8) .
Is the APROCENF valid and reliable? How do the nurses perceive its applicability in the care and management context? To answer these questions, this research was developed to verify reliability estimates and the construct validity of the "Assessment of the nursing care product" scale (APROCENF) and its applicability.

Methods
The APROCENF scale rests on the concept of care production (1) , management (9) and quality in health (10) , To verify the inter-rater reliability estimate and the construct validity, 40 and 172 nursing assessments at the end of the work shift were considered, respectively.
The sample size for the validity assessment complied with the criterion estimated for the factorial analysis, i.e. 20 times the number of instrument items (12) . Two pairs of nurses from two ICUs independently assessed 20 shifts each (inter-rater agreement) (8) . To assess the construct validity, 13 nurses applied the scale at the end of their work shifts. Each assessment was registered daily on a print scale the nurses received.
To investigate the nurses' perception on the use of the scale in professional practice, a questionnaire was applied with demographic and professional data and the participants also answered nine propositions on a five-point Likert scale (from I completely disagree to I completely agree), related to the pertinence, clarity, objectivity, complexity, grade, applicability and relevance of the instrument.
Descriptive statistics were used to characterize the sample and examine the frequency distributions for each item. The scale was considered as an ordinal measuring level and significance was set at p≤0.05for all analyses made.

The assessments
The The nursing care product was classified as poor  (Table 1).  which intends to explain the inter-group differences.
Significance was found between the afternoon shift and the other, and no significance between the morning and night shifts.

Reliability Test
The instrument presented a high Cronbach's alpha coefficient, equaling 0.85. The item-item correlation ranged between0.13 and 0.61.Correlations inferior to 0.30 involved the items: resources needed for care provision and interaction and multidisciplinary activity.
The item-total correlation ranged between 0.43 (interaction and multidisciplinary activity) and 0.69 (patient and/or family care) (Table2).

Construct validity
The KMO measure of sample adequacy was equal of difference among the loadings). Therefore, only the first factor was considered for interpretation (Table 3).

Discussion
To produce good outcomes in health care, the organization and delivery of the care needs to be improved, in order to comply with the safety, equity, punctuality, efficiency and efficacy dimensions with a focus on the needs of the subjects involved (15) . The nurses are responsible for the management dimensions of care and should carefully assess this process, proposing improvements for practice (16) .
The availability of valid and reliable instruments enhances the accuracy of the information collected, as well as the efficacy of the decision process (17)(18) .
Therefore, assessing the psychometric properties, including the reliability and construct validity of the APROCENF scale, has become essential.
In the reliability analysis, the correlation coefficient determines the intensity of the relation between the instrument items (item-item) and between each item and the total score (item-total), using 0.30 as the cutoff for validation (13) .Although this criterion was not fully complied with, correlations≥0.30were found with other items, justifying its maintenance in the instrument.
The assessment of the Cronbach's alpha coefficient when deleting the item for item 2, related to the Resources needed for care provision, and for item 6 (Interaction and multidisciplinary activity), evidenced no significant interference in the internal consistency of the instrument. In addition, these are important variables for care production from the perspective of CAS (2) and the patient safety concept (19) .Thus, the instrument continued with eight items. As regards the ICC, in view of the score, equivalence was obtained. well-defined structure when>0.70 (12) . and/or unattended needs and the occurrence of adverse events (10) and the nurses working at these institutions prioritize patient care management, mitigating these risks (21) ,also evidenced in this study. In the same sense, the adequate composition of the nursing team to respond to the patients' demands has been widely discussed and studied, due to its close relation with the quality and safety of care (22)  and team integration (23) .Therefore, teaching and interdisciplinary practice have been investigated, and are currently encouraged to improve the fragmented health systems, constituting a feasible and efficient service provision model (24) .Another challenge evidenced in this study that is in accordance with other findings (25) is related to the transfer of information among clinical teams through formal registers, reducing adverse incidents and improving the continuity of health care.
It can be highlighted that the scale permits not only measuring the nursing care product at the end of each work shift, but also mapping the critical aspects that compromise the productive system. Thus, it equips nurses, health managers, educators and researchers by targeting proposals for improvement towards the weaknesses evidenced.
According to the nurses' perception on the use of the scale in professional practice, the instrument is pertinent, relevant and supports management decision taking. They agree on the clarity and applicability as well, but argue some difficulty in using it daily, in view of the activities related to the work process, supporting earlier assessments (11) .
As all production systems aim for continuous

Conclusion
In this research, the reliability and construct validity of a tool to assess the nursing care product were evidenced. Its application in clinical practice allows the nurse managers to measure the efficiency and efficacy of the production process, contributing to management and care decisions.