Psychometric Analysis of the Scale for the Predisposition to the Occurrence of Adverse Events in Nursing Care Provided in ICUs

Objective: to present the result of the validity and reliability studies concerning the Scale for the Predisposition to the Occurrence of Adverse Events (EPEA). Method: construct validity was based on Principal Components Analysis. Results: reliability verified through Cronbach’s alpha indicated good reliability (structure α=0.80; process α=0.92). Conclusion: based on its psychometric indicators, the EPEA can be considered a valid measure to assess the attitudes of nurses in relation to factors that potentially lead to the occurrence of adverse events in ICUs.


Introduction
International concern with patient safety became more evident after the publication of the report "To err is human: building a safer health system" (1) .The report was based on data from two studies verifying the occurrence of adverse events in hospitalizations in Colorado and Utah (2.9%) and also in hospitalizations that took place in New York hospitals (3.7%).When these figures are extrapolated to the 33,600,000 hospitalizations in the USA reported in 1997, it is estimated that at least 98,000 patients died in New York and 44,000 died in Colorado and Utah due to errors committed in health care delivered.
"Errors can be defined as the non-intentional use of an incorrect plan to achieve an objective, or not properly performing a planned action." (2)International data indicate that errors in the health field affect one in every ten patients around the world and "the most important knowledge in patient safety is how to avoid harm during treatment and care delivery." (3)verse events are defined as "unfavorable clinical occurrences that result in death, risk of death, hospitalization or prolongation of an existing hospitalization, or significant, persistent or permanent impairment." (4)The occurrence of these adverse events during nursing care provided in ICUs is associated with significant prolongation of hospital stays and increased medical costs (5) .
In general, quality of care has been assessed in terms of results indicators (6) , e.g.assessing the success of nursing care based on the occurrence of an adverse event.However, the quality of nursing care provided in ICUs should be analyzed as one good among ideal working conditions (structure and process), which themselves derive from national and international recommendations concerning healthcare quality and patient safety in ICUs, and then the risks and benefits that arise from human fallibility while providing care.
Such fallibility can derive from different perceptions that nurses hold concerning their work environment and care protocols and that can influence their attitudes concerning conditions that potentially lead to the occurrence of adverse events.Social psychologists' discussion of attitudes is based on three important questions (7)  In the work context, attitude consists of a disposition to either perform an action or to omit it, which directly influences the quality of care delivered by nurses in ICUs.
Hence, "attitudes are a predisposition to negatively or positively react to certain objects, institutions, concepts or other people" (8) and are similar both to interests and opinions.One of the most important attributes of attitudes is their subjectivity, because it reflects the way people see an object and not necessarily how this object really is (9) .Because attitudes are part of the individual's subjectivity, they are associated with thoughts, feelings, and actions that guide the way people live.
Due to a lack of instruments in the literature addressing the attitudes of nurses concerning aspects of the structure and process, which can compromise the quality of nursing care provided in ICUs and have adverse events as a results indicator, and also because attitude is a psychological construct not always amenable to direct observation, the Scale for the Predisposition to the Occurrence of Adverse Events (EPEA) was developed.
Psychometric scales "aim to establish a relationship function between (physical, social) environmental stimuli and the individual's behavior" (10) , to the extent they assess how much a given stimulus influences an individual's behavior, enabling, as in the case of EPEA, the measurement of nurses' attitudes concerning factors that may lead to the occurrence of adverse events.
The development of EPEA was based on three major points: a) theoretical, which refers to the theory concerning the construct and which should support the instrument's development; b) empirical, which is the stage when the procedures concerning the instrument's application and data collection are defined; and c) analytical, which establishes the procedures for statistical analysis to be employed in a psychometric instrument to test its construct validity.
The validity of a psychometric test (11) seeks to verify whether the test measures what it is supposed to measure and the extent to which such measurement is achieved.The entire validation process seeks to ensure isomorphism, i.e. equivalence among the properties of the psychological attribute and the representation of this object in the form of a measure.It can be studied from three different perspectives: content validity, criterion validity, and construct validity (12) .
Because the EPEA, as a psychological measure based on a theory addressing a construct, represents a latent trait (attitude), its content validity was ensured through techniques that include expert and semantic analyses (13) .
approach that refers to the quality of nursing care provided in ICUs.After this stage, the instrument's final version came to be composed of 64 items grouped into two dimensions: structure (18 items) and process (46 items).
Criterion validity is defined (11) as the quality of a scale or test to function as a present or future predictor of a variable, operationally independent, called a criterion.This criterion can be a behavior displayed in an experimental or non-experimental situation, a future performance within a given field or profession, or another already validated instrument addressing the same construct.Since no instruments similar to EPEA were identified, as well as there being no directly observable behaviors that could serve as a proxy of these attitudes, we did not test criterion validity.
Construct validity takes into account the degree to which a given psychological instrument assesses the construct it was theoretically designed to assess.This procedure seeks to confer validity onto the instrument based on the analysis of its items, enabling an assessment of its psychometric quality.
Therefore, this study's objective was to present the results of the validity and reliability studies concerning the Scale for the Predisposition to the Occurrence of Adverse Events (EPEA).

Method
Methodological design focused on the development, reliability and validity tests of measurement instruments.

Study setting and participants
The study setting included six highly complex ICUs from general and university hospitals: three public and three philanthropic hospitals located in Salvador, BA, Brazil.These ICUs had 14.54 (SD=6.08)beds and 4.03 (SD=1.58)nurses per shift, on average.Our initial proposal was to conduct a populationbased study, however, given the limited number of public and philanthropic ICUs in Salvador that consented to participate in the study, and the fact that the nurses working in these units had more than one job and could not participate in the study more than once, we opted to use a convenience sample.The sample of the study implementing the validation process of the EPEA was composed of 128 nurses: 49.2% worked in public ICUs and 50.8% worked in philanthropic units.

Instruments
The EPEA was designed to assess the level of importance nurses attribute to aspects concerning the structure and process (ideal level), as well as their perception concerning the existence of these aspects in their work environment (actual level), which can influence the quality of nursing care provided in ICUs, which can be an adverse event (AE) as a results indicator.
The participants were asked to take a position in relation to these aspects in order to evaluate their attitudes and establish a relationship between stimuli and attitudes.Hence, we sought to connect attitudes and behaviors that function as predictors of AE, for each item of the instrument (Figure 1).A Likert scale containing five response intervals, ranging from "totally disagree" to "totally agree," was used.In addition to the EPEA items, we used a social-demographic questionnaire addressing information concerning the participants' professional and personal profiles, such as gender, age, number of jobs, and professional background, among others.Because the EPEA has two parallel (ideal and actual) polytomous scales (graded) for each item in this analysis, the first step was to order the items according to two different scales, so that each item could later be correlated with the total score and determine the degree in which it measures the same attitude that other items are supposed to measure.After reordering, the items that had an inverted answer were recoded so that an answer checked as "totally disagree" (score 1) was transformed into "totally agree" (score 5); "somewhat disagree" (score 2) into "somewhat agree" (score 4); "do not agree nor disagree," which indicates a neutral position, (score 3) was not modified; "somewhat agree" (score 4) into "somewhat disagree" (score 2); and the answers checked with "totally agree" (score 5) were recoded as "totally disagree" (score 1).
The next step was to build the variables dif_est_"x" (x= 1 to 18) and dif_proc_"x" (x= 1 to 46), for the items of the scales "structure" and "process," respectively, which represent the difference of results between the "ideal" and "actual" for each of the dimensions' items.
Therefore, the total score of each individual could be obtained by totaling the scores of each item.
A PCA was performed for the 64 items, with varimax orthogonal rotation for each of two approaches (structure and process), seeking to maximize the variance of loads within the factors and find independent factors, confirming the theoretical assumption that these factors are not correlated, and identify the factor structure that accounts for the best variance explained by the construct.This approach is corroborated by the Classical Test Theory (CTT), which seeks to define a test's psychometric quality as behavioral stimuli, in terms of criteria such as present or future behaviors (variables) (10) .Because they are behavior-oriented (t = tau), CTT studies observable physical realities as a result of tests (criterion measure).
To measure the level of internal consistency, which is based on the correlation between the different items of the same test, we used Cronbach's alpha.This coefficient assesses whether the items designed to measure the same construct produce similar results.Cronbach's alphas above 0.70 were considered satisfactory (14) .The measure of internal consistency (item-total correlation) permits verifying the instrument's reliability, as it determines the measure's level of precision.

Ethical aspects
The study's Project was submitted to and approved by the Institutional Review Board at the Federal University of Bahia (Process No. 14/2011 -FR 412506).

Results
The results from the PCA for the dimension "structure" presented a determinant of the correlation matrix ≈ 0.003, enabling its inversion and use of the matrix of correlation for the analysis.The use of PCA was corroborated by the KMO test, which was 0.726, showing that the studied sample had a good fit.
Bartlett's sphericity test presented a significant value with c 2 (153)=614.477(p<0.001),indicating that the correlations among the items are sufficient for the analysis.The explained variance for one dimension was 22.85%.
PCA was the extraction method and considered the 18 items initially proposed to evaluate the dimension "structure"; 12 items were kept because they presented factor loads above 0.30 (Table 1).Lobão WM, Menezes IG.
Once construct validity was ensured using the CTT, we investigated the instrument's reliability using Cronbach's alpha, which assesses the correlation among the scores obtained in each of the test's items (internal consistency).The EPEA's measure of internal consistency obtained the following coefficients: 0.80, which indicates good reliability for the 12 items of the scale "structure" and 0.92, which indicates optimum reliability for the 34 items in the scale "process".These results show the instrument has a good level of internal consistency, indicating there is coherence among the results of similar items.
After PCA, 15 items were removed from the 64 items initially proposed for the instrument because they presented factor loads below 0.30 (Figure 2): six items of the dimension "structure" and nine items from the dimension "process".

Discussion
In regard to the instrument's construct validity, even though the items that were excluded from the "structure" and "process" dimensions after PCA, were theoretically adjusted for the studied construct and dimension, they presented factor loads below 0.3, thus, low saturation in relation to the studied dimensions.
The fact that there was a political issue of great importance for nursing professionals among these items-the 30-hour workweek-drew our attention.
However, as this issue is still being discussed in the National Congress, the establishment of a 30hour workweek is not a reality experienced by the professionals who participated in the study, thus, hindering the saturation of that factor as a general factor of quality of care delivery.
The low factor load of the items dif_est_15 and dif_est_16 may be related to the fact that they refer to a working condition already common among the participants, evidenced by the average of 3.6 (SD=1.03)beds per nurse.Hence, the participants did not deem it a factor that potentially leads to the occurrence of AEs.This working condition, when not met, is related to burnout and dissatisfaction at work, resulting in a high turnover rate, increased mortality due to complications (failure to rescue), greater incidence of infections, and adverse events (5) .
The validation items (dif_proc20, dif_proc31, dif_proc34), were intended to assess the consistency of the responses provided by the participants and that were removed due to low saturation, possibly indicating that the nurses answered the scale in a predictable way.
That is, they rejected statements that were developed to : a) to what extent are internal mental attitudes related to actual behavior?; b) to what extent are individuals' internally organized views of world based on a systematic selection of points of view?; and c) why, at some point in life, do different people share the same opinions about specific issues?

Table 2 -
Saturation and Communality of items of the dimension "process" after orthogonal rotation