Acessibilidade / Reportar erro

Construction and content validation of the scale of predisposition to the occurrence of adverse events

Abstracts

This study aimed to present the results of the construction and content validation of the Scale of Predisposition for the Occurrence of Adverse Events (EPEA). A descriptive research with a qualitative approach was carried out. To study the quality of nursing care and design the scale, we performed a literature review that resulted in a preliminary instrument, composed of 90 items, divided into two dimensions to assess the quality of health care (structure and process). Expert analysis and semantic analysis were applied as techniques to study the content validity of the instrument. The findings indicate that the operational version of the EPEA was composed of 64 items, grouped into two dimensions: structure (18 items) and process (46 items). The EPEA is the first Brazilian measure constructed to assess the nurses' attitudes towards the factors that may predispose to the occurrence of adverse events in ICU.

Quality of Health Care; Iatrogenic Disease; Nursing Care; Psychometrics; Validation Studies


Neste estudo o objetivo foi apresentar os resultados da construção e validação de conteúdo da Escala de Predisposição à Ocorrência de Eventos Adversos (EPEA). Trata-se de pesquisa descritiva, de abordagem qualitativa. Para o estudo da qualidade do cuidado e elaboração dos itens do instrumento, foi realizado levantamento bibliográfico, que resultou em um instrumento preliminar, contendo 90 itens, divididos em duas dimensões para a avaliação da qualidade do cuidado em saúde (estrutura e processo). Foi realizada a validação de conteúdo do instrumento por meio das técnicas de análise de juízes e análise semântica. A partir dos resultados dessas análises, a EPEA ficou finalmente composta por 64 itens, agrupados em duas dimensões: estrutura (18 itens) e processo (46 itens). A EPEA é a primeira medida nacional construída para avaliar as atitudes dos enfermeiros frente aos fatores que podem predispor à ocorrência dos eventos adversos em UTI.

Qualidade da Assistência à Saúde; Doença Iatrogênica; Cuidados de Enfermagem; Psicometria; Estudos de Validação


Este estudio objetivó presentar los resultados de la construcción y validación de contenido de la Escala de Predisposición a Eventos Adversos (EPEA). Se trata de una investigación descriptiva, bajo enfoque cualitativo. Para estudiar la calidad del cuidado y preparación del instrumento, se realizó una revisión de la literatura, que resultó en un instrumento preliminar que contiene 90 ítems, divididos en dos dimensiones para evaluar la calidad del cuidado de salud (estructura y proceso). Se realizó la validación del contenido del instrumento por medio de las técnicas de análisis de los jueces y semántico. A partir de este análisis, la EPEA se compone de 64 ítems, agrupados en dos dimensiones: la estructura (18 ítems) y el proceso (46 ítems). La EPEA es la primera medida nacional, construida para evaluar las actitudes de los enfermeros a los factores que pueden predisponer a la aparición de efectos adversos en la UTI.

Calidad de la Atención de Salud; Enfermedad Iatrogénica; Atención de Enfermería; Psicometría; Estudios de Validación


ORIGINAL ARTICLE

Construction and content validation of the scale of predisposition to the occurrence of adverse events1 1 Paper extracted from Master's Thesis "Construção, validação e normatização da Escala de Predisposição à ocorrência de eventos adversos (EPEA)" presented to Escola de Enfermagem, Universidade Federal da Bahia, Brazil.

William Mendes LobãoI; Igor Gomes MenezesII

IMSc, Professor, Universidade do Estado da Bahia, Brazil

IIPhD, Adjunct Professor, Instituto de Psicologia, Universidade Federal da Bahia, Brazil

Corresponding Author Corresponding Author: William Mendes Lobão Universidade do Estado da Bahia Departamento de Ciências da Vida Rua Silveira Martins, 2555 Bairro: Cabula CEP: 41150-000, Salvador, BA, Brasil E-mail: willobao@gmail.com

ABSTRACT

This study aimed to present the results of the construction and content validation of the Scale of Predisposition for the Occurrence of Adverse Events (EPEA). A descriptive research with a qualitative approach was carried out. To study the quality of nursing care and design the scale, we performed a literature review that resulted in a preliminary instrument, composed of 90 items, divided into two dimensions to assess the quality of health care (structure and process). Expert analysis and semantic analysis were applied as techniques to study the content validity of the instrument. The findings indicate that the operational version of the EPEA was composed of 64 items, grouped into two dimensions: structure (18 items) and process (46 items). The EPEA is the first Brazilian measure constructed to assess the nurses' attitudes towards the factors that may predispose to the occurrence of adverse events in ICU.

Descriptors: Quality of Health Care; Iatrogenic Disease; Nursing Care; Psychometrics; Validation Studies.

Introduction

At the end of the 1990's, the publication of reports "To err is Human: building a safer health care system" alerted society to the lack of safety patients experience, estimating that, in the USA, between 44,000 and 98,000 Americans die every year due to health care errors(1). After this significant revelation, patient safety was included as one of the six dimensions of health systems quality in the report "crossing the quality chasm", published in 2011: patient safety, patient-centered objectives, effectiveness, efficiency, opportunity and equity(2).

Besides calling attention to the alarming number of deaths, the report "To err is Human" for the first time evidences the inherence of human error in an activity that used to unblemished as a result of the Hippocratic principle of non-maleficence, the "primum non nocere". At the same time, this report also defines that health care errors can be avoided by projecting systems to make their occurrence more difficult and to make it easier for professionals to make correct decisions.

Care quality is defined as "the extent to which health services increase the probability of reaching desired results in individual and population health which are consistent with current professional knowledge"(3). The main characteristics of this definition are: it identifies individuals and populations as the target public of quality promotion efforts; it is goal-oriented; it acknowledges the probability of unwanted events; it appoints the need for constant technical-scientific knowledge updates; and discusses quality measurement.

ICU patients are exposed to different risks that can be classified as: risk of tissue injury, infection caused by aspiration, hemorrhages caused by disconnection of drains and falls, evidenced by bedriddenness, accomplishment of invasive procedures, presence of drains, tubes and catheters, ineffective breathing pattern, use of mechanical ventilators, use of sedatives, psychomotor agitation and disequilibrium(4).

In Brazil, frequent studies have been done about the use of outcome indicators, mainly that of adverse event (AE), defined as "unfavorable clinical events resulting in death, risk of death, hospitalization or extension of a preexisting hospitalization, significant, persistent or permanent disability"(5) in the assessment of ICU Nursing care quality(6-7).

Based on the analysis of factors related to iatrogenic events at ICU's­(6), a study accomplished in São Paulo found that, concerning the nature of these events, out of 113 notified events in that research, catheter, endotracheal tube, probe and drain events were responsible for 40.7% (46), medication for 27.4% (31), equipment 18.6% (21), procedures 11.5% (13) and others 1.8% (2). Thus, when hospitalized at an Intensive Care Unit (ICU), the person receives care from a Nursing team and places not only confidence, but also his/her life in the hands of people (s)he does not know.

A research conducted in the United States(1) demonstrated that, when a severe adverse event or any other deviation from the expected care quality happens, besides the costs in terms of human life, it is estimated that additional costs, productivity loss and permanent disability correspond to between 17 and 29 billion dollars. In addition, these care quality deviations entail a loss of confidence in the ICU Nursing team and feelings of anxiety, impotence and blame towards the professionals involved(7-8).

Health care quality assessment, as a balance between risk and benefits, can be accomplished based on three approaches: a) structure - corresponds to the inputs, physical resources, financial resources, geographic location, equipment, accessibility and workforce qualification/specialization, permitting service delivery; b) processes (called performance today) - execution of actions based on a presupposed set of criteria, rules, standards, procedures and protocols, departing from a theoretical model that permits achieving the best care; c) outcomes - corresponds to the assessment of the successful/effective achievement of the health care goals and users or the population's satisfaction (cure, impeding the progression of the disease, restoring functional abilities, relief of pain/suffering and adverse event)(9).

In general care quality has been assessed with the help of the outcome indicator, like the assessment of nursing care success based on the occurrence of an adverse event for example. The quality of ICU Nursing care should be studied as a product between the ideal work conditions (structure and process), deriving from Brazilian and international ICU quality promotion and patient safety recommendations and the risks and benefits deriving of human beings' fallibility in care delivery. This fallibility can derive from nurses' different perceptions about their work environment and care protocols, and can interfere in their attitudes towards the conditions that can predispose to the occurrence of the adverse event. The "attitude is a mental and neural state of readiness that directly influences the individual's response to all objects and situations (s)he relates to"(10). In this study context, attitude represents a tendency towards an action or omission that directly influences the quality of the care nurses deliver at Intensive Care Units (ICU).

Based on three possible care quality assessment approaches, normative assessment 'involves judging an intervention, comparing the resources employed and their organization (structure), the services or goods produced (process) and the obtained results with criteria and standards"(11). Therefore, in this quality assessment concept, it is observed that judgment, expressed through the nurses' attitudes, plays an important role that influences the conditions that contribute to the occurrence of the AE.

In view of the lack of instruments in literature aimed at analyzing nurses' attitudes towards the structure and process aspects that can compromise ICU Nursing care quality, by predisposing to the occurrence adverse events, this study aimed to present the results of the construction and content validation of the Scale of Predisposition for the Occurrence of Adverse Events.

Method

This research used qualitative methods to presents the results of the construction and content validation of the Scale of Predisposition for the Occurrence of Adverse Events (EPEA).

Instruments

The construction of the Scale of Predisposition for the Occurrence of Adverse Events (EPEA) was based on three premises. The first refers to Donabedian's proposed theoretical approach(9), which guided the elaboration of the nursing care quality assessment indicators (structure, process and outcome indicators). For the sake of this research, the outcome descriptors were associated with the process approach, due to the behavioral aspect involved in the construction of outcome indicators. To give an example, when a nurse notifies the occurrence of a fall from the bed (adverse event), although the fall is an outcome indicator, the notification of the event represents a process. Hence, those approaches were defined as theoretical dimensions here to investigate the care quality construct.

The second premise is based on the systemic approach of error(12), which departs from the premise of the fallibility of human beings in their activities and the possibility that errors will occur in a wide range of organizations. In this model, protection barriers play a fundamental role and, when an adverse event (AE) occurs, the most important issue is not to identify who is to blame for the error, but to discover why the defense lines failed or whether a violation of conduct took place.

The final premise for the construction of the EPEA is based on the care quality promotion criteria and recommendations of Brazilian and international organizations, i.e.:

- Commitment to Hospital Quality(13): manual of Nursing indicators;

- Regional Nursing Council(14): 10 steps for patient safety;

- Brazilian Association of Intensive Medicine(15): safe ICU guide - GUTIS

- ANVISA-MS establishes minimum requisites for the functioning of intensive care units, based on the publication of RDC-7(16).

- WHO(17): nine patient safety solutions;

- American Nurses Association(18): The National Database of Nursing Quality Indicators (NDNQI);

- Joint Commission on Accreditation of Healthcare Organizations(19): National Patient Safety Goals;

To define the construct and elaborate the instrument items, a bibliographic survey was accomplished, based on the criteria and recommendations these organizations proposed, which resulted in a matrix of specifications that contained two dimensions, structure and process, and ninety descriptors related to the conditions that can compromise the quality of ICU Nursing care. For each descriptor, a corresponding item was constructed, so as to articulate attitudes and behaviors that serve as predictors of adverse events.

A five-point Likert scale was used, ranging from 1 (I totally disagree) to 5 (I totally agree), to assess the level of importance nurses attribute to the structure and process aspects (ideal level) and their perception about the existence of these aspects in their work environment (actual level). "The scale measure represents one out of different forms psychometric measurements can take"(20). It is mainly used in social psychology to study attitudes. In the theoretical approach of the Likert scale, an attitude (psychological property) represents a willingness to act. Thus, its scale serves to verify the subject's level of agreement with several assertions that express something favorable or unfavorable with regard to a psychological object. Thus, to assess the nurses' attitudes, they were asked to take a stand towards situations that can compromise ICU nursing care quality by predisposing to the occurrence of adverse events.


The preliminary version of the instrument comprised two parts, which were:

- Identification data of the intensive care unit, safeguarding the organization's name and sociodemographic identification data of the respondent, including gender, age range, time since graduation, among others, guaranteeing anonymity.

- Scale of Predisposition to the Occurrence of Adverse Events (EPEA), which originally contained 90 items, 28 for the structure dimension and 62 for the process dimension.

As a result of its uniformity, the EPEA is a measure that permits ranking/comparing answers among different respondents, facilitating their analysis. Its operating cost is lower and its administration time shorter, as it does not interfere in the service routine of the units under analysis.

EPEA Content Validation Study Procedures

Two procedures were used to analyze the quality of the developed items, based on the following theoretical premises: expert analysis and semantic analysis.

Expert Analysis

The EPEA items were submitted to expert analysis, involving the consultation of experts in the area of the construct being assessed, with a view to evaluating the pertinence of items with regard to a range of criteria that are considered important to guarantee their psychometric quality, thus validating the hypothesis that they represent the construct adequately.

Four experts with different profiles were selected, as follows: patient safety with a focus on adverse events; intensive care unit management; care and comfort in nursing; and pre-hospital handicap as a result of acute coronary syndromes.

Participants in this phase were asked to judge the instrument items, considering the pertinence of the nurse's attitude construct for ICU care quality (Question 1: Does the phrase refer to the assessment of Nursing care?) and the relation between the item and the dimension being assessed (Question 2: What dimension does the item belong to?). The experts were also asked to comment on and give suggestions to improve the items if necessary.

The subsequent phase was the descriptive analysis of the experts' opinions, in the form of a table in which all data the experts provided were consolidated, checking agreement frequencies and percentages with the construct and with the pertinence of the item being assessed. At that moment, it was defined that inter-rater agreement would correspond to items scoring above 75%. Items with agreement levels of 50% were maintained or modified and adapted if appointed by the experts.

Semantic Analysis

After modifying or excluding the items based on the expert analysis, semantic analysis was performed to analyze how the professionals in the sample understood the remaining items. For this phase, four nurses were chosen who work in emergency and intensive care at a private hospital that did not figure among the future data collection sites. The instrument was responded in the presence of the author, who intervened when requested to clarify doubts. When considered pertinent, all suggestions related to the lack of understanding of the item and suggested modifications were incorporated into the final version of the instrument.

Ethical Aspects

The research project was submitted to the Ethics Committee of Universidade Federal da Bahia, protocol No. 14/2011 - FR 412506, and approved without restrictions.

Results

The expert analysis results appointed that the following were removed from the 90 initial items: four items because they were not considered part of Nursing care assessment; four items due to semantic repetition, i.e. because they presented the same contents or idea; and 18 items due to a lack of agreement with the assessment of the construct or because the did not assess the theoretically designated dimension.

After excluding 26 items, the remaining 64 items were submitted to semantic analysis. Based on this analysis, only two items were modified, with a view to enhancing the clarity of the writing. No further items were excluded after this phase. The final instrument comprised five subdivisions, which were:

- Identification data of the intensive care unit, safeguarding the organization's name;

- Sociodemographic identification data of the respondent, guaranteeing anonymity;

- Inclusion of one item to assess the stress of ICU work;

- Orientation on the completion of the EPEA; and

- Scale of Predisposition for the Occurrence of Adverse Events (EPEA), consisting of 64 items, 18 to assess the structure dimension and 46 for the process dimension (Figure 2). Scores were inverted for five items in the process dimension (items: 38, 49, 52, 55, 56) to verify the consistency of the subjects' replies. Such items are called validation items. Hence, while the remaining scale items refer to positive behaviors or which indicate positive actions, the validation items present ideas that could be considered absurd if accepted as parts of ICU nursing practice.


Discussion

The Scale of Predisposition for the Occurrence of Adverse Events (EPEA) proposed the discussion of care quality as a balance between risks and benefits, in view of human beings' fallibility, besides proposing the use of adverse events as outcome indicators. Thus, ICU Nursing care quality becomes the product of the combination between ideal work conditions (structure and process), deriving from Brazilian and international recommendations to promote ICU patient safety and care quality, and intensive care nurses' attitudes towards the conditions that can predispose to the occurrence of the adverse event (outcome indicator).

Understanding the occurrence of these events, considering that errors are inherent in any human activity, understanding the complexity of Nursing care in a critical context like the ICU, using quality indicators and stimulating the voluntary notification of adverse events, with a focus on learning instead of reprehending, representing challenges for ICU managers. Associated with the consolidation of a safety culture, these measures can result in safer Nursing care, which users acknowledge as reliable.

In the attempt to assess nurses' attitudes towards the structure and process aspects that can compromise the quality of ICU Nursing care, using adverse events as outcome indicators, the construction of the EPEA completes a gap in the field of attitudinal measures for ICU nurses.

Although expert and semantic analyses can be characterized as part of the procedures to construct a scale, they are also considered initial phases for content validation purposes. Hence, complying with this construction phase also guarantees the scale's content validity.

Conclusions

This study aimed to present the results of the construction and content validation process of a scale to assess nurses' attitudes towards the structure and process aspects that can compromise the quality of ICU Nursing care, using the adverse event (AE) as an outcome indicator.

The content validity of the EPEA was considered adequate, in view of the item selection process and the careful assessment, including suggestions for improvement in the expert analysis as well as in the semantic analysis, thus guaranteeing that the scale items comply with the theoretical approach to ICU Nursing care quality.

As a limitation, it is appointed that, although no methodological determination on an exact number of experts was found in the consulted literature, this phase involved only four members, as one of the experts dropped out, which resulted in situations in which there was a tie in the internal assessment of the items. It would be better to use an uneven number of experts for content validation purposes in future studies.

With a view to investigating construct validity, the next step to study the psychometric qualities of the Scale of Predisposition for the Occurrence of Adverse Events is to submit it to nurses working at different intensive care units in public, non-for-profit and private hospital, so as to define sample limits for the sake of future standardization. We hope the EPEA contributes to permit the creation of more effective patient protection barriers and the investigation of causes of latent errors, as well as to investigate errors systemically and consolidate a patient safety culture instead of a Nursing error culture.

Received: Jan. 23rd 2012

Accepted: Aug. 3rd 2012

  • 1. Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer health system. [Internet]. Washington (DC): Institute of Medicine/National Academy Press; 2000. [acesso 17 fev 2011]. Disponível em: http://iom.edu/~/media/Files/Report%20Files/1999/To-Err-is-Human/To%20Err%20is%20Human%201999%20%20report%20brief.pdf
  • 2. Wachter RM. Segurança versus qualidade. In: Wachter RM. Compreendendo a segurança do paciente. Porto Alegre: Artmed; 2010. p. 45-56.
  • 3. Lohr KN, Schroeder SA, editors. A strategy for quality assurance in medicare. [Internet].vol 1. Washington(DC): National Academy Press; 1990. [acesso 02 dez 2010]. Disponível em: http://www.nap.edu/catalog/1547.html
  • 4. Carpenito LJ. Diagnósticos de enfermagem: aplicação a pratica clínica. 13ª ed. Porto Alegre: Artes Médicas; 2011. 1028 p.
  • 5
    Consulta pública n° 21, de 27 de abril de 2006 (BR). Dispõe sobre o Regulamento Técnico para Funcionamento de Serviços de Atenção ao Paciente Crítico e Potencialmente Crítico. 2006. [acesso 02 dez 2010]. Disponível em: http://www4.anvisa.gov.br/base/visadoc/CP/CP[14558-1-0].pdf
  • 6. Padilha KG. Ocorrências Iatrogênicas em Unidade de Terapia Intensiva (UTI): análise dos fatores relacionados. Rev Paul Enferm. [periódico na Internet]. 2006 [acesso 20 dez 2010]; 25(1):18-23. Disponível em: http://bases.bireme.br/cgi-bin/wxislind.exe/iah/online/?IsisScript=iah/iah.xis&src=google&base=LILACS&lang=p&nextAction=lnk&exprSearch=475688&indexSearch=ID
  • 7. Padilha KG, Kitahara PH, Gonçalves CCS, Sanches ALC. Ocorrências iatrogênicas com medicação em Unidade de Terapia Intensiva: condutas adotadas e sentimentos expressos pelos enfermeiros. Rev Esc Enferm USP. [periódico na Internet]. 2002 [acesso 11 jun 2010]; 36(1): 50-7. Disponível em: http://www.scielo.br/pdf/reeusp/v36n1/v36n1a07.pdf
  • 8. Coli RdCP, Anjos MFd, Pereira LL. Postura dos enfermeiros de uma unidade de terapia intensiva frente ao erro: uma abordagem à luz dos referenciais bioéticos. Rev. Latino-Am. Enfermagem. [periódico na Internet]. 2010 [acesso 05 jul 2012]; 18(3):27-33. Disponível em: http://www.scielo.br/pdf/rlae/v18n3/pt_05.pdf
  • 9. Donabedian A. The Definition of Quality and Approaches to Its Assessment (Explorations in Quality Assessment and Monitoring, Vol 1). Ann Arbor, MI: Health Administration Press; 1980.163 p.
  • 10. Allport GW. Attitudes. In: Murchison C, editor. Handbook of Social Psychology Worcester: Clark University Press; 1935. p. 784-98.
  • 11. Contandriopoulos AP, Champagne F, Denis JL, Pineault R. A avaliação na área de saúde. In: Hartz ZMdA, organizadora. Avaliação em Saúde: dos modelos conceituais à prática na análise da implantação de programas. Rio de Janeiro: Editora Fiocruz; 1997. p. 29-48.
  • 12. Reason J. Human error: models and management. BMJ. [periódico na Internet]. 2000 [acesso 17 jun 2011]; (30): p. 768-770. Disponível em: http://www.bmj.com/content/320/7237/768.full
  • 13
    Programa de Qualidade Hospitalar (PQH). Manual de Indicadores de Enfermagem -NAGEH. [Internet]. São Paulo(SP): APM/CREMESP; 2006. [acesso 20 dez 2010]. Disponível em: http://www.cqh.org.br/files/Manual%20de%20Indicadores%20NAGEH%20-%20V.FINAL.pdf
  • 14. Avelar AFM, Salles CLS, Bohomol E, Feldman LM, Peterlini MAS, Harada MJCS, et al. 10 Passos para a segurança do paciente. São Paulo (SP): COREN/REBRAENSP; 2010. 30 p.
  • 15. Réa-Neto Á, Castro JEC de, Knibel MF, Oliveira MC de. GUTIS: guia da UTI segura. São Paulo (SP): AMIB; 2010. 23p.
  • 16
    Resolução-RDC Nº 7, de 24 de fevereiro de 2010 (BR). Dispõe sobre os requisitos mínimos para funcionamento de Unidades de Terapia Intensiva e dá outras providência. 2006. [acesso 02 dez 2010]. Disponível em: http://www.amib.org.br/pdf/RDC-07-2010.pdf
  • 17
    World Health Organization (WHO). Nine patient safety solutions: Solutions to prevent health care-related harm. 2007. [acesso 15 mar 2011]. Disponível em: http://www.who.int/mediacentre/news/releases/2007/pr22/en/index.html
  • 18
    American Nurses Association (ANA). The National Database of Nursing Quality Indicators (NDNQI). 2001.[acesso 18 jun 2010]. Disponível em: http://www.nursingworld.org/MainMenuCategories/ThePracticeofProfessionalNursing/PatientSafetyQuality/Research-Measurement/The-National-Database/NDNQIBrochure.aspx
  • 19. Joint Commission on Accreditation of Healthcare Organizations (JCAHO). Accreditation Program: Critical Access Hospital. 2011. [acesso 25 fev 2011]. Disponível em: http://www.jointcommission.org/assets/1/6/2011_NPSGs_CAH.pdf
  • 20. Pasquali L, Capovilla AGS, Alonso AOL, Alves AR, Borba ACP, Batista CG, et al. Instrumentação Psicológica: Fundamentos e Práticas. Porto Alegre: Artmed; 2010.560 p.
  • Corresponding Author:
    William Mendes Lobão
    Universidade do Estado da Bahia
    Departamento de Ciências da Vida
    Rua Silveira Martins, 2555
    Bairro: Cabula
    CEP: 41150-000, Salvador, BA, Brasil
    E-mail:
  • 1
    Paper extracted from Master's Thesis "Construção, validação e normatização da Escala de Predisposição à ocorrência de eventos adversos (EPEA)" presented to Escola de Enfermagem, Universidade Federal da Bahia, Brazil.
  • Publication Dates

    • Publication in this collection
      31 Aug 2012
    • Date of issue
      Aug 2012

    History

    • Received
      23 Jan 2012
    • Accepted
      03 Aug 2012
    Escola de Enfermagem de Ribeirão Preto / Universidade de São Paulo Av. Bandeirantes, 3900, 14040-902 Ribeirão Preto SP Brazil, Tel.: +55 (16) 3315-3451 / 3315-4407 - Ribeirão Preto - SP - Brazil
    E-mail: rlae@eerp.usp.br