Risk Management in Technovigilance: construction and Validation of a Medical-Hospital Product Evaluation Instrument

Abstracts

Com a contínua incorporação de tecnologias na saúde, o gerenciamento de risco hospitalar deve ser implantado para sistematizar o monitoramento de eventos adversos, executando ações para o controle e eliminação de seus danos. Como parte dessas ações, a tecnovigilância atua nos procedimentos para aquisição, utilização e controle da qualidade de produtos e equipamentos na saúde. O objetivo deste estudo foi construir e validar um instrumento de avaliação de produto médico-hospitalar. Trata-se de estudo quantitativo, exploratório, longitudinal e de desenvolvimento metodológico, fundamentado no modelo de gestão de qualidade Seis Sigma, que tem como base principal as etapas componentes do Ciclo DMAIC. Para a obtenção de dados e a validação de conteúdo, utilizou-se a técnica Delphi com profissionais da Rede Brasileira de Hospitais Sentinela. Concluiu-se que o instrumento desenvolvido possibilitou avaliar o produto, diferenciando os resultados entre as marcas testadas e retratando o propósito inicial do estudo de qualificar as avaliações realizadas.

Gerenciamento de Segurança; Estudos de Avaliação; Estudos de Validação; Garantia da Qualidade dos Cuidados de Saúde

Con la continua incorporación de tecnologías en la salud, la administración de riesgo hospitalario debe ser implantada para sistematizar la monitorización de eventos adversos, ejecutando acciones para el control y eliminación de sus daños. Como parte de esas acciones, la Tecnovigilancia actúa en los procedimientos de adquisición, utilización y control de calidad de productos y equipamientos en el área de la salud. El objetivo de este estudio fue construir y validar un instrumento de evaluación de productos Médico hospitalarios. Se trata de un estudio cuantitativo, exploratorio, longitudinal y de desarrollo metodológico, fundamentado en el modelo de gestión de calidad Seis Sigma, que tiene como base principal las etapas componentes del Ciclo DMAIC. Para la obtención de datos y la validación de contenido, se utilizó la técnica Delphi con profesionales de la Red Brasileña de Hospitales Centinela. Se concluyó que el instrumento desarrollado posibilitó evaluar el producto, diferenciando los resultados entre las marcas probadas y retratando el propósito inicial del estudio de calificar las evaluaciones realizadas.

Administración de la Seguridad; Estudios de Evaluación; Estudios de Validación; Garantía de la Calidad de Atención de Salud

ORIGINAL ARTICLE

Risk Management in Technovigilance: Construction and Validation of a Medical-Hospital Product Evaluation Instrument

Cleuza Catsue Takeda Kuwabara^I; Yolanda Dora Martinez Évora^II; Márcio Mattos Borges de Oliveira^III

^IRN, Ph.D. in Nursing. E-mail: [e-mail]

^IIRN, Ph.D. in Nursing, Full Professor, Escola de Enfermagem de Ribeirão Preto, Universidade de São Paulo, WHO Collaborating Centre for Nursing Research Development, SP, Brazil. E-mail: [e-mail]

^IIIEngineer, Ph.D. in Engineering, Associate Professor, Faculdade de Economia, Administração e Contabilidade de Ribeirão Preto, Universidade de São Paulo, SP, Brazil. E-mail: [e-mail]

^{Corresponding Author}

ABSTRACT

With the continuous incorporation of health technologies, hospital risk management should be implemented to systemize the monitoring of adverse effects, performing actions to control and eliminate their damage. As part of these actions, Technovigilance is active in the procedures of acquisition, use and quality control of health products and equipment. This study aimed to construct and validate an instrument to evaluate medical-hospital products. This is a quantitative, exploratory, longitudinal and methodological development study, based on the Six Sigma quality management model, which has as its principle basis the component stages of the DMAIC Cycle. For data collection and content validation, the Delphi technique was used with professionals from the Brazilian Sentinel Hospital Network. It was concluded that the instrument developed permitted the evaluation of the product, differentiating between the results of the tested brands, in line with the initial study goal of qualifying the evaluations performed.

Descriptors: Safety Management; Evaluation Studies; Validation Studies; Quality Assurance, Health Care.

Introduction

Currently, the insertion of new technologies is one of the major concerns of hospital management. The World Health Organization estimates that around 50% of all therapeutic advances available did not exist ten years ago. These technologies are available to the health system and incorporated into clinical practice, providing benefits that were unimaginable until recently, with significant results regarding the heightened expectations of improved quality of life for the populations⁽¹⁾.

It is noted, however, that technological innovation has caused numerous problems, such as: lack of local planning for its incorporation, unsatisfactory quality, little technical and scientific training and competence of professionals, misuse, development out of pace with the evolution of local maintenance services, and high costs, which severely impact the hospital organizations⁽¹⁾.

The increasing demand for a health care professional capable of accompanying the assistance and technological development, in many cases, reflects the concern to avoid risk to the health of the patient, which may be associated, or not, with the occurrence of human errors. This calls for effective actions to integrate the management response to this situation, modeling professionals for the new technical activities that reality requires⁽²⁾, as the probable or imminent risk that the patient is submitted is not visualized correctly, or even detected in time. In this context, it refers to an event related to medical products, the utilization, the acquisition and use of medical-hospital materials with regards to their quantity and quality⁽³⁾, among others.

Hospital risk management (HRM) is presented as a new perspective on the subject, enabling the administrator of health to look at the care that the patient is actually submitted, analyzing, investigating, proposing solutions and implementing actions to try to remedy these problems or, at least pre-empt their occurrence⁽⁴⁾. The purpose of risk management is to identify the likely origin of adverse events, to evaluate the damage caused and to take appropriate decisions pertaining to these issues.

Historically, risk management in health began in the United States of America, from the mid-1920s, in the context of the "crisis of the medical error" when, processes and operating systems focusing on prevention, detection, control or elimination of risks that could cause harm to patients/clients, were generated. The risk management program developed in that country constituted one of the internal activities aimed at guaranteeing the quality of care provided. It was later implemented in other countries in Europe, Oceania and Latin America⁽⁵⁾.

The health worker is also contemplated in this scenario, where the risk conferred to the patient or the products used in their care can reflect on their own safety. Medical and nursing teams are the ones that suffer most from this reflection, considering their direct contact with these risks⁽⁶⁾. However, other actors, such as workers of the support areas of pharmacy, laboratory, laundry, nutrition and maintenance, are not excluded from this scenario.

In 2001, the concept of HRM was introduced in Brazil by the National Sanitary Surveillance Agency (ANVISA), of the Ministry of Health, with the Sentinel Hospitals project, and currently brings almost two hundred institutions together. The adoption of measures to enhance quality of health care and the rational management of supplies and hospital-medical equipment (Technovigilance), with a view to ensuring quality and safety are two of the developments of the project. Other areas, such as Pharmacovigilance (surveillance of drugs) and Haemovigilance (surveillance of blood products), are also part of the project.

With the continuing incorporation of health technology, hospital risk management should be implemented for the systemization of the monitoring of adverse events, performing actions for the control and elimination of their damage. As part of these actions, Technovigilance acts in the procedures for acquisition, utilization and quality control of products and equipment in health.

In this sense, quality is a fundamental requirement, which provides the security needed for the required results. To establish a methodology for evaluating a hospital-medical product in the pre-acquisition phase is important and becomes necessary, given that adverse events may even occur in normal situations with good quality products.

Given the wide range of products offered in the Brazilian market and the gap found in the literature regarding validated models of evaluation instruments for medical products, the following research question emerged: "Is it possible to construct and validate, together with the specialists, an evaluation instrument for quality control of medical-hospital materials aimed at preventing potential risks to patients and the health team?". The aim of this study was to construct and validate an evaluation instrument for medical-hospital products (EIMHP) to support risk management in the area of Technovigilance.

Methodology

This is a quantitative, exploratory, longitudinal and methodological development study. The investigation was structured using the quality management techniques with Six Sigma methodology^(7-8), to support risk management in the area of Technovigilance, and using the Delphi technique^(9-10), to obtain data.

Six Sigma is principally based on the component phases of the DMAIC cycle: Define; Measure; Analyze; Improve; and Control. This model is used to improve performance through the application of statistical methods and tools in order to: define problems and situations to be improved; measure to obtain information and data, analyze the information collected; incorporate and better understand the processes; and control the processes or existing products. The DMAIC cycle allows the study of variability in the quality of the processes of the institution through the application in each problem outlined^(7-8).

The Delphi technique is used to obtain data from professional experts and judges. It is used to determine the content validity and represents a useful tool for the formation of a judgment of a group, being widely used in the construction and adaptation of measuring instruments, as it is characterized by flexibility, in which the researcher responsible for the study establishes the rules regarding the number of stages. It is especially recommended when there are no quantitative data or they cannot be projected into the future with certainty, given the expectation of structural changes or determinant factors of future trends^(9-10).

The study project was submitted to the Research Ethics Committee (register CONEP 268, Protocol 129/2008) and approved by the Director Superintendent of the University Hospital of Londrina (HUL), the origin of the study. All participants were told about the research, based on the Free Prior Informed Consent Terms to meet the requirements of Resolution 196/96 of the National Health Council. The unit of research outlined for this study consisted of hospitals belonging to the Sentinel Hospital Project of the National Agency of Sanitary Surveillance/ANVISA, of the Ministry of Health Considering the three Delphi stages performed, the sample totaled 144 participants, with 139 specialists from the Delphi stage 1, 5 judges who participated in the Delphi stage 2 and 5 Judges of the Delphi stage 3.

Results

Application of the DMAIC cycle

With the use of the tool based on the Six Sigma methodology, developed in the phases of the DMAIC cycle, in each of two projects: a) institutional project, which outlined the "Workshop on management of hospital supply", elaborating there the category of problems designated "normative" and b) quality of medical product project, stemming from the application of the DMAIC cycle on the "normative" category, where the item "Problems with the products" was made evident and which had quality as the main problem explored. Thus, in the first phase of the DMAIC cycle "Definition", in this project, a strategy to create an instrument to evaluate medical material was adopted consolidating the present study.

Construction and validation of the instruments

In this study, the Delphi technique was used, in three stages, to obtain data from professional specialists and judges of the research.

Delphi Stage 1 - Initial Validation

This stage was outlined in two rounds. Initially, a questionnaire was elaborated, based on the venous infusion equipment, which is considered a product of wide use in the health area. The instrument consisted of three questions, with a total of 55 items, to indicate those items which would compose the evaluation instrument. The construction of the questionnaire was supported by the standards of the Brazilian Association of Technical Standards (ABNT), by the related literature and by prints of medical-hospital materials previously developed at the University Hospital of Londrina. At this stage, invitations were sent to 195 professionals, with 157 specialists participating in the first round. The initial result showed that 13 (23.6%) of the 55 items had not reached the established consensus of 80% concordance. Therefore, they were returned in the second round to 121 (77%) specialists who had not indicated one or more of these items.

With the return of 98 (80%) specialists, the established percentage was obtained. This number, added to 36 (23%) specialists that did not participate in the second round, since they had already indicated all 13 items, totaled a sample of 134 specialists, ending the Delphi stage 1. In the Delphi technique abstention of 30% to 50% of respondents in the first round and 20% to 30% in the second can typically occur⁽⁹⁾. In this study, there was the abstention of 38 (19.5%) specialists in the first round and 23 (19%) in the second round of the Delphi stage 1.

Delphi Stage 2 - Validation of content

From the result of the Delphi stage 1, a model of EIMHP was developed, which was sent, in Delphi stage 2, to five judges for content validation, which occurred in two rounds. In the first part of the instrument, denominated "General Information" (Figure 1), the items were analyzed for the level of concordance using four criteria: a) Not relevant b) Slightly relevant c) Relevant and d) Very relevant. It was determined that the statements "Relevant" and "Very relevant" would be accepted as sufficient to validate answers. The results showed that for these items there was no indication for the options "Not relevant" and "Slightly relevant"⁽¹¹⁾.

The validation of the second part of EIMHP – "specific items" (Figure 2) of the infusion equipment occurred through the use of six attributes: objectivity, clarity, relevance, precision, credibility and variability. These attributes were scored with the predetermined criteria: a) contemplates the requirement b) contemplates the requirement, but needs minor alteration; c) unable to contemplate the requirement, and d) does not contemplate the requirement^(11-12).

In the first round of this stage, the comments and suggestions of the judges were related to the changes in the wording, to standardize the terms, such as "colored for coloration"; to exclude terms like "have" and to promote other changes, as shown in Figure 3. In the second round, suggestions and amendments were sent to judges, obtaining the consensus of all.

Delphi stage 3 – Application of the formula

Given the validity of the content, in a third Delphi stage, the judges applied the EIMHP to three brands of Simple equipment (A, B, C) and three brands of Reservoir equipment (D, E, F). The evaluation took place, indicating the options presented, that have scales of values, in the instrument, as shown in Table 1.

Table 2 summarizes the evaluations, showing the means of all topics and the final mean obtained by the brands tested by the judges.

Discussion

Hospital risk management, from the standpoint of quality and safety, permeated the basis of this study. This is a current issue that has been widely discussed in scientific papers^(13-15) in recent years, intrinsically related to safety and quality in health procedures. Accordingly, Pharmacovigilance has a greater approach, when the quality and safety in the act of medication is the main focus. Regarding studies related to Technovigilance it was observed that, despite the many problems experienced in practice by health professionals, studies on the quality of medical-hospital materials and the difficulties of their use are scarce.

In developing an evaluation instrument, the scientific and validation methodological procedures are essential steps for achieving this. Content validation seeks to value the opinion formers and recognizes the invaluable contribution of the experts who possess knowledge on the subject. The selection stage of these professionals should be designed with accuracy and responsibility^(16-17). In this study, the information obtained from qualified professionals that make up the Brazilian Network of Sentinel Hospitals/ANVISA contributed to value this condition.

The Delphi technique has been applied widely in the construction and adjustment of measuring instruments, and is characterized by flexibility, in which the researcher establishes the rules regarding the: number of phases, number of specialists and level of consensus to consider the instrument valid^(9-10). It is used when there is a lack of data, when the requirement is to stimulate new ideas, when there is a need to promote a multidisciplinary approach or when there is lack of consensus on a determined subject⁽⁹⁾. In this study, the three stages and the four rounds performed allowed the methodological purposes to be fulfilled and the purposes of the construction and validation of content of the EIMHP to be met.

Observing the means of the topics for each brand, the values varied within the mean limits of scores. This demonstrates that the application of the EIMHP allowed the differentiation between brands and that these analyzes were very similar among the judges. The means of each topic varied between the concepts "Regular", "Good", "Very good" and "Excellent" in two brands (B and F). Even with this variation, the results of the general mean of these brands were of concepts higher than expected. This was due to the mean score, which has a range that makes such a measure possible.

The application of the instrument revealed differences in relation to the results of the brands, especially for the numerical values, an important factor to make an objective evaluation. However, the attributes of qualification contributed to the immediate comprehension of the results, conforming to the values presented in Table 1. The general means showed that the six models of materials received approval, considering that all concepts had remained within the established parameters, where the lowest mean attributed was of the concept "good" in two models and four received the concept "Very good". The result of the evaluation of the EIMHP is adequate and differentiates the evaluations between brands tested, which, by definition, were different.

Conclusion

The application of Six Sigma methodology with the use of the DMAIC tool proved important to the extent that the systemic rational for the analysis of the problems was a necessary condition and contributed to improving intersectoral relationships, when collaborators in the administrative area began to better comprehend the care needs. The reverse also occurred when care professionals comprehended the administrative activities involved in each problem categorized. Through the results of the Delphi technique and the participation of the professionals of the Sentinel Network, who carefully and critically demonstrated their opinions and experiences. These opinions cannot be considered absolute truths, but represent important views of specialists who operate in the context of health.

The construction of an instrument for the evaluation of a medical-hospital product – in this study, infusion equipment – was based on the need to obtain a validated model for widespread use in health institutions and had to be able to support one of the steps of the acquisition of that product. To evaluate individual component parts as to their purpose and the ability of the equipment to fulfill its requirements, will certainly benefit the nursing team, who use these for their work routine. The validated model subsidizes the stage of pre-acquisition and enables the use of a better quality product which is, in principle, safer for the patient and the nurse.

The research findings permit the conclusion that the instrument developed is adequate for evaluating infusion equipment. It also permits the consideration that contributes to facilitate the activity of medication, when the performance of the different models and brands of infusion equipment used by the nurse are perceived and distinguished in the practice. The possibility of applying the methodology of this study when developing instruments to evaluate other medical-hospital materials will contribute to their quality and safety in use.

It is important to emphasize that, for any measuring instrument, the evaluation of validity should be a continuous process, endless in its application, in order to verify the requirements for adaptation or reformulation, according to the different realities in which it is used. Considering the limited scientific contributions in the study area, it is concluded that disclosure of this study collaborates with the theme "health safety and risk management in the area of Technovigilance".

Received: Jul. 1^st 2009

Accepted: Jul. 16^th 2010

• ¹
  Conselho Nacional de Secretários de Saúde - CONASS. Ciência e Tecnologia em Saúde, Brasília: CONASS; 2007. v. 4.
• 2. Florence G, Calil SJ. Uma nova perspectiva no controle dos riscos da utilizaçăo de tecnologia médico-hospitalar. Rev MultiCięncia. 2005; (5) [Acesso em: 14 set. 2008] Disponível em: URL: http://www.multiciencia.unicamp.br/intro_05.htm
• 3. Peterlini MAS, Pereira SR. Os Erros Humanos: Abrangęncia e Tipos. In: Harada MJCS, et al, organizadoras. O Erro Humano e a Segurança do Paciente. Săo Paulo (SP): Atheneu; 2006. p. 19-26.
• ⁴
  Ministério da Saúde (BR). Agência Nacional de Vigilância Sanitária, Rede Sentinela [Acesso em: 15 jul. 2008] Disponível em: URL:http://www.anvisa.gov.br
• 5. Adami NP. Componentes da Qualidade e a Prevençăo do Erro Humano. In: Harada MJCS, Pedreira MLG, Peterlini MAS, Pereira SR, organizadoras. O Erro Humano e a Segurança do Paciente. Săo Paulo (SP): Atheneu; 2006. p. 4360.
• 6. Macdaniels T. Concepts and Analytical for Risk Analysis and Management. Horizons Pol Res Clin [serial on line] 2002 5(3):20-2. [Acesso: 26 jul.2004] Disponível em: URL:http://www.policyresearch.gc.ca/v5n3_e.pdf
• 7. Pande PS, Neuman RP, Cavanagh RR. Estratégia Six Sigma: como a GE, a Motorola e outras grandes empresas estăo aguçando seu desempenho. Rio de Janeiro: Qualitymark; 2007.
• 8. Santos AB, Martins MF. Modelo de referęncia para estruturar o Seis Sigma nas organizaçőes. Rev Gestăo e Produçăo 2008 janeiro-abril; 15(1):43-56.
• 9. Wright JTC, Giovinazzo RA. DELPHI Uma ferramenta de apoio ao planejamento prospectivo. Cad Pesqui Admin 2000 abril-junho; 1(12):54-65.
• 10. Couto GR. Nurses conceptualization on childbirth preparation. Rev. Latino-Am. Enfermagem. 2006 março-abril; 14(2):190-8.
• 11. Fernandes MVL. Indicadores de avaliaçăo de práticas de controle e prevençăo de infecçăo do trato urinário associado a cateter. Construçăo e validaçăo. [dissertaçăo]. Săo Paulo (SP): Escola de Enfermagem da Universidade de Săo Paulo/USP; 2005.
• 12. Vituri DW. Desenvolvimento e Validaçăo de um instrumento para avaliaçăo da qualidade do cuidado de enfermagem. [dissertaçăo]. Maringá (PR): Universidade Estadual de Maringá/UEM; 2007.
• 13. Silva D, Grou CR, Miasso AI, Cassiani SHB. Medication preparation and administration: analysis of inquiries and information by the nursing team. Rev. Latino-Am. Enfermagem. 2007 set.-out.; 15(5):1010-7.
• 14. Schatkoski A, Wegner W, Algeri S, Pedro ENR. Safety and protection for hospitalized children: literature review. Rev. Latino-Am. Enfermagem. 2009 june; 17(3):410-6.
• 15. Costa LA, Valli C, Alvarenga AP. Medication dispensing errors at a public pediatric hospital. Rev. Latino-Am. Enfermagem. 2008 oct.; 16 (5):812-7.
• 16. Galdeano LE, Rossi LA. Validaçăo de conteúdo diagnóstico: critérios para seleçăo de expertos. Cięnc Cuidado Saúde. 2006 jan.-fev.; 5(1):60-6.
• 17. Vargas D, Luis MAV. Construçăo e validaçăo de uma escala de atitudes frente ao álcool, ao Alcoolismo e ao alcoolista. Rev. Latino-Am. Enfermagem. 2008 set.-out; 16(5):895-902.

Publication Dates

• Publication in this collection
  01 Dec 2010
• Date of issue
  Oct 2010

History

• Accepted
  16 July 2010
• Received
  01 July 2009