Risk Management in Technovigilance : Construction and Validation of a Medical-Hospital Product Evaluation Instrument

With the continuous incorporation of health technologies, hospital risk management should be implemented to systemize the monitoring of adverse effects, performing actions to control and eliminate their damage. As part of these actions, Technovigilance is active in the procedures of acquisition, use and quality control of health products and equipment. This study aimed to construct and validate an instrument to evaluate medical-hospital products. This is a quantitative, exploratory, longitudinal and methodological development study, based on the Six Sigma quality management model, which has as its principle basis the component stages of the DMAIC Cycle. For data collection and content validation, the Delphi technique was used with professionals from the Brazilian Sentinel Hospital Network. It was concluded that the instrument developed permitted the evaluation of the product, differentiating between the results of the tested brands, in line with the initial study goal of qualifying the evaluations performed.


Introduction
Currently, the insertion of new technologies is one of the major concerns of hospital management.The World Health Organization estimates that around 50% of all therapeutic advances available did not exist ten years ago.These technologies are available to the health system and incorporated into clinical practice, providing benefits that were unimaginable until recently, with significant results regarding the heightened expectations of improved quality of life for the populations (1) .
It is noted, however, that technological innovation has caused numerous problems, such as: lack of local planning for its incorporation, unsatisfactory quality, little technical and scientific training and competence of professionals, misuse, development out of pace with the evolution of local maintenance services, and high costs, which severely impact the hospital organizations (1) .www.eerp.usp.br/rlaeKuwabara CCT, Évora YDM, Oliveira MMB.
The increasing demand for a health care professional capable of accompanying the assistance and technological development, in many cases, reflects the concern to avoid risk to the health of the patient, which may be associated, or not, with the occurrence of human errors.This calls for effective actions to integrate the management response to this situation, modeling professionals for the new technical activities that reality requires (2) , as the probable or imminent risk that the patient is submitted is not visualized correctly, or even detected in time.In this context, it refers to an event related to medical products, the utilization, the acquisition and use of medical-hospital materials with regards to their quantity and quality (3) , among others.
Hospital risk management (HRM) is presented as a new perspective on the subject, enabling the administrator of health to look at the care that the patient is actually submitted, analyzing, investigating, proposing solutions and implementing actions to try to remedy these problems or, at least pre-empt their occurrence (4) .The purpose of risk management is to identify the likely origin of adverse events, to evaluate the damage caused and to take appropriate decisions pertaining to these issues.
Historically, risk management in health began in the United States of America, from the mid-1920s, in the context of the "crisis of the medical error" when, processes and operating systems focusing on prevention, detection, control or elimination of risks that could cause harm to patients/clients, were generated.
The risk management program developed in that country constituted one of the internal activities aimed at guaranteeing the quality of care provided.It was later implemented in other countries in Europe, Oceania and Latin America (5) .
The health worker is also contemplated in this scenario, where the risk conferred to the patient or the products used in their care can reflect on their own safety.Medical and nursing teams are the ones that suffer most from this reflection, considering their direct contact with these risks (6) .However, other actors, such as workers of the support areas of pharmacy, laboratory, In this sense, quality is a fundamental requirement, which provides the security needed for the required results.To establish a methodology for evaluating a hospital-medical product in the pre-acquisition phase is important and becomes necessary, given that adverse events may even occur in normal situations with good quality products.
Given the wide range of products offered in the Brazilian market and the gap found in the literature regarding validated models of evaluation instruments for medical products, the following research question emerged: "Is it possible to construct and validate, together with the specialists, an evaluation instrument for quality control of medical-hospital materials aimed at preventing potential risks to patients and the health team?".The aim of this study was to construct and validate an evaluation instrument for medical-hospital products (EIMHP) to support risk management in the area of Technovigilance.

Methodology
This is a quantitative, exploratory, longitudinal and methodological development study.The investigation was structured using the quality management techniques with Six Sigma methodology (7)(8) , to support risk management in the area of Technovigilance, and using the Delphi technique (9)(10) , to obtain data.processes or existing products.The DMAIC cycle allows the study of variability in the quality of the processes of the institution through the application in each problem outlined (7)(8) .
The Delphi technique is used to obtain data from professional experts and judges.It is used to determine the content validity and represents a useful tool for the formation of a judgment of a group, being widely used in the construction and adaptation of measuring instruments, as it is characterized by flexibility, in which the researcher responsible for the study establishes the rules regarding the number of stages.It is especially recommended when there are no quantitative data or they cannot be projected into the future with certainty, given the expectation of structural changes or determinant factors of future trends (9)(10) .

Application of the DMAIC cycle
With the use of the tool based on the Six Sigma methodology, developed in the phases of the DMAIC cycle, in each of two projects: a) institutional project, which outlined the "Workshop on management of hospital supply", elaborating there the category of problems designated "normative" and b) quality of medical product project, stemming from the application of the DMAIC cycle on the "normative" category, where the item "Problems with the products" was made evident and which had quality as the main problem explored.Thus, in the first phase of the DMAIC cycle "Definition", in this project, a strategy to create an instrument to evaluate medical material was adopted consolidating the present study.

Construction and validation of the instruments
In this study, the Delphi technique was used, in three stages, to obtain data from professional specialists and judges of the research.With the return of 98 (80%) specialists, the established percentage was obtained.This number, added to 36 (23%) specialists that did not participate in the second round, since they had already indicated all 13 items, totaled a sample of 134 specialists, ending the Delphi stage 1.In the Delphi technique abstention of 30% to 50% of respondents in the first round and 20% to 30% in the second can typically occur (9) .In this study, there was the abstention of 38 (19.5%) specialists in the first round and 23 (19%) in the second round of the Delphi stage 1.

Delphi Stage 2 -Validation of content
From the result of the Delphi stage 1, a model of EIMHP was developed, which was sent, in Delphi stage 2, to five judges for content validation, which occurred in two rounds.In the first part of the instrument, denominated "General Information" (Figure 1), the items were analyzed for the level of concordance using four criteria: a) Not relevant b) Slightly relevant c) Relevant and d) Very relevant.It was determined that the statements "Relevant" and "Very relevant" would be accepted as sufficient to validate answers.The results showed that for these items there was no indication for the options "Not relevant" and "Slightly relevant" (11) .
Kuwabara CCT, Évora YDM, Oliveira MMB.  5. General specifications required for purchase -Summary of technical description of the team and summary of the packaging requirements.
6. Place to note the observations of the evaluator and their identification -Place to explain the observations about the test and its results.
-Data from the evaluator (name, professional category, signature, signature of the area supervisor, and date stamp).
7. General guidelines for completing the evaluation instrument -Items that guide the completion of the evaluation form.
The validation of the second part of EIMHP -"specific items" (Figure 2) of the infusion equipment occurred through the use of six attributes: objectivity, clarity, relevance, precision, credibility and variability.These attributes were scored with the predetermined criteria: a) contemplates the requirement b) contemplates the requirement, but needs minor alteration; c) unable to contemplate the requirement, and d) does not contemplate the requirement (11)(12) .

Atributes of evaluation
Poor Regular Good Very good Excellent 1 2 3 4 5 1 Packaging 1.1 Size of the printed letters are adequate for easy reading 1.2 Ease of identification of the product name 1.3 Ease of identification of the date of fabrication 1.4 Ease of identification of the sterilization date 1.5 Ease of identification of the use by date(s) 1.6 Ease of identification of the batch number.
1.7 Ease of visualization of the contents 1.8 Specific edge that allows aseptic opening 1.9 Edges sealed properly maintaining sterility until use 1.10 Adequate size of packaging (size proportional to the product) 1.11 Adequate storage of the product inside the packaging (folding, kinking, etc.).In the first round of this stage, the comments and suggestions of the judges were related to the changes in the wording, to standardize the terms, such as "colored for coloration"; to exclude terms like "have" and to promote other changes, as shown in Figure 3.In the second round, suggestions and amendments were sent to judges, obtaining the consensus of all.

Adequate size of letters
The size of the printed letters is adequate to allow reading.
Ease of aseptic opening Edge specific to allow aseptic opening Edges adequately sealed Edges sealed adequately maintaining sterility until use Leak from Lateral filter Lateral filter efficient (when present) Effective self-sealing membrane without leak after multiple punctures Effective self-sealing membrane without leak after multiple punctures in 24 hours.

Adequate flow of solution
Durability of the equipment in use at the CIP, conforms to the recommendation of the manufacturer (24, 48, 72h, etc.).

Delphi stage 3 -Application of the formula
Given the validity of the content, in a third Delphi stage, the judges applied the EIMHP to three brands of Simple equipment (A, B, C) and three brands of Reservoir equipment (D, E, F).The evaluation took place, indicating the options presented, that have scales of values, in the instrument, as shown in Table 1.

Discussion
Hospital risk management, from the standpoint of quality and safety, permeated the basis of this study.This is a current issue that has been widely discussed in scientific papers (13)(14)(15) in recent years, intrinsically related to safety and quality in health procedures.Accordingly, Pharmacovigilance has a greater approach, when the quality and safety in the act of medication is the main focus.Regarding studies related to Technovigilance it was observed that, despite the many problems experienced in practice by health professionals, studies on the quality of medical-hospital materials and the difficulties of their use are scarce.
In developing an evaluation instrument, the scientific and validation methodological procedures are essential steps for achieving this.Content validation seeks to value the opinion formers and recognizes the invaluable contribution of the experts who possess knowledge on the subject.The selection stage of these professionals should be designed with accuracy and responsibility (16- 17) .In this study, the information obtained from qualified professionals that make up the Brazilian Network of Sentinel Hospitals/ANVISA contributed to value this condition.
The Delphi technique has been applied widely in the construction and adjustment of measuring instruments, and is characterized by flexibility, in which the researcher establishes the rules regarding the: number of phases, number of specialists and level of consensus to consider the instrument valid (9)(10) .It is used when there is a lack of data, when the requirement is to stimulate new ideas, when there is a need to promote a multidisciplinary approach or when there is lack of consensus on a determined subject (9) .In this study, the three stages and the four rounds performed allowed the methodological purposes to be fulfilled and the purposes of the construction and validation of content of the EIMHP to be met.
Observing the means of the topics for each brand, the values varied within the mean limits of scores.
allowed the differentiation between brands and that these analyzes were very similar among the judges.
The means of each topic varied between the concepts "Regular", "Good", "Very good" and "Excellent" in two brands (B and F).Even with this variation, the results of the general mean of these brands were of concepts higher than expected.This was due to the mean score, which has a range that makes such a measure possible.
The application of the instrument revealed differences in relation to the results of the brands, especially for the numerical values, an important factor to make an objective evaluation.However, the attributes of qualification contributed to the immediate comprehension of the results, conforming to the values presented in Table 1.The general means showed that the six models of materials received approval, considering that all concepts had remained within the established parameters, where the lowest mean attributed was of the concept "good" in two models and four received the concept "Very good".The result of the evaluation of the EIMHP is adequate and differentiates the evaluations between brands tested, which, by definition, were different.

Conclusion
The application of Six Sigma methodology with the use of the DMAIC tool proved important to the extent that the systemic rational for the analysis of the It is important to emphasize that, for any measuring instrument, the evaluation of validity should be a continuous process, endless in its application, in order to verify the requirements for adaptation or reformulation, according to the different realities in which it is used.
Considering the limited scientific contributions in the study area, it is concluded that disclosure of this study collaborates with the theme "health safety and risk management in the area of Technovigilance".
laundry, nutrition and maintenance, are not excluded from this scenario.In 2001, the concept of HRM was introduced in Brazil by the National Sanitary Surveillance Agency (ANVISA), of the Ministry of Health, with the Sentinel Hospitals project, and currently brings almost two hundred institutions together.The adoption of measures to enhance quality of health care and the rational management of supplies and hospital-medical equipment (Technovigilance), with a view to ensuring quality and safety are two of the developments of the project.Other areas, such as Pharmacovigilance (surveillance of drugs) and Haemovigilance (surveillance of blood products), are also part of the project.With the continuing incorporation of health technology, hospital risk management should be implemented for the systemization of the monitoring of adverse events, performing actions for the control and elimination of their damage.As part of these actions, Technovigilance acts in the procedures for acquisition, utilization and quality control of products and equipment in health.

Six
Sigma is principally based on the component phases of the DMAIC cycle: Define; Measure; Analyze; Improve; and Control.This model is used to improve performance through the application of statistical methods and tools in order to: define problems and situations to be improved; measure to obtain information and data, analyze the information collected; incorporate and better understand the processes; and control the www.eerp.usp.br/rlaeRev. Latino-Am.Enfermagem 2010 Sep-Oct;18(5):943-51.
The study project was submitted to the Research Ethics Committee (register CONEP 268, Protocol 129/2008) and approved by the Director Superintendent of the University Hospital of Londrina (HUL), the origin of the study.All participants were told about the research, based on the Free Prior Informed Consent Terms to meet the requirements of Resolution 196/96 of the National Health Council.The unit of research outlined for this study consisted of hospitals belonging to the Sentinel Hospital Project of the National Agency of Sanitary Surveillance/ANVISA, of the Ministry of Health Considering the three Delphi stages performed, the sample totaled 144 participants, with 139 specialists from the Delphi stage 1, 5 judges who participated in the Delphi stage 2 and 5 Judges of the Delphi stage 3.
This stage was outlined in two rounds.Initially, a questionnaire was elaborated, based on the venous infusion equipment, which is considered a product of wide use in the health area.The instrument consisted of three questions, with a total of 55 items, to indicate those items which would compose the evaluation instrument.The construction of the questionnaire was supported by the standards of the Brazilian Association of Technical Standards (ABNT), by the related literature and by prints of medical-hospital materials previously developed at the University Hospital of Londrina.At this stage, invitations were sent to 195 professionals, with 157 specialists participating in the first round.The initial result showed that 13 (23.6%) of the 55 items had not reached the established consensus of 80% concordance.Therefore, they were returned in the second round to 121 (77%) specialists who had not indicated one or more of these items.

Figure 1 -
Figure 1 -General product information -composition of the first part of the Instrument for the Evaluation of Medicalhospital Products.Londrina, PR, 2009

Figure 2 -
Figure 2 -Specific product information -composition of the second part of the Instrument for the Evaluation of Medical-hospital Products.Londrina, PR, 2009

Figure 3 -
Figure 3 -Items of the Evaluation Instrument for Medical-hospital Products changed during Delphi Stage 2 -validation of content.Londrina, PR, 2009 problems was a necessary condition and contributed to improving intersectoral relationships, when collaborators in the administrative area began to better comprehend the care needs.The reverse also occurred when care professionals comprehended the administrative activities involved in each problem categorized.Through the results of the Delphi technique and the participation of the professionals of the Sentinel Network, who carefully and critically demonstrated their opinions and experiences.These opinions cannot be considered absolute truths, but represent important views of specialists who operate in the context of health.The construction of an instrument for the evaluation of a medical-hospital product -in this study, infusion equipment -was based on the need to obtain a validated model for widespread use in health institutions and had to be able to support one of the steps of the acquisition of that product.To evaluate individual component parts as to their purpose and the ability of the equipment to fulfill its requirements, will certainly benefit the nursing team, who use these for their work routine.The validated model subsidizes the stage of pre-acquisition and enables the use of a better quality product which is, in principle, safer for the patient and the nurse.The research findings permit the conclusion that the instrument developed is adequate for evaluating infusion equipment.It also permits the consideration that contributes to facilitate the activity of medication, when the performance of the different models and brands of infusion equipment used by the nurse are perceived and distinguished in the practice.The possibility of applying the methodology of this study when developing instruments to evaluate other medical-hospital materials will contribute to their quality and safety in use.

Evaluation items for venous infusion equipment -second part Atributes of evaluation Poor Regular Good Very good Excellent
3.5 Transparency (allows clear visualization of the solution) 3.6 Adequate coloration for photosensitive medication 3.7 Adequate coloration for infusion of enteral diet (This figure continue in the next page) www.eerp.usp.br/rlaeRev. Latino-Am.Enfermagem 2010 Sep-Oct;18(5):943-51.

Table 1 -
Composition of values of the EIMHP, according to the attributes of qualification and the mean score, for use in evaluating the equipment by the judges.Londrina,

Table 2 -
Results of the evaluation of Simple and Reservoir equipment of the brands A, B, C, D, E and F, according to mean scores, by topic, general mean of the brand and attributes of qualification.Londrina, PR, 2009