POST-DISHCARGE SURVEILLANCE IN SURGICAL SITE INFECTION: VALIDATION OF AN INSTRUMENT

Paulo, Brasil. ABSTRACT Objective: to create and validate an instrument for detecting potential cases of surgical site infection through post-discharge telephone surveillance. Method: a methodological study using psychometric analyzes to develop and validate an instrument for conducting post-discharge surveillance of surgical site infection. Results: the instrument had a total content validity coefficient equal to 0.87. It was applied to a sample of 100 patients and compared to a medical and nursing physical examination to detect surgical site infection, resulting in satisfactory Cohen’s kappa (0.83), Cronbach’s alpha (0.87) and Comparative Fit Index (0.998). The difference between the time spent on telephone calls for patients positive for surgical site infection was statistically greater than the time spent on calls for patients negative for surgical site infection (p <0.001). Sensitivity was 76.4%, with specificity of 100%, negative predictive values of 92.5%, positive values of 100% and accuracy of 94%. Conclusion: the instrument was validated in content, criteria and


INTRODUCTION
Health Care-Associated Infections (HAIs) are recognized as a public health concern; therefore, it is necessary that regional and national authorities develop actions to reduce the risk of their acquisition. [1][2][3] In the United States of America (USA), Surgical Site Infection (SSI) is already the most common, corresponding to 31% of HAI among hospitalized patients, resulting in increased costs related to health care and patients' hospital length of stay. [4][5] In Brazil, SSI ranks third among HAIs and increases the risk of death by two to eleven times. 4 It is noteworthy that, with surgical hospitalizations increasingly shorter, patients recover partially at home, and thus it is estimated that 19% to 84% of SSIs are diagnosed after hospital discharge. 6 The Unified Health System (SUS -Sistema Único de Saúde) is responsible for most of surgeries performed in Brazil. According to data from the Ministry of Health, in 2019 about 2.4 million surgical procedures were performed by SUS hospitals or accredited by it. 7 Given the large number of surgical procedures performed and considering that most cases of SSI will manifest after discharge, major challenges are posed for conducting Post-Discharge Surveillance (PDS). Among them, we can highlight the lack of structural, human and financial resources. A recent study that assessed PDS in 193 university hospitals found that only 29.3% (n=22) institutions reported performing the PDS, the preferred method being the telephone, followed by outpatient return. 8 Thus, it is necessary to implement PDS actions, preferably through active search methods, with validated instruments, which are able to provide reliable data on SSI incidence. These results will allow the assessment of SSI prevention and control actions instituted during hospitalization, in addition to reducing the occurrence of underreporting. [7][8] Therefore, this study proposed to create and validate an instrument for the post-discharge detection of potential cases of SSI, through telephone PDS.

METHOD
This is a methodological study, with the purpose of elaborating and validating an instrument for carrying out PDS of possible cases of SSI using psychometric analyzes.
For that, psychometric concepts were used, which relate theories and measurement techniques to validate instruments. 9 In this investigation, content validation, concurrent criterion and construct concepts were applied.
Content validity allows assessing, through specialists in the thematic area or construct in question, whether a set of specific items reflects a domain of content, i.e., whether a constructed instrument reflected the conceptual definition applied to the scale. To this end, development of a set of items, validation by expert judges and analysis of content validation were carried out. 9 The construction of a set of items took place through review of national and international guidelines. 6,10-11 Four questions were prepared with different wording that addressed the same clinical indicator, 9 so that judges could choose the formulation that best met the proposed assessment criteria, such as relevance, clarity and comprehensiveness. 9 The instrument was made available to judges through an electronic platform, which allowed access to the commitment term for participation in this research, the letter of guidelines for assessment and the instrument itself.

4/12
The recommendation was adopted to indicate at least three and at most five expert judges for the composition of a judgde panel for instrument validation. 12 Thus, in the present investigation, five judges were selected, according to their expertise and experience in the study area, i.e., professional performance linked to the perioperative period and/or HAI, with lato sensu or strictu sensu postgraduate studies and scientific production in the field.
The agreement between judges on the aspects proposed for instrument assessment was assessed using Content Validity Coefficient and Total Agreement (tCVC), a method used to individually obtain the coefficient for each assessed criterion (icCVC), with the tCVC of each item proposed in the instrument, according to the formula below: Thus, the final instrument was composed of the 10 items that had the highest tCVC values. After this stage, concurrent criteria validation was carried out in a private hospital located in the state of São Paulo, with 300 surgical procedures/month of medium and high complexity. The correlation between the proposed instrument and the medical and nursing physical examination for SSI detection was assessed (gold standard). 3 A convenience sample of 100 patients older than 18 years was used. The sample composition in the stage of validation of competing criteria met the recommendation of inclusion of 10 patients for each item that comprised the instrument's final version. 13 Patients undergoing potentially contaminated and contaminated surgery, who had telephone contact and who assisted one of the researcher's five telephone contact attempts were included.
Data collection took place between April and June 2017, using the Vigi-A instrument through telephone contact between the 14 th and 16 th postoperative days. Then, an outpatient return was scheduled for the medical and nursing consultation, aiming at comparing the responses obtained in the telephone interview with patients' medical and nursing exam.
In the construct validation stage, the instrument's reliability was calculated, i.e., its coherence and its constancy of results, measured using Cronbach's alpha coefficient, which measured the correlation between the instrument items. 14 The result can have a maximum value of 1, with the minimum acceptable value equal to 0.7. 15 Cohen's kappa coefficient was used to measure the agreement between two different types of measurements, such as the instrument created and the physical examination (gold standard). 3 The Confirmatory Factor Analysis (CFA) model, estimated by weighted least squares, was used to verify the patterns of correlations between variables. 14 Moreover, the Comparative Fit Index (CFI) was verified, considering that values above 0.90 indicate a good adjustment, and the Root Mean Square Error of Approximation (RMSEA), an absolute fit index that compares a hypothetical model to a perfect model, adopting as good fit parameters values < 0.05 and values < 0.08 to indicate a reasonable adjustment. [16][17] In addition to the psychometric analysis described above, descriptive statistical measures were employed. Furthermore, accuracy, sensitivity, specificity, predictive and negative values of the instrument were calculated. Welch's t test was used to compare the means observed in the different methods used for post-high surveillance with distinct standard deviations, due to the inequality of the observed variances. The significance level adopted was α=5%. Calculations were performed using SPSS, 22 (IBM, Corp, Armonk, New York, USA) and R 3.5 (R Foundation for Statistical Computing, Vienna, Austria).

5/12
All expert judges and patients participating in the present investigation received information regarding the research objectives and the possibility, at any time, of giving up their participation in this study, without any kind of harm or loss, and expressed their agreement. The judges signed a commitment term, and patients signed an Informed Consent Form.

RESULTS
Five expert judges made up the panel of evaluators, four nurses, two of whom work in HAI control and two in perioperative nursing, and an infectious disease physician.
All of them took stricto sensu and lato sensu graduate courses, one (20%) with specialization in the area of prevention and control of HAI, two (40%) with master's degree in perioperative nursing and two (40%) with a PhD in nursing and medicine. The time of experience in the area varied between two and more than thirty years. All professionals were from the state of São Paulo.
The questions asked about each of the items assessed (purulent drainage, edema, abscess, localized heat, hyperthermia, pain, redness, increased sensitivity, dehiscence and drainage by drain) were assessed by judges using a Likert-type scale, according to their relevance, clarity and comprehensiveness.
The questions that obtained a tCVC index equal to or greater than 0.80 (37.24%) remained unchanged in the instrument's final version, called Vigi-A.
The "localized heat" and "abscess" criteria were reformulated, considering judges' suggestions, as they did not obtain acceptable agreement rates on any of the questions in the set of items prepared for the first assessment.
The question reformulated in the criterion "abscess" had an index equal to 0.80. The question regarding the "localized heat" criterion had a satisfactory index of 0.92. The instrument's Agreement Validity Coefficient and Total Content was equal to 0.87. Chart 1 presents the final instrumental (Vigi-A) composed of ten questions.
Concurrent and construct criteria validity were performed with a sample of 100 patients, mostly women (58; 58%), who underwent potentially contaminated surgeries, especially hysterectomies, rectosigmoidectomies and cholecystectomies. Patients had a mean age of 45 (SD ± 16 years), with minimum values of 17 years and maximum of 81 years. The mean surgical time was 117 minutes (SD ± 53 minutes), ranging from 40 minutes to 300 minutes.
However, when submitted to physical examination by the nurse and subsequent confirmation by the surgeon, only 20; 20% of the cases maintained the diagnosis for SSI, with 100% of them being superficial SSI.

Há pus no líquido do dreno?
Outras informações e observações relevantes: The mean time of telephone calls for the application of Vigi-A with patients with confirmed SSI was 3 minutes and 2 seconds (standard deviation of 0.73 seconds), varying between 2 minutes and 13 seconds to 5 minutes. For patients who did not show symptoms of SSI, the calls lasted between 2min and 13s and 3min and 50s, with an average of 2min and 55s. There was a statistically significant difference between the time spent on calls made for patients who had SSI and those who did not (p=0.001), and for patients with SSI the telephone call was 1.4 times longer than for those without SSI (p <0.001), according to Figure 1. Internal consistency, i.e., the questionnaire's reliability, measured by Cronbach's alpha, was 0.87. Cohen's kappa was 0.83, indicating equivalence between the two measurement instruments used in this study, Vigi-A and outpatient return with physical examination of patients.
In the Combinatorial Factor Analysis, two items, abscess and purulent drainage by drain, were excluded, as the patients included in the sample did not present these complications. The other items are shown in Figure 2. CFI was 0.998, a value considered good; in contrast, the Root Mean Square Error of Aproximation was 0.130, considered insufficient. Vigi-A had a sensitivity of 76.9% and specificity of 100%. The positive predictive value was equal to 100% and the negative predictive value was equal to 92.5%. Its accuracy was 94% for the detection of SSI, through PSD by telephone search.

DISCUSSION
The instrument proposed in this study, called Vigi-A, obtained in its tCVC content validation equal to 0.87, in the assessment of items that addressed signs and symptoms characteristic of all types of SSI, regardless of their topography.
In the analysis of its reliability, the instrument obtained a satisfactory Cronbach's alpha; in an analysis of agreement between the proposed instrument and a gold standard, considering the outpatient return with physical examination, obtained an equally satisfactory result, indicating that Vigi-A adequately measured what he proposed. 9,15 Combinatorial Factor Analysis demonstrated that the instrument items obtained acceptable indexes of adjustment, confirming the clinical impression that the item purulent drainage is the most important for suspicion of cases of SSI, which is in line with that described in scientific literature. 1,11 Although the sample size followed that indicated by the scientific literature for instrument validation studies, 13 it may have influenced the unsatisfactory result of the Root Mean Square Error of Aproximation, as it is known that this test tends to reject true models, when the sample is relatively small. 17 Using the same gold standard used by this study in comparison to telephone calls to PDS, a previous survey 16 observed a reliability of 0.84, concluding in its investigation that telephone PDS is a reliable method for identifying cases of SSI. Previous analyzes assessing PDS by telephone found sensitivity data ranging from 73% to 100%. 18 Although previous studies have worked with some stage of validation in the construction of instruments aimed at preventing SSI, [18][19][20] it can be said that this is the first national study that covered all stages of content validation, criteria and construct for creating a PDS instrument for use over telephone calls.
Among the investigations that analyzed PDS instruments, research carried out in Tanzania sought to assess post-discharge follow-up phone calls with the application of a questionnaire, aiming to improve the detection of post-cesarean SSI cases. It was observed that the method's sensitivity was 73.3%, identifying 26.3% of these cases. 18 In fact, telephone PDS collaborates to detect cases of SSI, especially those considered superficial, which would be underreported without carrying out this active surveillance, as observed in the present study, given that 20% of cases of SSI in the analyzed institution would have been underreported without PDS.
Likewise, a previous study performed only the content validation of an instrument composed of 40 open and closed questions for telephone PDS, directed to the specific diagnosis of endometritis or other events expected between gynecological surgeries. It was found that, of the 140 cases of SSI found in a sample of surgical patients, 62.9% were diagnosed after discharge and only 27.7% during hospitalization. 19 A study constructed and performed the content validation of a checklist of items related to the prevention of SSI, to be applied in the intraoperative period, together with the checklist of safe surgery. 20 It should be noted that the checklist was incorporated into the health unit's computerized system. 20 In Brazil, a survey of 84 patients, from the time of admission to seven days after their discharge, identified that 41 cases of SSI were diagnosed during PDS, highlighting that the increasingly early hospital discharge impacts on the monitoring of possible SSIs or any other adverse event. 21 Understanding that the World Health Organization (WHO) considers HAI to be one of the most frequent adverse events in the world, although underreporting rates are important and motivated by 9/12 several reasons, the difficulty of collecting reliable data is highlighted. Thus, acting to ensure patient safety implies not only ensuring adequate care, but also monitoring data through health surveillance, including PDS. 22 Considering that SSI is a problem for the safety of patients undergoing surgical procedures, coupled with the growth in outpatient and early discharge surgeries, a need arises for collective multiprofessional efforts to adequately notify these events, outline strategies with a focus on communication and educational practice, in order to disseminate information about the notification process. 23 For this reason, it should be noted that quality surveillance by SSIs does not necessarily require high investment, but mainly presupposes standardization of protocols and improvement of practices to conduct the programs proposed with quality.
As a limitation of this study, instrument validation stands out among a convenience sample, composed of patients submitted to potentially contaminated and contaminated procedures.
Thus, it is suggested to carry out further investigations in other groups of patients, expanding the validity of the instrument compared to other surgical procedures and professional realities.

CONCLUSION
The developed instrument Vigi-A was validated in the content, criteria and construct phases for the population covered in the study.
It is believed that PDS presented here will be useful for detecting potential cases of SSI, especially those considered superficial, which are often underreported in health services.
Thus, with the use of a validated instrument and the standardization of a process for PSD of SSI, it is possible to establish judicious comparisons of SSI rates between different institutions.