Surgical Site Infection in Patients Submitted to Orthopedic Surgery : The NNIS Risk Index and Risk Prediction

The applicability of the risk index for surgical site infection of the National Nosocomial Infection Surveillance (NNIS) has been evaluated for its performance in different surgeries. In some procedures, it is necessary to include other variables to predict. Objective: to evaluate the applicability of the NNIS index for prediction of surgical site infection in orthopedic surgeries and to propose an alternative index. The study involved a historical cohort of 8236 patients who had been submitted to orthopaedic surgery. Statistical analysis was performed using multivariate logistic regression to fit the model. The incidence of infection was 1.41%. Prediction models were evaluated and compared to the NNIS index. The proposed model was not considered a good predictor of infection, despite moderately stratified orthopedic surgical patients in at least three of the four scores. The alternative model scored higher than the NNIS models in the prediction of infection.


Introduction
Surgical site infection (SSI) is the second or third most frequent infection among surgical patients.It is responsible for approximately 17% of all healthcarerelated infections (1) .In Brazil, SSI ranks third among infections at health services and corresponds to between 14% and 16% of infections among hospitalized patients, with an 11% incidence rate (2) .SSI related to orthopedic procedures represents a severe and catastrophic complication for patients, surgeons and hospital institutions, as an infection can extent the patient's hospitalization time by up to two weeks, double re-hospitalization rates, increase care costs by more than 300%, besides causing important physical limitations that significantly reduce patients' quality of life after the surgery (3) .Incidence levels of orthopedic SSI can range between 0.8 and 71% (4)(5)(6)(7)(8)(9) .risk factors for patients or procedures that entail greater risks for infection, they can plan preventive actions and control strategies that result in decreased infection rates (10) .

SSI control constitutes a quality indicator of surgical
In the 1970's, the Center for Disease Control and Prevention (CDC) in Atlanta proposed the National Nosocomial Infection Surveillance (NNIS) System for the epidemiological surveillance of hospital infections -IHs (11) .
The NNIS system has been developing indexes to predict infection risk in the surgical patient population, considering uncontrollable extrinsic and intrinsic risk factors.The range of risk factors in surgical patients needs to be used as a parameter to adjust the ratios (12) .
Since 1997, studies demonstrate that the NNIS risk Index has not complied with its function of predicting the surgical site infection risk of specific procedures in an accessible, simple and objective way, with good discriminatory power (19)(20) .
The NNIS Infection Risk Index has been applied in Brazilian studies that aimed to predict infection risks in some types of specific surgeries, such as cardiothoracic, cardiovascular, digestive, neurological and pediatric surgeries (17)(18)(19)(20) .Results were controversial.In all of these studies, other specific variables had to be included for each type of procedure, which permitted the construction of alternative surgical infection risk prediction models.
One of the reasons to assess the infection prediction power of the NNIS Risk Index in specific surgeries is due to its easy applicability in daily hospital practice (15) .
The goal of this study was to assess the NNIS Risk Index to predict SSI in patients submitted to orthopedic surgeries, as well as to promote an alternative index for application at the study hospitals.

Design and Study Variables
In a historical cohort, information on 8,236 patients submitted to general orthopedic surgical procedures, classified as NNIS procedures.This were inserted in the database of a hospital infection control program called Computerized Hospital Infection Control System -SACIH (15) .These patients were attended at four general and teaching hospital for tertiary care delivery, located in different regions of Belo Horizonte, Minas Gerais,

Brazil.
The response variable under analysis was the presence or absence of SSI.The following independent variables were assessed: hospital (coded as 0,1,2,3); ASA (I, II, III, IV and V, according to the American

Statistical Analysis
STATA 8.1 software was used for statistical data analysis (21) .Simple frequency distribution, central trend measures (mean or median) and variability measures (standard deviation, quartiles and minimum and maximum values) were used to characterize and describe the patients submitted to orthopedic surgeries.
Global incidence levels were calculated per NNIS Surgical Infection Risk Index category, hospital and type of surgical procedure.For calculation purposes, the number of SSI cases among orthopedic surgical patients was used as the numerator, while the total number of orthopedic surgical patients during the study period was used as the denominator.
Univariate and multivariate analyses were conducted, using logistic regression with a view to developing SSI prediction models (22) .In the multivariate analysis, initial modeling was based on the variables selected in the univariate analysis (statistical association www.eerp.usp.br/rlaeRev. Latino-Am.Enfermagem 2011 Mar-Apr; 19(2):269-76.
with SSI and p≤0.20), as well as variables that did not show statistically significant differences but are described in literature as associated with surgical site infection.Variables with more than two categories were transformed into indicative variables called "dummies" (22) .
First, the complete model was constructed.Then, the variables were removed step by step until the final model was defined.To define the best final model, the likelihood-ratio test was used, as well as the β (Beta) coefficients, odds ratio (OR) and p<0.05 (22) .
ROC "Receiver Operating Characteristic Curve" analysis was the method chosen to assess the accuracy (discriminatory power between individuals with and without the event under analysis) of the alternative SSI prediction models) (23) .To check the strength of the correlation between the NNIS Surgical Infection Risk Index and SSI occurrence, the Goodman-Kruskal (G) test -Gamma Coefficient was used.This test is particularly adequate to analyze variables with an ordinal measurement level, like in the case of the risk ratio under analysis.It ranges between -1 and +1.If the analyzed variables are independent, the coefficient is close to zero (24) .

Characterization of orthopedic surgical patients and SSI Incidence
The patients' average age was 34.6 years (sd: 27), ranging from 0 (younger than 1 year) to 99 years, with a median age of 30 years (quartiles 1 and 3: 10 and 57).The mean preoperative hospitalization time was 3.8 days (sd: 25.6), ranging from 0 to 595 days, with a median time of 1 day (quartiles 1 and 3: 0 and 1).
Among the 8,236 patients who submitted to orthopedic surgical procedures, 116 surgical site infections (SSI) were identified.The global infection incidence level was 1.41% (95% confidence interval: 1.18 -1.76) for the study period.
SSI incidence levels for the orthopedic procedures were: 2.2% for PROS-Q and PROS-O; 1.7% for OMS; 1.5% for PROS-J; 1.4% for FUS and 1.1% for FX.

NNIS Surgical Infection Risk Index
SSI incidence levels per NNIS Index Ratio category were: score 0 = 1.1%; score 1 = 1.8%; score 2 = 2.8%; score 3 = 5.3%.A linear increase in incidence levels was observed to the extent that the risk factors of the NNIS Index increase.An assessment of the NNIS Index' efficiency in the 8,236 orthopedic surgical patients showed that this ratio layered patients in only three of its four risk categories.
The analysis of the NNIS Risk Index for specific orthopedic procedures (FUS, FX, OMS, PROS-Q, PROS-O and PROS-J) revealed the inefficiency of this ratio to stratify patients submitted to these six procedures among its four categories.This fact may be due to the characteristic of the study sample or the Index' deficient SSI prediction when the even under analysis is rare.The NNIS Index divided patients in only two layers (score 0 and score 1).Less than 8% of patients were classified in score 2. No patients were classified in score 3 for FUS, PROS -J, PROS -O and PROS -Q procedures.
In the NNIS Risk Index variable, score 0 (patients without any risk factor) was considered a reference for the univariate analysis.An upward trend in the OR was observed as the number of risk factors for each score increases (Table 1).Nevertheless, patients classified as score 3 showed OR=5.2 (95% confidence interval: 0.7-39.4,p=0.11).This score, however, showed no statistical significance for SSI, as only one infected patient was classified as score 3. Ercole FF, Chianca TCM, Duarte D, Starling CEF, Carneiro M.
The accuracy of the NNIS Index was also assessed through the ROC "Receiver Operating Characteristic Curve".The area under the curve represents the distinction of all possible pairs of individuals with SSI or not.The probability is measured that a patient with SSI will present a higher (estimated probability) than the of a patient without infection.The higher the area, the greater the corresponding model's predictive capacity.
The model called NNIS Index (ASA, surgical wound contamination potential and duration of surgery) displayed 0.58 for the area under the corresponding curve, indicating low power to predict infected patients.
The NNIS Risk Index was analyzed through the Gamma Coefficient.The result (G=0.31;95% confidence interval: 0.154-0.455,p=0.000 for Chi-square -χ 2 distribution with 3 degrees of freedom -gl) was considered very low (Table 2), in line with what shown in the area under the ROC Curve (Figure 1).In general, it can be inferred that the NNIS Index moderately stratifies orthopedic surgical patients.The model, however, was not a good infection predictor, as the estimated incidence levels were very low.

Alternative Models
Based on the univariate analysis results, initially, seven variables were selected for the alternative models,  3 shows the logistic regression estimated for this model, represented in the ROC Curve for the sake of a better visualization of the results (Figures 1 and 2).
The likelihood-ratio test indicated that the general anesthesia variable, although not statistically significant, should return to the final alternative model with a view to a better adjustment.
The analysis of the area under the curve in the alternative model shows that it gets closer to the upper left border of the graph when compared with the NNIS model.The alternative model showed an area of 0.75 under the curve, revealing good accuracy or good predictive power of the test to detect patients with SSI (Figures 1 and 2).
It is highlighted that these low levels can reflect effective and consolidated epidemiological surveillance at the research hospitals, but can also derive from infection under-notification, due to the lack of patient control after hospital discharge and to registration problems in the hospital databases (4,20) .
SSI patients stratified according to the NNIS Index showed a low but increasing incidence level.SSI incidence levels rose for scores 0, 1, 2 and 3, corresponding to 1.1%, 1.8%, 2.8% and 5.3%, respectively.Incidence levels were expected to increase with risk factors.Also, growing SSI rates were found in a study as the risk factors of the NNIS Index increased (14) .
The analysis of the NNIS Index' stratification power for the six orthopedic procedure types (FUS, FX, OMS, PROS-Q, PROS-O and PROS-J) showed that it stratified the orthopedic surgical patients in only three layers in four out of six procedures.Different studies have shown inadequacies in the NNIS Index, affirming this index' general inability to predict SSI risk in different types of surgical procedures (16)(17)20) .
The use of the NNIS Index in a wide range of procedures does not permit extending the results to specific populations and procedures.To analyze specific procedures with characteristic peculiarities, more adequate SSI prediction models for these particular situations should be created (16)(17)(18)(19)(20) .
The NNIS Index showed no statistical association with SSI.According to the Goodman-Kruskal (Gamma), low predictive power of the SSI is observed for orthopedic surgical patients (G=0.31).The confirmation of this result is displayed in the area under the ROC Curve, calculated at 58%.In conclusion, the power of the NNIS Index to discriminate true positive patients for SSI is low.A similar result was found in a prospective cohort of digestive surgery patients (20) .
This study proposed an alternative infection risk prediction model for orthopedic surgery patients.
The suggested alternative model contains, besides the three variables of the NNIS Index (ASA, surgical wound contamination potential and duration of the surgery), three other variables (prosthesis, number of patients' epidemiological surveillance.When identifying Ercole FF, Chianca TCM, Duarte D, Starling CEF, Carneiro M. Society of Anesthesiologists' criterion); surgical wound contamination potential (clean, potentially clean, contaminated and infected); duration of surgery (≤120 and ≥120 minutes); prosthesis (no and yes); type of surgical procedure (FUS = fusion and arthrodesis; FX = reduction of open fracture; OMS = other procedures in the musculoskeletal system; PROS-Q = hip prosthesis, PROS-O = other types of prosthesis and PROS-J= knee prosthesis); number of professionals during the surgery (1-4, 5-8, 9-16 professionals); antibiotic prophylactics (no, use of cefazoline, use of clindamycin and associates); trauma (no, yes); general anesthesia (no, yes); age (in years, continuous); preoperative time (in minutes, continuous).The NNIS Surgical Infection Risk Index (comprising ASA, surgical wound contamination potential and surgery duration) was analyzed according to its categories: score 0 (three absent factors), score 1 (only one factor present), score 2 (two factors presents), score 3 (three factors present).This Index attributes scores 0 and 1, according to the presence or absence of the risk factor.
After obtaining the best alternative SSI risk prediction model, the ROC Curve of the new predictive model was compared with the ROC Curve of the NNIS Surgical Infection Risk Index model.It should be highlighted that, at the four hospital, no post-hospital discharged monitoring of surgical patients is accomplished.Approval for the research project was obtained from the Institutional Review Board at UFMG (process ETIC 274).
according to the established statistical significance (p≤0.20).These were: ASA, prosthesis, surgical wound contamination potential, NNIS Surgical Infection Risk Index, antibiotic prophylaxis, hospital and number of professionals during the surgery.Seven other variables of acknowledged importance in literature comprised the multivariate logistic regression analysis.These were: procedure type, trauma, emergency nature of the surgery, general anesthesia, age, preoperative hospitalization time and duration of surgery.Based on these variables, four SSI risk prediction models were specified, which were compared with the NNIS Index model.Only one of the four constructed models effectively predicted SSI risk.All possible interactions among the variables included in the model were tested.To obtain the final alternative model, the researchers decided to remove all statistically significant interactions, such as ASA and hospital, surgical wound contamination potential and prosthesis, hospital and number of professionals during the surgery, due to the collinearity between each interaction and with other variables in the database.Variables like trauma, emergency nature of the surgery, preoperative time, procedure type, age and antibiotic prophylaxis were removed from the full model, one by one in the presented order, with p<0.05.The final alternative model included the following variables: ASA, prosthesis, surgical wound contamination potential, hospital, number of professionals during the surgery, general anesthesia and duration of the surgery.Table

Table 1 -
Logistic regression estimates for NNIS/CDC Surgical Infection Risk Index assessment, Belo Horizonte,

Table 2 -
Assessment of Correlation between NNIS

Table 3 -
Logistic regression estimates to define the Alternative Predictive Model, Belo Horizonte, MG