Applicability of the simplified acute physiology score (SAPS 3) in brazilian hospitals

Abstracts

BACKGROUND AND OBJECTIVES: The SAPS 3 (Simplified Acute Physiology Score 3) prognostic system is composed of 20 parameters, represented by an acute physiology score and assessment of the previous status, aimed at establishing a predictive mortality index for patients admitted to intensive care units (ICU). The objective of this study was to validate this system and determine its discriminatory power in surgical patients in Brazil. METHODS: This is a prospective study undertaken in two surgical ICUs of two different hospitals over a one-year period; patients younger than 16 years, who stay at the ICU for less than 24 hours, readmitted to the unit, and those admitted for dialysis were excluded from the study. The predictive ability of the SAPS 3 index to differentiate survivors and non-survivors was determined by the ROC curve and calibration by the Hosmer-Lemeshow goodness-of-fit test. RESULTS: One thousand three-hundred and ten patients were included in the study. Gastrointestinal surgeries predominated (34.9%). Eighteen was the lower SAPS 3 index and the highest was 154, with a mean of 48.5 ± 18.1. The predicted and real hospital mortality was 10.3% and 10.8%, respectively; the standardized mortality ratio (SMR) was 1.04 (95%CI = 1.03-1.07). Calibration by the Hosmer and Lemeshow method showed X² = 10.47 p = 0.234. The SAPS 3 score that better discriminated survivors and non-survivors was 57, with sensitivity of 75.8% and specificity 86%. Among the patients with SAPS 3 index higher than 57, 73.5% did not survive versus 26.5% who survived (OR= 1.32, 95%CI 1.23-1.42, p < 0.0001). CONCLUSIONS: The SAPS 3 system is valid for the Brazilian population of surgical patients, being a useful indicator of critical patients and to determine greater care in this group.

INTENSIVE CARE; MORTALITY, Hospital

JUSTIFICATIVA E OBJETIVOS: O sistema prognóstico SAPS 3 (Simplified Acute Physiology Score 3) é composto de 20 variáveis, representadas por escore fisiológico agudo e avaliação do estado prévio, visando estabelecer índice preditivo de mortalidade para pacientes admitidos em unidades de terapia intensiva (UTI). O estudo teve objetivo de validar este sistema e verificar o poder discriminatório deste índice em pacientes cirúrgicos do Brasil. MÉTODO: Estudo prospectivo, realizado em duas UTI especializadas em pacientes cirúrgicos de dois diferentes hospitais, no período de um ano, excluiuse pacientes com idade inferior a 16 anos, que permaneceram tempo inferior a 24 horas na UTI, readmitidos e aqueles admitidos para procedimento dialítico. A habilidade preditiva do índice SAPS 3 em diferenciar sobreviventes e não sobreviventes foi verificada utilizando curva ROC e a calibração pelo teste Hosmer-Lemeshow goodness-of-fit. RESULTADOS: Foram incluídos no estudo 1.310 pacientes. Operações gastrintestinais foram predominantes (34,9%). O menor valor do índice SAPS 3 foi 18 e o maior 154, média de 48,5 ± 18,1. A mortalidade hospitalar prevista e real foi de 10,3% e de 10,8%, respectivamente, razão de mortalidade padronizada (SMR) foi 1,04 (IC95% = 1,03-1,07). A calibração pelo método Hosmer e Lemeshow mostrou X² = 10,47 p = 0,234. O valor do escore SAPS 3 que melhor discriminou sobreviventes e não sobreviventes foi 57, com sensibilidade de 75,8% e especificidade de 86%. Dos pacientes com índice SAPS 3 maior que 57, 73,5% não sobreviveram versus 26,5% de sobreviventes (OR = 1,32 IC95% 1,23 - 1,42, p < 0,0001). CONCLUSÕES: O sistema SAPS 3 é válido na população brasileira de pacientes cirúrgicos, sendo útil para indicar pacientes graves e determinar maiores cuidados neste grupo.

MORTALIDADE, Hospitalar; TERAPIA INTENSIVA

SCIENTIFIC ARTICLE

João Manoel Silva Junior, TSA, M.D.^I; Luiz M Sá Malbouisson, TSA, M.D.^II; Hector L Nuevo, M.D.^III; Luiz Gustavo T Barbosa, M.D.^III; Lauro Marubayashi, M.D.^IV; Isabel Cristina Teixeira^V; Antonio Paulo Nassar Junior, M.D.^VII; Maria Jose Carvalho Carmona, TSA, M.D.^VII; Israel Ferreira da Silva, TSA, M.D.^VIII; José Otávio Costa Auler Júnior, TSA, M.D.^IX; Ederlon Rezende, M.D^X

^ICoordenador da Unidade Crítica de Pacientes Cirúrgicos, Corresponsável do CET/SBA do HSPE; Médico Responsável pela Parte Científica da Unidade de Terapia Intensiva do HSPE; Mestre em Ciências Médicas, FMUSP

^IIMédico-coordenador da UTI Cirúrgica da Disciplina de Anestesiologia do HC da FMUSP; Médico-coordenador da Unidade Cirurgia de Pacientes Críticos do HSPE; Doutor em Ciências Médicas - FMUSP

^IIIME3 do Serviço de Anestesiologia do HSPE

^IVAnestesiologista; Diretor da Cooperativa Médica de Anestesiologistas de São Paulo do HSPE

^VEnfermeira da Unidade Cirurgia de Pacientes Críticos do HSPE

^VIMédico-intensivista, TEAMiB; Diarista da UTI Cirúrgica da Disciplina de Anestesiologia do HC da FMUSP

^VIIProfessora Doutora da Disciplina de Anes tesiologia da FMUSP; Diretora do Serviço de Anestesiologia do HC da FMUSP; Supervisora do Serviço de Terapia Intensiva Cirúrgica do Instituto do Coração (InCor) do HC da FMUSP

^VIIIAnestesiologista; Corresponsável do CET/SBA do HSPE; Responsável pela Residência Médica do IAMSPE; Supervisor do Serviço de Anestesiologia do HSPE

^IXProfessor Titular da Disciplina da Anestesiologia da FMUSP; Diretor do Serviço de Anestesiologia e Terapia Intensiva Cirúrgica do InCor do HC da FMUSP

^XMédico-intensivista, TEAMiB; Diretor do Serviço de Terapia Intensiva do HSPE

Correspondence to

SUMMARY

BACKGROUND AND OBJECTIVES: The SAPS 3 (Simplified Acute Physiology Score 3) prognostic system is composed of 20 parameters, represented by an acute physiology score and assessment of the previous status, aimed at establishing a predictive mortality index for patients admitted to intensive care units (ICU). The objective of this study was to validate this system and determine its discriminatory power in surgical patients in Brazil.

METHODS: This is a prospective study undertaken in two surgical ICUs of two different hospitals over a one-year period; patients younger than 16 years, who stay at the ICU for less than 24 hours, readmitted to the unit, and those admitted for dialysis were excluded from the study. The predictive ability of the SAPS 3 index to differentiate survivors and non-survivors was determined by the ROC curve and calibration by the Hosmer-Lemeshow goodness-of-fit test.

RESULTS: One thousand three-hundred and ten patients were included in the study. Gastrointestinal surgeries predominated (34.9%). Eighteen was the lower SAPS 3 index and the highest was 154, with a mean of 48.5 ± 18.1. The predicted and real hospital mortality was 10.3% and 10.8%, respectively; the standardized mortality ratio (SMR) was 1.04 (95%CI = 1.03-1.07). Calibration by the Hosmer and Lemeshow method showed X² = 10.47 p = 0.234. The SAPS 3 score that better discriminated survivors and non-survivors was 57, with sensitivity of 75.8% and specificity 86%. Among the patients with SAPS 3 index higher than 57, 73.5% did not survive versus 26.5% who survived (OR= 1.32, 95%CI 1.23-1.42, p < 0.0001).

CONCLUSIONS: The SAPS 3 system is valid for the Brazilian population of surgical patients, being a useful indicator of critical patients and to determine greater care in this group.

Keywords: INTENSIVE CARE: surgical; MORTALITY, Hospital: SAPS 3 predictive score, SAPS 3.

RESUMEN

JUSTIFICATIVA Y OBJETIVOS: El sistema de pronóstico SAPS 3 (Simplified Acute Physiology Score 3), se compone de 20 variables, representadas por una puntuación fisiológica aguda y por una evaluación del estado previo, con el fin de establecer el índice predictivo de mortalidad para los pacientes admitidos en las unidades de cuidados intensivos (UCI). El estudio quiso validar ese sistema y verificar el poder discriminatorio de ese índice en pacientes quirúrgicos de Brasil.

MÉTODO: Estudio prospectivo, realizado en dos UCIs especializadas en pacientes quirúrgicos de dos hospitales diferentes, en el período de un año, donde quedaron excluidos pacientes con edad inferior a los 16 años, que permanecieron un tiempo inferior a 24 horas en la UCI, los readmitidos y los que fueron admitidos para el procedimiento de diálisis. La habilidad predictiva del índice SAPS 3 para diferenciar a los sobrevivientes y a los no sobrevivientes, se constató utilizando la curva ROC y la calibración a través del test Hosmer-Lemeshow goodness-of-fit.

RESULTADOS: Se incluyeron en el estudio 1310 pacientes. Las operaciones gastrointestinales fueron predominantes (34,9%). El menor valor del índice SAPS 3 fue 18 y el mayor 154, un promedio de 48,5 ± 18,1. La mortalidad hospitalaria prevista y real alcanzó los 10,3% y 10,8% respectivamente, la razón de mortalidad estandarizada (SMR) fue 1,04 (IC95% = 1,03-1,07). La calibración por el método Hosmer y Lemeshow mostró X2 = 10,47 p = 0,234. El valor de la puntuación SAPS 3 que desglosó mejor a los sobrevivientes y a los no sobrevivientes fue 57, con una sensibilidad de un 75,8% y una especificidad de un 86%. De los pacientes con el índice SAPS 3 mayor que 57, un 73,5% no sobrevivieron contra un 26,5% de sobrevivientes (OR = 1,32 IC95% 1,23 - 1,42, p < 0,0001).

CONCLUSIONES: El sistema SAPS 3 es valido en la población brasileña de pacientes quirúrgicos, siendo útil para indicar pacientes graves y determinar mayores cuidados en ese grupo.

INTRODUCTION

The number of surgical patients admitted to intensive care units (ICUs) has increased considerable over the last few years¹. A study shows that more than 40 million surgeries are performed every year in the USA and England, and some of them are moderate to high risk procedures. The mortality for high risk patients ranges from 9.7% in the USA to 35.9% in England. The surgical outcome of those patients is influenced by the preoperative physiological status and adequate postoperative care². Thus, data predictive of morbidity and mortality risks are fundamental for this group of patients³.

Therefore, the development, validation, and refinement of prognostic indexes in severely ill patients, such as the Acute Physiology and Chronic Health Evaluation (APACHE)^4,5, Simplified Acute Physiology Score (SAPS)^6,7, and Mortality Prediction Model (MPM)^8,9 are important contributions for intensive care therapy. Prognostic indexes quantify acute and chronic physiologic disruption during admission, estimating the mortality to correct errors and improve the performance of the intensive care unit¹⁰.

The SAPS 3 prognostic system was recently developed in a worldwide cohort^11,12. It is composed of 20 different parameters easily measurable on admission of the patient to the ICU.

Those parameters, divided into three parts, demographic data, reasons for admission to the ICU, and physiologic parameters, represent the degree of disease disruption and assessment of the health status before hospital admission, indicating a premorbid condition.

Each parameter has a score according to the severity of the physiologic disruption. In theory, 16 is the lowest score possible and 217 the highest. Physiologic parameters included are: temperature, systolic blood pressure, heart and respiratory rates, oxygenation, arterial pH, sodium, potassium, creatinine, bilirubin, hematocrit, leukocytes, platelets, and Glasgow coma scale (^{Annex I}).

Several studies^11,12 have validated this system, giving their creators important improvement of this prognostic index. In South America, the index was calibrated with a level of 1.3, i.e., the relationship between observed and predicted mortalities is 1.3. Recently, Soares and Salluh¹³ validated the SAPS 3 in a Brazilian cohort of cancer patients, obtaining good results.

Although this prognostic index has been incorporated in several clinical assay protocols in the intensive care environment^14,15, only one study¹⁶ was developed in surgical ICU patients in Europe, proving to be better than other indexes used before in this population.

Therefore, the objective of the present study was to evaluate the discriminatory power of the SAPS 3 system in a Brazilian population of surgical patients of two tertiary hospitals regarding hospital mortality.

METHODS

This study was undertaken in two intensive care units of two different tertiary Brazilian hospitals in São Paulo, with a total of 24 beds, coordinated primarily by a nurse and a physician. Residents care for patients under the supervision of attending physicians.

This study was approved by the Ethics on Research Commission of both hospitals, and signed consent forms were deem unnecessary, since this is an observational study. Data were gathered by an especially trained nurse.

Consecutive patients admitted to the intensive care units from March 1, 2008 to March 1, 2009 were included in the study. Patients younger than 16 years, who stayed in the ICU for less than 24 hours, readmitted to the unit, and those admitted only for hemodialysis were excluded. Patients were followedup until discharge from the hospital or death.

Data were collected in the first hour after admission to the ICU, using the worst parameter, except for the Glasgow coma scale (the best performance was used). Intubated patients received the best score on verbal response if they did not present neurological deficit; otherwise, they received a score of 1. Ocular and motor responses were evaluated according to the Glasgow coma scale.

Data were divided in: 1) demographic; 2) diagnostic; 3) previous health status; and 4) physiologic parameters (systolic blood pressure, axillary temperature, heart rate, oxygenation, arterial pH, sodium, potassium, creatinine, total bilirubin, hematocrit, leukocytes, platelets, and Glasgow coma scale). The SAPS 3 score was calculated according to those parameters and calibration proposed by the original study for South America¹¹.

Demographic data were expressed as mean ± standard deviation, median (25-75 percentile), or frequency and percentage. To test the discrimination (capacity to classify survivors and non-survivors) sensitivity and specificity tests were used for different SAPS 3 scores, plotting a ROC (Receiver Operating Characteristics) curve, calculating the respective area. The best discriminating value was determined by the maximal sensitivity and specificity. The higher value resulting from this product was the cutoff point.

95% Confidence intervals were computed for true and false positive rates and for the correct classification rate of the outcome. The Hosmer-Lemeshow goodness-of-fit C-statistic test was used to assess concordance between the observed and expected number of survivors and non-survivors in relation to the probability of death (calibration)¹⁷. In this analysis, p > 0.05 indicates good test adjustment. The standardized morbidity ratio (SMR) was calculated by dividing the observed by the predicted mortality rate.

Bicaudal statistical tests were used and the level of significance of 0.05 was used. The Chi-square test was used for categorical parameters. The SPSS 13.0 for Windows, Inc., Chicago, IL, USA, was used to analyze the data.

RESULTS

Out of 1,831 patients admitted during the study period, 1,310 were included and 521 were excluded from the study for several reasons (Table I). Mean patient age was 67.1 ± 15.3 years, and 60.5% were females. Gastrointestinal surgeries predominated (34.9%), followed by orthopedic surgeries (28.2%). The lower SAPS 3 score was 18 and the higher was 154, with a mean of 48.5 ± 18.1 (Table II).

The observed mortality was 10.8% and predicted mortality was 10.3% (SMR = 1.04 95%CI 1.03 to 1.07). The SAPS 3 score of 57 showed better sensitivity (75%) and specificity (86%) for in hospital mortality, with an area under the curve of 0.86 (area = 0.5; p < 0.001, 95%CI: 0.83 to 0.88); therefore, this was the level that better discriminated the mortality in this population of surgical patients (Figure 1).

Patient distribution and their SAPS 3 scores showed that patients with scores equal or lower than 57 had higher rates of survival, but the same was not observed with scores higher than 57. Among patients with SAPS 3 scores higher than 57, 73.5% did not survive versus 26.5% of survivors (OR = 1.32, 95%CI 1.23-1.42, p < 0.0001) (Figure 2).

Patient calibration according to the Hosmer-Lemeshow test showed good adjustment (p = 0.234 and x² = 10.47); therefore, the probability of hospital death increases considerably death increases considerably with higher SAPS 3 scores (Figure 3).

DISCUSSION

Due to the increasing technical-scientific apparatus and qualification of professionals, intensive care units currently concentrate a large proportion of health care resources. Thus, it is evident that concerns are proportional to the growth of those units.

Good management of those resources is fundamental to equate quality of care with the resources destined to those units. Prognostic indexes represent one of the measures more commonly adopted to determine the cost-benefit ratio of those specialized units. Those indexes allow determining the severity of the population cared for at a specific unit, and they can guide the allocation of personnel and equipment; on the other hand, they allow the periodical evaluation of the team performance by comparing, for example, predicted and observed mortality rates. This assessment method is important for the longitudinal follow-up of the performance of a specific unit.

The SAPS 3 score demonstrated good discriminatory power (ability to distinguish survivors and non-survivors). Observed mortality was very close to the predicted mortality, i.e., 10.8% versus 10.3% (SMR = 1.04) respectively, providing good calibration for this sample.

The SAPS 3 score was developed using data from 303 ICUs and 16,784 patients¹¹. However, the SAPS 3 system was not developed to be representative of all types of patients, especially in specific areas or individual types of diseases, since it was developed using a general ICU population. Therefore, external validation is extremely important before applying this score to any type of patient, such as surgical patients. For a long time, surgical patients were evaluated by the ASA physical status, which gives information on the health status before surgery and, therefore, it is extremely limited to predict the worse evolution and hospital outcome.

Sakr Y et al.¹⁶ evaluated 1,851 surgical patients in the ICU, in which the majority were Cardiac Surgery patients. In this study, the discriminatory assessment of the SAPS 3 was better than that of the APACHE II and SAPS II, but with poor calibration (probability to estimate mortality correlating with the observed mortality). The present study, in which surgical patients undergoing non-cardiac interventions were evaluated in two different ICUs, showed better results. Good discriminatory power and good calibration were observed, which valorized this new assessment in a population in which the index had not been tested before emphasizing that a multicenter study can reduce possible bias than studies undertaken in only one center.

The prediction of the SAPS 3 model is based exclusively on data evaluated during the first hour after admission to the ICU^11,12. Half of the original predicted power of the SAPS 3 score derives from information evaluated before admission to the ICU¹¹. Prognostic systems that include measurements after the first 24 hours in the ICU are not valid for ICU screening. Besides, scores obtained more than 24 hours after admission often reflect standard care and not the real clinical status of the patient. This greater advantage of the SAPS 3 can justify its superiority over other prognostic scores. Thus, external validation is necessary to assess the performance of this score in other ICU populations.

In 952 ICU patients, Soares and Salluh observed that SAPS II and SAPS 3 had excellent discrimination in Brazilian ICUs¹³. This Brazilian study demonstrated that the European SAPS 3 overestimated hospital mortality in this population and the data were not surprising, since in the original model of Moreno et al.¹¹, SAPS 3 had the worse calibration for South and Central America. On the other hand, the calibration applied in the Brazilian study showed good mortality discriminatory power, besides showing the closest ratio between observed and predicted mortality. Soares and Salluh¹³ also demonstrated that previous score systems, such as APACHE II, are not satisfactory anymore because they had lower discriminatory power and significant lack of calibration for some populations, such as oncologic patients. It seems that the APACHE II is obsolete nowadays¹⁸. Knaus, the creator of this system, warns researchers to stop using this score to evaluate patient outcome¹³.

Other models, such as SAPS II, proved to be efficient in some populations, especially in the elderly, but with a tendency to overestimate hospital mortality¹⁹.

Due to the easiness to calculate the SAPS 3 index, which does not require more complex analysis, it is suggested that it should be routinely used in ICUs to stratify surgical patients with greater probability of death.

In this context, using the SAPS 3 model in the Brazilian population of surgical patients is relevant, besides considering possible limitations associated with the prediction model.

However, although it was demonstrated that the SAPS 3 system had good discriminating and calibration power, the present study has potential limitations. It can be criticized by the relatively small study population; however, it was designed to have adequate statistical power, and it was undertaken in two intensive care units of different hospitals, which eliminates some biases. Although it is an important matter, the data gathered was not evaluated in the present study. Bias related with the data gathered is limited, since the study was carried on by a trained investigative nurse. Previous studies showed that this condition reduces inter-observer variability²⁰.

We can conclude that the SAPS 3 prediction system proved to be a useful tool to determine which patients will need more care, for the evolution of high risk surgical patients, and it can be used in our country.

REFERENCES

• 1. Almeida SLS, Amendola CP, Horta VM et al. - Hiperlactatemia à admissão na UTI é um determinante de morbimortalidade em intervenções cirúrgicas não cardíacas de alto risco. Rev. Bras. Ter. Intensiva 2006; 18:360-365.
• 2. Bennett-Guerrero E, Hyam JA, Shaefi S et al. - Comparison of PPOSSUM risk-adjusted mortality rates after surgery between patients in the USA and the UK. Br J Surg 2003;90:1593-1598.
• 3. Marshall JC, Cook DJ, Christou NV et al. - Multiple organ dysfunction score: a reliable descriptor of a complex clinical outcome. Crit Care Med 1995;23:1638-1652.
• 4. Knaus WA, Zimmerman JE, Wagner DP et al. - APACHE - acute physiology and chronic health evaluation: a physiologically based classification system. Crit Care Med 1981;9:591-597.
• 5. Knaus WA, Wagner DP, Draper EA et al. - The APACHE III prognostic system: risk prediction of hospital mortality in critically ill hospitalized adults. Chest 1991;100:1619-1636
• 6. Le Gall JR, Loirat P, Alperovitch A et al. - A simplified acute physiology score for ICU patients. Crit Care Med 1984;12:975-977.
• 7. Le Gall JR, Lemeshow S, Saulnier F. - A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA 1993;270:2957-2963
• 8. Baue EA. - Multiple, progressive or sequential systems failure: A syndrome of the 1970s. Arch Surg 1975;110:779-781.
• 9. Lemeshow S, Teres D, Klar J et al. - Mortality probability models (MPM II) based on an international cohort of intensive care unit patients. JAMA 1993;270:2478-2486.
• 10. Cook R, Cook D, Tilley J et al. - Multiple organ dysfunction: baseline and serial component scores. Crit Care Med 2001;29:2046-2050.
• 11. Moreno RP, Metnitz PG, Almeida E et al. - SAPS 3 - From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med 2005;31:1345-1355.
• 12. Metnitz PG, Moreno RP, Almeida E et al. - SAPS 3 - From evaluation of the patient to evaluation of the intensive care unit. Part 1: Objectives, methods and cohort description. Intensive Care Med 2005;31:1336-1344.
• 13. Soares M, Salluh JI. - Validation of the SAPS 3 admission prognostic model in patients with cancer in need of intensive care. Intensive Care Med 2006;32:1839-1844.
• 14. Silva Jr JM, Neves EF, Santana TC et al. - Importância da hipercloremia no intraoperatório. Rev Bras Anestesiol 2009;59:304-313.
• 15. Ledoux D, Canivet JL, Preiser JC et al. - SAPS 3 admission score: an external validation in a general intensive care population. Intensive Care Med 2008;34:1873-1877.
• 16. Sakr Y, Krauss C, Amaral AC et al. - Comparison of the performance of SAPS II, SAPS 3, APACHE II, and their customized prognostic models in a surgical intensive care unit. Br J Anaesth 2008;101:798-803.
• 17. Lemeshow S, Teres D, Pastides H et al. - A method for predicting survival and mortality of ICU patients using objectively derived weights. Crit Care Med 1985;13:519-525.
• 18. Campos EV, Silva JM Jr, Silva MO et al. - Uso do MODS modificado em pacientes sépticos no departamento de emergência para predizer mortalidade. Rev Bras Ter Intensiva 2005;17:74-79.
• 19. Metnitz PG, Valentin A, Vesely H et al. - Prognostic performance and customization of the SAPS II: results of a multicenter Austrian study. Simplified acute physiology score. Intensive Care Med 1999;
• 20. Polderman KH, Jorna EM, Girbes AR. - Inter-observer variability in APACHE II scoring: effect of strict guidelines and training. Intensive Care Med 2001;27:1365-1369.

Annex I

Applicability of the simplified acute physiology score (SAPS 3) in brazilian hospitals

Aplicabilidad de la puntuación fisiológica aguda simplificada (SAPS 3) en hospitales brasileños

Publication Dates

• Publication in this collection
  09 Feb 2010
• Date of issue
  Feb 2010

History

• Accepted
  07 Oct 2009
• Received
  09 July 2009