ABSTRACT
Objective:
The European Surgical Outcomes Study described mortality following in-patient surgery. Several factors were identified that were able to predict poor outcomes in a multivariate analysis. These included age, procedure urgency, severity and type and the American Association of Anaesthesia score. This study describes in greater detail the relationship between the American Association of Anaesthesia score and postoperative mortality.
Methods:
Patients in this 7-day cohort study were enrolled in April 2011. Consecutive patients aged 16 years and older undergoing inpatient non-cardiac surgery with a recorded American Association of Anaesthesia score in 498 hospitals across 28 European nations were included and followed up for a maximum of 60 days. The primary endpoint was in-hospital mortality. Decision tree analysis with the CHAID (SPSS) system was used to delineate nodes associated with mortality.
Results:
The study enrolled 46,539 patients. Due to missing values, 873 patients were excluded, resulting in the analysis of 45,666 patients. Increasing American Association of Anaesthesia scores were associated with increased admission rates to intensive care and higher mortality rates. Despite a progressive relationship with mortality, discrimination was poor, with an area under the ROC curve of 0.658 (95% CI 0.642 - 0.6775). Using regression trees (CHAID), we identified four discrete American Association of Anaesthesia nodes associated with mortality, with American Association of Anaesthesia 1 and American Association of Anaesthesia 2 compressed into the same node.
Conclusion:
The American Association of Anaesthesia score can be used to determine higher risk groups of surgical patients, but clinicians cannot use the score to discriminate between grades 1 and 2. Overall, the discriminatory power of the model was less than acceptable for widespread use.
Anesthesiology; Reproducibility of results; Mortality; Postoperative period
RESUMO
Objetivo:
O European Surgical Outcomes Study foi um estudo que descreveu a mortalidade após a cirurgia de pacientes internados. Em uma análise multivariada, foram identificados diversos fatores capazes de prever maus resultados, os quais incluem idade, urgência do procedimento, gravidade e porte, assim como o escore da American Association of Anaesthesia. Este estudo descreveu, com mais detalhes, o relacionamento entre o escore da American Association of Anaesthesia e a mortalidade pós-operatória.
Métodos:
Os pacientes neste estudo de coorte com duração de sete dias foram inscritos em abril de 2011. Foram incluídos e seguidos, por no máximo 60 dias, pacientes consecutivos com idade de 16 anos ou mais, internados e submetidos à cirurgia não cardíaca e com registro do escore da American Association of Anaesthesia em 498 hospitais, localizados em 28 países europeus. O parâmetro primário foi mortalidade hospitalar. Foi utilizada uma árvore decisória, com base no sistema CHAID (SPSS), para delinear os nós associados à mortalidade.
Resultados:
O estudo inscreveu um total de 46.539 pacientes. Em função de valores faltantes, foram excluídos 873 pacientes, resultando na análise 45.666. Aumentos no escore da American Association of Anaesthesia se associaram com o acréscimo das taxas de admissão à terapia intensiva e de mortalidade. Apesar do relacionamento progressivo com mortalidade, a discriminação foi fraca, com uma área sob a curva ROC de 0,658 (IC 95% 0,642 - 0,6775). Com o uso das árvores de regressão (CHAID), foram identificadas quatro discretas associações dos nós da American Association of Anaesthesia com mortalidade, estando o escore American Association of Anaesthesia 1 e o escore da American Association of Anaesthesia 2 comprimidos em um mesmo nó.
Conclusão:
O escore da American Association of Anaesthesia pode ser utilizado para determinar grupos de pacientes cirúrgicos de alto risco, porém os médicos não podem utilizá-lo para realizar a discriminação entre os graus 1 e 2. Em geral, o poder discriminatório do modelo foi menos do que aceitável para uso disseminado.
Anestesiologia; Reprodutibilidade de resultados; Mortalidade; Período pós-operatório
INTRODUCTION
In 1940, the American Society of Anaesthesiology (ASA) asked a committee of three physicians to develop a system for the collection and tabulation of statistical data for anesthesia that could be applicable under any circumstances. The ASA score(11 Saklad M. Grading of patients for surgical procedures. Anesthesiology. 1941;2(3):281-4.) that originated from this project has since developed into one of the most commonly used clinical scoring systems in the world. The score was originally designed to focus only on the preoperative comorbid state of the patient and not the surgical procedure or any other factors that could influence the outcome of surgery.
The score was originally described by four categories(22 Little JP. Consistency of ASA grading. Anaesthesia. 1995;50(7):658-9.) that ranged from a healthy patient (class 1) to one with an extreme systemic disorder that is an imminent threat to life (class 4). Subsequently, two further classes were added, classes 5 and 6, which were subsequently collapsed so that they could be applied to moribund patients who were not expected to survive 24 hours, with or without surgery. A sixth class has since been described to be used exclusively for declared brain-dead organ donors.
Despite its apparent simplicity, this score is conceptually complex because it combines elements from the patient status before surgery (in classes 1 to 3) together with elements from the subjective opinion of the anesthesiologist (classes 4 and 5). Some authors add a sixth class for patients who are anesthetized just for organ retrieval (Table S1, no electronic supplementary material). The ASA score is not the only score that has followed this approach, but the relative merits of a purely objective score based solely on patient characteristics versus the incorporation of the subjective opinion of physicians remains controversial.(33 Kruse JA, Thill-Baharozian MC, Carlson RW. Comparison of clinical assessment with APACHE II for predicting mortality risk in patients admitted to a medical intensive care unit. JAMA. 1988;260(12):1739-42.) For these reasons, we decided to analyze the performance of the ASA score after almost 60 years of use in clinical practice in a large multicenter, multinational database.
METHODS
The European Surgical Outcomes Study (EuSOS) database(44 Pearse RM, Moreno RP, Bauer P, Pelosi P, Metnitz P, Spies C, Vallet B, Vincent JL, Hoeft A, Rhodes A; European Surgical Outcomes Study (EuSOS) group for the Trials groups of the European Society of Intensive Care Medicine and the European Society of Anaesthesiology. Mortality after surgery in Europe: a 7 day cohort study. Lancet. 2012;380(9847):1059-65.) was used in this study. The primary objective of EuSOS was to describe mortality rates and patterns of critical care resource use for patients undergoing non-cardiac surgery across several European nations. The design of the study and the results of the EuSOS have been described elsewhere.(44 Pearse RM, Moreno RP, Bauer P, Pelosi P, Metnitz P, Spies C, Vallet B, Vincent JL, Hoeft A, Rhodes A; European Surgical Outcomes Study (EuSOS) group for the Trials groups of the European Society of Intensive Care Medicine and the European Society of Anaesthesiology. Mortality after surgery in Europe: a 7 day cohort study. Lancet. 2012;380(9847):1059-65.) In brief, the European cohort study was performed between 0900 (local time) on April 4, 2011 and 0859 on April 11, 2011. All adult patients (older than 16 years) admitted to participating centers for elective or non-elective inpatient surgery commencing during the 7-day cohort period were eligible for inclusion in the study. Patients undergoing planned day case surgery, cardiac surgery, neurosurgery, or radiological or obstetric procedures were excluded. Participating hospitals represented a voluntary convenience sample that was identified based on the membership of the European Society of Intensive Care Medicine (ESICM) and the European Society of Anaesthesiology (ASA) and by the direct approach from national study coordinators. Ethics requirements differed by country. The primary study was approved in the coordinating center (Barts and The London School of Medicine and Dentistry, Queen Mary University of London - London, United Kingdom).
Cohort description
For this sub-study, all of the patients within the EuSOS database were included. Patients lacking a description of their ASA status were excluded from the study (92 patients). Other exclusion criteria derived from the sensitivity analysis of the EuSOS score and defined to exclude the effects of very small centers or extreme deviations regarding the reported mortality were as follows: (1) any site that enrolled less than 10 patients during the study week, (2) any site with a hospital mortality rate either above the 95th centile or below the 5th centile, and (3) any patient with missing data for hospital mortality.
Outcomes
The primary outcome used in this study was survival at the time of hospital discharge. Patients were followed until hospital discharge, death or 60 days after hospital admission.
Statistical analysis
The data were analyzed using SPSS, version 19.0 (SPSS Inc, Chicago, USA). Categorical variables are presented as numbers (%), and continuous variables are presented as means (SD) when normally distributed or medians (IQR) when not normally distributed. the Chi squared and Fisher’s exact tests were to compare categorical variables, and the t test or ANOVA was used to compare continuous variables. Significance was set at p < 0.05. Because the rate of missing values was very low (< 0.05%), no imputation procedures were performed, and all of the variables were analyzed case wise. Discrimination of the score was assessed by the area under the receiver operating characteristics curve (aROC) and computed as suggested by Hanley and McNeil.(55 Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29-36.) To further characterize the effect of the ASA score on the vital status at the time of hospital discharge, we used regression trees with the CHAID procedure in SPSS v 19 (SPSS Inc, Chicago, USA) and Kaplan-Meier curves with vital status at hospital discharge as the dependent variable and patient censoring at hospital discharge.
A logistic multi-level regression analysis was used to determine whether the effect of the ASA score on hospital mortality was affected by other variables. To minimize the correlation with variables that were already included in the ASA score, comorbid diseases that were present at hospital admission were not used in the model because they are included in the definitions of the first 3 classes of the ASA score. The first step was to identify factors that were independently related to hospital mortality in the multivariate analysis. The following factors were entered into the model based on their relationship to the outcome in the univariate analysis: age, gender, urgency of surgery (reference urgent), laparoscopic surgery, seniority of the surgeon, seniority of the anesthesiologist, grade of surgery and surgical procedure category. Due to the multiplicity of tests performed and to avoid spurious associations and over-fitting, only p values less than 0.01 were considered significant and included in the model to allow for a more robust and consistent result. All of the entered factors were biologically plausible and had a sound scientific rationale and a low rate of missing data (see main paper). The results of the univariate analysis model are reported as odds ratios (OR) with 95% confidence intervals (CI).
RESULTS
A total of 45,666 patients from 366 centers in 28 European countries were included in the study. The basic characteristics of the analyzed patients are presented in table 1. Among the patients, 11,431 were classified as ASA I (25.0%), 21,193 as ASA II (46.4%), 11,411 as ASA III (3.4%), 1,543 as ASA IV and 88 as ASA 5 (0.2%).
As expected, the majority of the physiologic derangements were positively and significantly correlated to the ASA score. The ASA score presented a very good relationship with survival at the time of hospital discharge, as presented in figure 1A and 1B (Figure 1A: raw numbers; Figure 1B: percentages). It should be noted, however, that given the very large differences in the numbers of patients in each class, with most patients concentrated in classes I and II, the clinical utility of this relationship is low.
American Association of Anaesthesia and vital status at hospital discharge (as numbers on the top, and as % of patients by class on the bottom). Striped bars represent survival at hospital discharge, and black bars are death before hospital discharge.
Complete data for the sensitivity, false positive rate, specificity (true negative rate), predictive value for dying in the hospital, predictive value for surviving and overall correct classification are described in detail in table 2. Discrimination for the ASA score was poor, with an aROC of 0.658 ± 0.008 (95% confidence interval of 0.642 to 0.675) (Figure 2).
Sensitivity, false positive rate, specificity (true negative rate), predictive value for dying in the hospital, predictive value for surviving and overall correct classification
Area under the receiver operating characteristic (ROC) curve for the 5 categories of the American Association of Anaesthesia score. The aROC was 0.656 with a standard error of 0.008 (95% confidence interval of 0.642 - 0.675). The asymptotic significance of the curve was < 0.001.
In the univariate analysis, several variables were significantly associated with the ASA score (Table S2 in the electronic supplementary material). In the multivariate analysis, only the ASA score, age, surgical procedure category, grade of surgery, urgency of surgery and country remained significant (Table 3). The adjusted odds ratios for the ASA classes were 0.007 [0.005 - 0.011], 0.794 [0.659 - 0.958], 1.416 [1.151 - 1741], 5.267 [4.123 - 6.727], 18.393 [11.056 - 30.600] for classes I to V, respectively.
Multivariable analysis of outcome determinants (American Society of Anesthesiologists and its variables purposefully excluded)
When the regression trees (CHAID) were applied to this cohort, the results demonstrated that ASA classes I and II should be collapsed together (Figure 3). By merging ASA categories I and II, the percentage of correct classifications increased to 97%, and the score predicted 0.20% of the survivors and 99.8% of the deaths.
Regression trees (CHAID) for the different classes of the American Association of Anaesthesia score.
These results were confirmed by the Kaplan-Meier curves, again using survival at hospital discharge as the dependent variable and patient censoring at hospital discharge, although the results must be considered with caution given the large number of censored patients. The survival function (Figure S1 A in the electronic supplementary material), log survival function (Figure S1 B in the electronic supplementary material), and hazard function (Figure S1 C in the electronic supplementary material), all of which utilized vital status at hospital discharge as the outcome variable, are presented below.
DISCUSSION
The principal finding of this analysis was that ASA was a poor predictor of survival until hospital discharge in a large population of patients undergoing in-patient non-cardiac surgery. However, by collapsing ASA categories I and II, the performance of the score improved in low risk patients, for whom the performance of the score was less accurate.
Almost 60 years after its original description, and despite the fact that it is one of the most used models to assess risk in patients submitted to surgery, the overall performance of the ASA score as a tool to predict in-hospital deaths following surgery was found to be poor. This result is in contrast to those obtained for other, more modern, severity scores that are designed to forecast vital status at hospital discharge after admission to the intensive care unit (ICU), such as the APACHE II,(66 Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818-29.) the SAPS II,(77 Le Gall JR, Lemeshow S, Saulnier F. A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA. 1993;270(24):2957-63. Erratum in: JAMA 1994;271(17):1321.) and the SAPS 3 systems.(88 Moreno RP, Metnitz PG, Almeida E, Jordan B, Bauer P, Campos RA, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR, SAPS 3 Investigators. SAPS 3-From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med. 2005;31(10):1345-55. Erratum in: Intensive Care Med. 2006;32(5):796.) In this case, a direct comparison between the ASA scores and these other scores is not possible because the latter scores have been ascertained only in patients who have been admitted to the ICU (thus, in principle, more severely affected patients) and not in all of the enrolled patients.
A surprising number of deaths were classified as ASA I. This result has a number of
possible explanations, including the following: incorrect scoring of the patients, or
a mortality rate that is much greater than that anticipated in this class or
classification rules that are not easy to apply. Table S1 shows that the
patients were classified with significant comorbidities, e.g., metastatic cancer was
classified as ASA I. We do not believe that ongoing attempts to subdivide ASA
III(99 Schwam SJ, Gold MI, Craythorne UW. The ASA Physical Status
Classification: a revision. Anesthesiology. 1982;57(3):A439.) or to add
additional categories(1010 Higashizawa T, Koga Y. Modified ASA physical status (7 grades) may be
more practical in recent use for preoperative risk assessment. J Anesthesiol
[Internet]. 2007;15(1). [cited 2015 Apr 22]. Available from:
https://ispub.com/IJA/15/1/7401
https://ispub.com/IJA/15/1/7401...
) will improve the performance of the score, as very clearly
demonstrated by the regression trees. At a time when economic constraints and the
pursuit of quality of care and maximization of patient safety are a priority, care
should be taken when using this instrument to detect such cases.
This study has many strengths but also some limitations. First, a very large population of patients who were submitted to non-cardiac surgery in 28 countries in Europe were studied, using real life data registered by professionals in a heterogeneous sample, and a score with questionable reliability.(1111 Aronson WL, McAuliffe MS, Miller K. Variability in the American Society of Anesthesiologists Physical Status Classification Scale. AANA J. 2003;71(4):265-74.) However, by design, we did not perform a serious intra and inter-observer reliability analysis, thus hampering the significance of the results.
However, the simplicity of the ASA system - which was potentially one of the keys to its success - may be less relevant to modern practice. The poor discrimination, which indicates the absence of forecasting a precise mortality rate for patient populations (thus making it impossible to assess its calibration) during an important historical period, had a crucial impact on the development of modern methods.
In a specialty like anesthesia, in which the mortality rates have been reduced by a log factor from 1 anesthesia-related death in 5000 procedures in the 1980s to less that 1 in 250.000 in 1998,(1212 Gaba DM. Anaesthesiology as a model for patient safety in health care. BMJ. 2000;320(7237):785-8.) it is time to move forward.
CONCLUSION
In conclusion, in the present study, the American Association of Anaesthesia score was able to determine higher risk groups of surgical patients, but clinicians cannot use this score to discriminate between lower risk groups (grades 1 and 2). Overall, the discriminatory power of the model was less than acceptable to recommend its widespread use.
-
Funding: The study was funded by the European Society of Intensive Care Medicine and by the European Society of Anaesthesia.
-
Responsible editor: Jorge Ibrain Figueira Salluh
-
*
Available in the electronic supplementary material
REFERÊNCIAS
-
1Saklad M. Grading of patients for surgical procedures. Anesthesiology. 1941;2(3):281-4.
-
2Little JP. Consistency of ASA grading. Anaesthesia. 1995;50(7):658-9.
-
3Kruse JA, Thill-Baharozian MC, Carlson RW. Comparison of clinical assessment with APACHE II for predicting mortality risk in patients admitted to a medical intensive care unit. JAMA. 1988;260(12):1739-42.
-
4Pearse RM, Moreno RP, Bauer P, Pelosi P, Metnitz P, Spies C, Vallet B, Vincent JL, Hoeft A, Rhodes A; European Surgical Outcomes Study (EuSOS) group for the Trials groups of the European Society of Intensive Care Medicine and the European Society of Anaesthesiology. Mortality after surgery in Europe: a 7 day cohort study. Lancet. 2012;380(9847):1059-65.
-
5Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29-36.
-
6Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818-29.
-
7Le Gall JR, Lemeshow S, Saulnier F. A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA. 1993;270(24):2957-63. Erratum in: JAMA 1994;271(17):1321.
-
8Moreno RP, Metnitz PG, Almeida E, Jordan B, Bauer P, Campos RA, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR, SAPS 3 Investigators. SAPS 3-From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med. 2005;31(10):1345-55. Erratum in: Intensive Care Med. 2006;32(5):796.
-
9Schwam SJ, Gold MI, Craythorne UW. The ASA Physical Status Classification: a revision. Anesthesiology. 1982;57(3):A439.
-
10Higashizawa T, Koga Y. Modified ASA physical status (7 grades) may be more practical in recent use for preoperative risk assessment. J Anesthesiol [Internet]. 2007;15(1). [cited 2015 Apr 22]. Available from: https://ispub.com/IJA/15/1/7401
» https://ispub.com/IJA/15/1/7401 -
11Aronson WL, McAuliffe MS, Miller K. Variability in the American Society of Anesthesiologists Physical Status Classification Scale. AANA J. 2003;71(4):265-74.
-
12Gaba DM. Anaesthesiology as a model for patient safety in health care. BMJ. 2000;320(7237):785-8.
Data availability
Publication Dates
-
Publication in this collection
Apr-Jun 2015
History
-
Received
20 Jan 2015 -
Accepted
16 Apr 2015
