Probabilistic Model for Prediction of Prognostics in Myocardial Revascularization: Complications in Coronary Surgery

Abstract

Introduction:

Risk scores evaluate pre-operatory risk and present support for clinical decisions, however the performance of these tools in samples different from the original ones remains unclear.

Objectives:

Investigate the external validity of risk scores (STS and Euroscore) in cardiac surgery and the predictive performance of clinical features derived from the sample.

Methods:

Retrospective Cohort study conducted between October, 2010, and April, 2015. We used logistic regression to identify risk factors for hospital morbidity. The sample was divided for cross-validation, with 2/3 of the patients selected for model fitting and 1/3 for prediction testing. The performance of risk scores and clinical features was evaluated through AUROC and calibraton the Hosmer-Lemeshow test (H-L).

Results:

Data was retrieved from 472 patients who underwent coronary cardiac surgery in Hospital Santa Izabel da Santa Casa, BA. Mean age was 62.8 years old and 32.5% of the sample were women. Traditional surgical risk scores did not present significant discriminative performance for this sample. Factors associated with the outcome after adjusting for covariates were: age, previous myocardial revascularization and pre-surgical creatinine levels. The adjusted model presented similar discrimination and calibration values during training (AUROC = 0,72; IC 95% 0,59-0,84; H-L valor p: 0,41) and validation (AUROC = 0,70; IC 95% 0,55 - 0,84; H-L valor p: 0,197).

Conclusion:

Traditional scores may be inaccurate when applied to different environments. New risk scores with good predictive power can be developed using local clinical variables.

Keywords:
Thoracic Surgery/complications; Myocardial Infarction; Myocardial Revascularization; Risk Factors; Forecasting

Resumo

Fundamento:

Escores de risco avaliam risco pré-operatório e permitem definir cuidados durante a intervenção, porém a performance destes instrumentos em amostras distintas das originais é pouco investigada.

Objetivos:

Testar a validade externa de escores de risco cirúrgico cardíaco (STS e Euroscore) e investigar o poder preditivo de características clínicas da amostra.

Métodos:

Estudo de coorte retrospectivo realizado entre outubro de 2010 e abril de 2015. Fatores de risco para morbidade hospitalar foram identificados através de regressão logística. A amostra foi separada para validação cruzada, com 2/3 dos pacientes usados no ajuste do modelo e 1/3 para predições. A performance do STS, do Euroscore e de variáveis clínicas na amostra foi avaliada através de estatística-C (área sob a curva ROC) e calibração através do pelo de Hosmer-Lemeshow (H-L).

Resultados:

72 pacientes foram operados de doença arterial coronariana no Hospital Santa Izabel da Santa Casa, BA. A idade média foi 62,8 anos e 32,5% eram mulheres. Os escores de risco não apresentaram poder discriminativo significativo para amostra. Os fatores identificados como preditores independentes para o desfecho foram: idade, revascularização prévia e creatinina prévia. O modelo ajustado apresentou valores de discriminação e calibração semelhantes no ajuste (AUROC = 0,72; IC 95% 0,59-0,84; H-L valor p: 0,410) e na validação cruzada (AUROC = 0,70; IC 95% 0,55 - 0,84; H-L valor p: 0,197).

Conclusão:

Escores de risco apresentaram desempenho insatisfatório. Variáveis clínicas permitiram a construção de um modelo com boa performance para predição de morbidade nos pacientes operados de revascularização.

Palavras-chave:
Cirurgia Torácica/complicações; Infarto do Miocárdio; Revascularização Miocárdica; Fatores de Risco; Previsões

Introduction

Multivariate probabilistic models have been used in cardiac surgery to estimate the risk of fatal and nonfatal complications.¹ The goal is to evaluate the balance between risks and benefits of procedures for patients, with a better allocation of resources.² There are some risk scores of death and occurrence of complications in patients undergoing myocardial revascularization surgery such as EuroSCORE³ and STS Score.⁴ Assessing the prognosis related to the natural history of a clinical condition, predictive variables are reproducible in different settings.^5,6 On the other hand, when predicting the success or complications of medical procedures, it is possible that predictor variables have a variable value depending on the environment in which the procedure is implemented. This is because differences in the way a treatment is applied can make patients more or less vulnerable to risk determinants. Using a retrospective cohort performed at a tertiary hospital in Salvador, the present study aimed to test the external validity of traditional risk scores for surgical myocardial revascularization, to identify risk markers for myocardial revascularization surgery, and to construct a regional prediction model for complications related to the procedure.

Methods

Study design

Observational study of a retrospective cohort, using a database retrieved from the institution's records, fed with variables recorded between preoperative and patient discharge, used for research, after approval by the hospital's ethics council, under the registry number 24304713.9.3001.5544.

Sample selection criteria

All patients submitted to myocardial revascularization surgery at our service at Santa Izabel Hospital between October 2010 and April 2015 were included in the study sample. Patients with associated surgeries or those performed at other institutions were excluded.

Variables studied

The variables included in the analysis were: Gender, age, weight, height, body mass index, chronic obstructive pulmonary disease (COPD) - use of bronchodilator or corticoid, peripheral arteriopathy - intermittent claudication, carotid artery obstruction greater than 50%; left ventricular dysfunction - moderate 30-50% and significant less than 30%; previous neurological dysfunction - motor dysfunction affecting ambulation or daily function; previous cardiac surgery - previous opening of the pericardium; pre-and postoperative serum creatinine; endocarditis - antibiotic therapy for endocarditis at the time of surgery; unstable angina - use of venous nitrate; recent infarction - less than 90 days; pulmonary hypertension - pulmonary artery systolic pressure greater than 60 mmHg; previous myocardial revascularization; post-infarction ventricular septal defect; diabetes - use of oral hypoglycemic or insulin; smoking; hypertension - antihypertensive use; dyslipidemia - total cholesterol greater than 200 mg / dl, hypertriglyceridemia greater than 150 mg / dl, HDL cholesterol less than 40 mg / dl women and less than 50 mg / dl men; number of coronary lesions greater than 75%; left coronary trunk lesion greater than 50%; preoperative hypoxemia - artery oxygen pressure lower than 60 mmHg, emergency / urgency surgery - need for intervention within 48 hours due to imminent risk of death or unstable clinical-hemodynamic status; hemodynamic instability - ventricular tachycardia, ventricular fibrillation, cardiac arrest, mechanical ventilation, intra-aortic balloon use.

Definition of outcome

The main analysis was performed considering the composite outcome of major morbidity, including: stroke, stroke (central neurological deficit persisting for more than 72 hours); Prolonged intubation (more than 48 hours); Reoperation (tamponade or hemostasis); Mediastinitis (need for surgical reintervention, plus antibiotic therapy with or without positive culture), and death within 30 days after the surgical procedure. The events that made up the outcome were chosen based on models developed and validated from previous studies in cardiovascular surgery.⁷

Statistical analysis

Three logistic regression models were adjusted to test the predictive power of the scores in the sample: EuroSCORE, STS Mortality, and STS Morbidity. Each model was adjusted using the points of the respective score as the only independent variable. A proper model was adjusted following the algorithm proposed by Hosmer and Lemeshow⁸ considering results of bivariate analysis and biological plausibility. The sample was divided into two parts: cohort derivation, intended for bivariate analysis and fit of the models (2/3 of the original sample, randomly selected); Cohort validation to test the obtained model (1/3 of the original sample, randomly selected). After obtaining the coefficients from the sample used for derivation, the model was tested using the validation sample. The Area under ROC Curve (AUROC) and model adequacy statistics are presented for comparison purposes. The analyzes were conducted using the programming language and R development environment.

Results

Between October 2010 and April 2015, information from 472 consecutive patients submitted to myocardial revascularization was included in the database. No patient was excluded because of lack of information. The mean age was 63 ± 8.6 years, 22.5% were women, 37% were diabetics, 32% were smokers, 18% had left ventricular dysfunction and 29% had left coronary artery disease. The incidence of the composite outcome was 37 cases (7.8%), resulting in death 12 (2.5%), stroke 15 (3.2%), cardiac tamponade 4 (0.8%), reoperation for revision of hemostasis 9 (1.9%), mediastinitis 1 (0.2%) and prolonged intubation 21 (4.4%).

Prognostic value of traditional risk scores

The EuroSCORE did not show accuracy for prediction of surgical complications, with C-statistic of 0.507 (95% CI 0.415 - 0.599, p = 0.310). The same was observed with the STS score, with STS Morbidity C-statistic of 0.568 (95% CI 0.473-0.665, p = 0.160) and STS Mortality of 0.550 (95% CI 0.452-0.643, p = 0.860). - Figure 1.

Figure 1
A: STS Mortality; B: STS Morbidity; C: Euroscore.

ROC Curves.

Derivation of the proper model

The variables predictor candidates were selected in 2/3 random of the total sample through univariate analysis, considering statistical significance (p < 0.20). Table 1. Multiple logistic regression analysis according to Hosmer-Lemeshow algorithm: cerebrovascular disease, previous myocardial revascularization, previous creatinine, age and ejection fraction.

Table 1
Bivariate analysis for combined outcome in derivation sub sample

	Total (n = 304)	Combined outcome		Value p
		No (n = 281)	Yes (n = 23)
Age §	62.4 ± 8.9	61.9 ± 8.9	67.6 ± 8.2	0.003
Male gender	228 (75.0)	209 (74.4)	19 (82.6)	0.531
BMI(Kg/m2) * §	26.6 ± 3.6	26.6 ± 3.6	26.8 ± 3.9	0.788
Previous clearance //	83 (68 - 103)	84 (68 - 104)	81 (69 - 91)	0.454
Previous creatinine //	0,9 (0.8 - 1.1)	0.9 (0.8 - 1.1)	1.0 (0.9 - 1.3)	0.031
COPD †	6 (2.0)	6 (2.1)	0 (0.0)	1.000
Peripheral artery disease	13 (4.3)	11 (3.9)	2 (8.7)	0.619
Diabetes (In use of Hypoglycemic agent + Insulin)	109 (35.9)	100 (35.6)	9 (39.1)	0.909
Vascular Brain Disease	2 (0.7)	1 (0.4)	1 (4.3)	0.149
‡ Previous CABG	5 (1.6)	3 (1.1)	2 (8.7)	0.044
Emergency	19 (6.2)	18 (6.4)	1 (4.3)	1.000
Smoker	94 (30.9)	88 (31.3)	6 (26.1)	0.900
Antyhipertensive drug	263 (86.5)	242 (86.1)	21 (91.3)	0.728
Dyslipedemia	263 (86.5)	243 (86.5)	20 (87.0)	1.000
Coronary trunk lesion	87 (28.6)	82 (29.2)	5 (21.7)	0.597
Ejection Fraction
> 50	238 (78.3)	221 (78.6)	17 (73.9)	0.099
50-30	52 (17.1)	47 (16.7)	5 (21.7)
< 30	2 (0.7)	1 (0.4)	1 (4.3)

• ^*
  BMI: body mass index;
• ^†
  COPD: chronic obstructive coronary disease;
• ^‡
  MR: myocardial revascularization. All data presented with n (%), except stated otherwise.
• ^§
  Mean +- Standard deviation.
• ^//
  Average (percentile 25 - percentile 75).

The independent predictors, related to outcome, identified in the final model according to their statistical significance were: previous myocardial revascularization (OR 8.519 95% CI 1.026-59.381 p = 0.029), previous creatinine (OR 2,217 95% CI 0.815-6.018 p = 0.095) and age (OR 1.081 95% CI 1.025-1.145 p = 0.006). Table 2. In the shunt sample (2/3 of the total sample) the ROC curve was 0.72 (95% CI 0.60-0.84, p < 0.001). The resulting logistics model followed the formula below:

Table 2
Coefficients for adjusted models

Suitable model*	Odds Ratio (IC 95%)	Coefficient	Default error	p Value of the coefficient (Statistics Wald)
Previous myocardial revascularization	8.519 (1.026 - 59.381)	2.142	0.979	0.029
Previous creatinine	2.217 (0.815 - 6.018)	0.796	0.476	0.095
Age	1.081 (1.025 - 1.145)	0.077	0.028	0.006
FE ≤ 50	1.251 (0.360 - 3.617)	0.224	0.577	0.698
Vascular Brain Disease	1.398 (0.506 - 397.795)	2.637	1.489	0.077

• ^*
  Constant Value for Model: -8,422; Default error: 1,955; p Value: < 0,001

$\mathit{P}\left(\mathit{y}\right)=\frac{1}{1+{\mathit{e}}^{\left(8.422+0.07*\mathit{Age}+2.14*\mathit{M}{\mathit{R}}_{\mathit{prev}}+0.07*{\mathit{Cr}}_{\mathit{prev}}\right)}}$

AgePrevious

Validation was performed in 168 patients (1/3 of the total sample) randomly selected. The score was discriminated by the area under the ROC curve of 0.70 (95% CI: 0.55-0.84); P = 0.008. Calibration by the Hosmer-Lemeshow test (p = 0.197).

Discussion

The use of multivariate models in the form of scores represents the most accurate mean to predict risk, being superior to that predicted subjectively by clinical impression.⁹ And even showing good accuracy in different populations, especially in the clinical context,^5,6 the results of the present study suggest greater caution regarding the external validity of these scores in the field of cardiac surgery. In addition, a score developed in our local sample demonstrated good accuracy in an independent validation cohort, which may be the predictor in places with different characteristics of the traditional score validator centers. Models not readjusted to the local context may present bias in predicting risk in cardiac surgery and should be systematically compared with regional models.¹⁰ Among the various scores used to predict death and occurrence of complications in cardiac surgery, EuroSCORE³ and STSscore⁴ are the most widespread and validated. Since the end of the nineties, several centers have used EuroSCORE to validate it.^11-13 In the United States, it was more accurate compared to other predictor models when validated in the database with more than 500.000 patients of the Society of Thorac Surgery.¹⁴ However, a systematic review evaluating the performance of the EuroSCORE concluded that the model overestimates surgical risk based on five studies of different nationalities.¹⁵ The present study corroborates the findings, finding unsatisfactory results for the predictive capacity of the evaluated scores, in contrast to a good performance of the locally adjusted model. The development of local risk scores presents growth, finding difficulties in implementation, but presenting progressive improvements and contributing to the identification of risk factors.^16-18 A recent body of results shows a better performance of models fitted with local data in relation to EuroSCORE, Parsonet Score and Ontario Risk Score.¹⁹ Other studies in cardiovascular surgery suggest that most information on prognosis is contained in a few clinical variables, showing that simple models are as effective as complex models.²⁰ Although it is better suited than traditional scores, the score derived from the sample in this study is not intended for use in other services. Performing in only one center limits the external validity, where characteristics of the patients and the care body of the institution may vary.

Conclusion

Risk scores in cardiovascular surgery should be revalidated locally and the development of simple local predictive models may present better results in a delimited environment.

• Sources of Funding
  There were no external funding sources for this study.
• Study Association
  This article is part of the thesis of master submitted by Valcellos José da Cruz Viana, from Escola Bahiana de Medicina e Saúde Pública.

• ¹
  Asimakopoulos G, Al-Ruzzeh S, Ambler G, Omar RZ, Punjabi P. Amrani M, al. An evaluation of existing risk stratification models as a tool for comparison of surgical performances for coronary artery bypass grafting between institutions. Eur J Cardiothorac Surg. 2003;23(6):935-41.
• ²
  Pikanem O, Niskanen M, Rehnberg S, Hippelainen M, Hynyem M. Intra-institutional prediction of outcome after cardiac surgery: comparison between a locally derived model and the EuroSCORE. Eur J Cardiothorac Surg. 2000;18(6):703-10
• ³
  Nashef SA, Roques F, Michel P, Gauducheau E, Lemeshow S, Salamon R. European system for cardiac operative risk evaluation(EuroSCORE). Eur J Cardiothorac Surg. 1999;16(1):9-13.
• ⁴
  Shahian MD, O'Brien SM, Filardo G, Ferraris VA, Haan CK, Rich JB, et al; Society of Thoracic Surgeons Quality Measurement Task Force. The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 1 - coronary artery bypass grafting surgery. Ann Thorac Surg. 2009;88(1 Suppl):S2-22.
• ⁵
  Wilson PW, D'Agostino RB, Levy D, Belanger AM, Silbershatz H, Kanel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97(18):1837-47.
• ⁶
  Granger CB, Goldberg RJ, Dabbous O, Pieper KS, Eagle KA, Cannon CP, et al Global Registry of Acute Coronary Events Investigators ;. Predictors of hospital mortality in the global registry of acute coronary events. Arch Intern Med. 2003,163(19):2345-53.
• ⁷
  Smith LR, Harrel FE, Rankin JS, Califf RM, Pryor DB, Muhlbaier LH, et al. Determinants of early versus late cardiac death in patients undergoing coronary artery bypass graft surgery. Circulation. 1991;84(5 Suppl):III245-53.
• ⁸
  Hosmer DW, Lemeshow S. Applied logistic regression. 2nd ed. New York: John Wiley and Sons; 2000.
• ⁹
  Yan AT, Yan RT, Huynh T, Casanova A, Raimondo FE, Fitchett DH, et al; Canadian Acute Coronary Syndrome Registry 2 Investigators. Understanding physicians' risk stratification of acute coronary syndromes: insights from the Canadian ACS 2 Registry. Arch Intern Med. 2009;169(4):372-8.
• ¹⁰
  Ivanov J, Tu JV, Naylor CD. Ready-made, recalibrated, or Remodeled? Issues in the use of risk indexes for assessing mortality after coronary artery bypass graft surgery. Circulation. 1999;99(16):2098-104.
• ¹¹
  Carvalho MR, Souza e Silva NA, Klein CH, Oliveira GM. Application of the EurSCORE in coronary artery bypass in public hospitals in Rio de Janeiro, Brazil. Rev Bras Cir Cardiovasc. 2010;25(2):209-17.
• ¹²
  Yap CH, Reid C, Yii M, Rowland MA, Mohajeri M, Skillington PD, et al. Validation of the EuroSCORE model in Australia. Eur J Cardiothorac Surg. 2006;29(4):441-6.
• ¹³
  Zheng Z, Li Y, Zhang S, Hu S; Chinese CABG registry Study. The Chinese coronary artery bypass grafting registry study: how well does the EuroSCORE predict operative risk for Chinese population? Eur J Cardiothorac Surg. 2009;35(1):54-8.
• ¹⁴
  Geissler HJ, Holzl P, Marohl S, Kuhin-Regnier F, Mehlhorn U, Sudkanp M, et al. Risk stratification surgery: comparison of six score systems. Eur J Cardiothorac Surg. 2000;17(4):400-6.
• ¹⁵
  Gogbashian A, Sedrakyan A, Treasure T. EurSCORE: a systematic review of international performance. Eur J Cardiothorac Surg. 2004;25(5):695-700. Erratum in: Eur J Cardiothorac Surg. 2004;26(2):463.
• ¹⁶
  Gomes RV, Tura B, Mendonça Filho HT, Almeida Campos LA, Rouge A, Matos Nogueira PM, et al. A first postoperative day predictive score of mortality for cardiac surgery. Ann Thorac Cardiovasc Surg. 2007;13(3):159-64.
• ¹⁷
  Cadore MP, Guaragna JC, Anacker JF, Albuquerque LC, Bodanese LC, Piccoli Jda C, et al. A score proposal to evaluate surgical risk in patients submitted to myocardial revascularization surgery. Rev Bras Cir Cardiovasc. 2010;25(4):447-56. Erratum in: Rev Bras Cir Cardiovasc. 2011;26(1):144.
• ¹⁸
  Mejía OA, Lisboa LA, Puig LB, Moreira LF, Dallan LA, Pomerantzeff PM, et al. InsCor: a simple and accurate method for risk assessment in heart surgery. Arq Bras Cardiol. 2013;100(3):246-54.
• ¹⁹
  Antunes PE. Eugenio L. Ferrão de Oliveira J. Antunes MJ. Mortality risk prediction in coronary surgery: a locally developed model outperforms external risk models. Interact Cardiovasc Thorac Surg. 2007;6(4):437-41.
• ²⁰
  Jones RH, Hannan EL, Hammermeister KE, Delong ER, O&'Connor GT, Luepker RV, et al. Identification of preoperative variables needed for risk adjustment of short-term mortality after coronary artery bypass graft surgery. The Working Group Panel on the Cooperative CABG Database Project. J Am Coll Cardiol. 1996;28(6):1478-87.

Publication Dates

• Publication in this collection
  Jul-Aug 2017

History

• Received
  14 Sept 2016
• Reviewed
  11 Oct 2016
• Accepted
  03 Mar 2017