Validation of the Grace Risk Score to Predict In-Hospital and 6-Month Post-Discharge Mortality in Patients with Acute Coronary Syndrome

Abstract

Background

The wide range of clinical presentations of acute coronary syndrome (ACS) makes it indispensible to use tools for risk stratification and for appropriate risks management; thus, the use of prognosis scores is recommended in the immediat clinical decision-making.

Objective

To validate the Global Registry of Acute Coronary Events (GRACE) score as a predictor of in-hospital and 6-month post-discharge mortality in a population diagnosed with ACS.

Methods

This is a prospective cohort study of consecutive patients diagnosed with ACS between May and December 2018. GRACE scores were calculated, as well as their predictive value for in-hospital and 6-month post-discharge mortality. The validity of the model was assessed by two techniques: discriminative power using the area under the receiver operating characteristic curve (AUC) and goodness-of-fit, using the Hosmer-Lemeshow (HL) test, at the 5% level of significance.

Results

A total of 160 patients were included, mean age 64 (±10.9) years; of which 60% were men. The risk model showed to have satisfactory ability to predict both in-hospital mortality, with an area under the curve (AUC) of 0.76 (95% confidence interval [CI], 0.57-0.95; p = 0.014), and 6-month post-discharge mortality, with AUC of 0.78 (95%CI, 0.62-0.94), p = 0.002. The HL test indicated good-fit for both models of the GRACE score.

Conclusion

In this study, the GRACE risk score for predicting mortality was appropriately validated in patients with ACS, with good discriminative power and goodness-of-fit. The results suggest that the GRACE score is appropriate for clinical use in our setting.

Acute Coronary Syndrome; Mortality; Prognosis

Introduction

Cardiovascular disease is the leading cause of mortality in Brazil and worldwide, and ischemic heart disease accounts for a large portion of this concerning scenario.¹ Among its forms of presentation, acute coronary syndrome (ACS) has a wide range of severity.² However, the use of validated mathematical models of clinical prediction is essential and recommended in national and international guidelines for the management of patients with ACS.³,⁴

With this stratification, high-risk patients may receive more aggressive antiplatelet and antithrombotic therapy and early invasive intervention, whereas lower-risk patients may receive less aggressive treatments.³,⁵

Based on the Global Registry of Acute Coronary Events (GRACE) report, the GRACE score was designed with 8 variables analyzed on patient's admission,⁶ 5 semi-quantitative ones (age range, heart rate, systolic blood pressure, plasma creatinine, and Killip class) and 3 dichotomic ones: cardiac arrest at admission, ST-segment deviation, and elevation of cardiac markers. The final score can range from 0 to 372.⁶

Therefore, the present study aimed to assess the value of the GRACE score as a predictor of in-hospital prognosis and 6-month post-discharge prognosis in patients with ACS in our setting.

Methods

Sample Selection

All individuals admitted with a diagnosis of ACS at Hospital de Clínicas de Passo Fundo (HCPF), Brazil, from May to December 2018 were selected. Demographic, clinical, and angiographic variables were prospectively collected. The patients were treated according to the criteria of the attending physician, without intervention from researchers. The study was approved by the Research Ethics Committee of Faculdade IMED, in compliance with the Resolution 466/2012 of the National Health Council.

Inclusion criteria were: age 18 years or older, symptoms suggestive of acute coronary ischemia on admission, and presence of at least one of the following characteristics: changes suggestive of ACS on electrocardiogram (ECG), elevation of serum biochemical markers of myocardial necrosis, and/or documented previous coronary artery disease with angiography showing coronary obstruction ≥ 50%. Patients whose ACS was triggered by secondary factors, such as trauma or surgery, were excluded. No patient refused to participate in the study, and all of them provided informed consent to participate in the research. Hospitalization outcomes were obtained by phone interview or outpatient visit 6 months after hospital discharge.

GRACE Score

The GRACE score was published in 2004 based on the GRACE registry, which was designed to reflect the full of patients with ACS. Data were obtained in 14 countries (Europe, North and South America, Australia, and New Zealand), including 94 hospitals, of which 6 were Brazilian, with a total population of 17142 patients. The aim of the score was to develop a tool to estimate of probability risk of 6-month mortality or myocardial infarction so as to facilitate the stratification of patients with ACS.

Based on the GRACE score, patients were classified into low (< 1%), intermediate (1 to 3%), and high risk (> 3%) for in-hospital mortality. For the 6-month post-discharge prognosis, patients were divided into those with low (< 3%), intermediate (3 to 8%), and high mortality risk (> 8%).³,⁵

Clinical outcomes

The primary outcome was defined as in-hospital and 6-month post-discharge mortality. With regard to secondary outcomes, the accuracy of the GRACE score was assessed in the different presentations of ACS.

Statistical analysis

Categorical data were presented as frequencies (percentages), and continuous variables were presented as mean and standard deviation (SD) or median and interquartile range (IQR). The Kolmogorov-Smirnov test was used to verify of distribution. The level of significance was set at p < 0.05 for all analyses. The discriminative power of the score with regard to in-hospital and 6-month post-discharge was assessed using the C statistics. The area under the receiver operating characteristic curve (AUC) represented the accuracy of the GRACE score in distinguishing survivors from non-survivors. Along with this analysis, cutoff values were identified to define the best prognostic sensitivity and specificity, with their 95% confidence intervals (CI). Goodness-of-fit for the scores was assessed by the Hosmer-Lemeshow test and by the dispersion graph between predicted mortality at each risk decile and the observed mortality. The analysis was performed using the SPSS 20.0, Minitab 16 and MedCalc, version 19.1 software.

Results

Sample characteristics

The sample consisted of 160 patients. Two patients (1.25%) were lost to follow-up, due to absence of outpatient follow-up and telephone contact failure. Demographic and clinical characteristics with regard to the prevalence of cardiovascular risk factors and initial presentation are presented in Table 1.

Table 1
– Sample characteristics

Variables	Distribution
Sample	160
Age (years)	64 (10.9)
Male gender	96 (60%)
BMI (kg/m2)	28.2 (5.1)
Treatment on SUS	142 (88.8%)
Diabetes	112 (70%)
Hypertension	121 (75.6%)
Sedentary	66 (41.25%)
Smoking	50 (31.25%)
Systolic blood pressure (mm Hg)	138 (29.1)
Heart rate (bpm)	78.7 (20.7)
Killip class > 1	16 (10%)
ST-segment depression	43 (26.9%)
Creatinine (mg/L)	1.07 (0.94-1.25)
Positive HS troponin	100 (62.5%)
ACS UA NSTEMI STEMI	60 (37.5%) 58 (36.25) 42 (26.25%)
GRACE score - Intra-hospital	111.5 (94.3-139.5)
GRACE score - 6 months after discharge	95.5 (80.5-117)

• Creatinine and GRACE scores were expressed in median and interquartile range. The remaining (continuous) variables were expressed as mean (SD).
• BMI: body mass index; SUS: Brazilian Unified Health System (Sistema Único de Saúde); HS: high sensitivity; ACS: acute coronary syndrome; UA: unstable angina; NSTEMI: non-ST segment elevation myocardial infarction; STEMI: ST segment elevation myocardial infarction; and GRACE: Global Registry of Acute Coronary Events.

Primary outcome

In-hospital mortality was 5.1% (8 deaths). Six hospital deaths were caused by cardiogenic shock, and 2 by infectious complications with septic shock. The Hosmer-Lemeshow test for the in-hospital GRACE score yielded a c² of 7.14 (p = 0.522) and an AUC of 0.76 (95% confidence interval (CI), 0.57-0.95). Six-month post-discharge mortality was 7% (11 deaths). Among the patients who died after hospital discharge, 2 had sudden death, and 1 had a new episode of MI. The results for the 6-month post-discharge GRACE score showed c² of 4.53 (p = 0.81) and AUC of 0.78 (95%CI, 0.62-0.94). Therefore, both predictions exhibited a good-fit (Figure 1).

Figure 1
– ROC curves for in-hospital and 6-month post-discharge GRACE scores. AUC: Area under the receiver operating characteristic curve (C statistics).

According to the ROC curve, the best cutoff value for the in-hospital GRACE score was 179, with sensitivity of 50% and specificity of 98%. Conversely, the best cutoff value for the 6-month post-discharge GRACE score was 119.5, with sensitivity of 72.7% and specificity of 81.6% (Table 2).

Table 2
– Predictive accuracy of in-hospital and 6-month post-discharge GRACE scores

	Area (AUC)	Specificity	Sensitivity	p-value
In-hospital GRACE score	0.76 (0.57-0.95)	98%	50%	0.014
6-month post-discharge GRACE score	0.78 (0.62-0.94)	81.6%	72.7%	0.002

• AUC: area under the receiver operating characteristic curve.

Secondary outcome

The accuracy of the GRACE score in the different forms presentations of ACS was also tested. There was no outcome UA to be analyzed.

With regard to the non-ST segment elevation myocardial infarction (NSTEMI), mortality rate was 5.3%, all of which occurred during hospitalization. The in-hospital GRACE score had a c² of 5.96 (p = 0.425) and an AUC of 0.64. The cutoff value was 121.5, with sensitivity of 66.7% and specificity of 74.1%. Conversely, the GRACE score 6 months after discharge had a c² of 5.6 (p = 0.102) and an AUC of 0.59. The cutoff value was 98.5, with sensitivity of 66.7% and specificity of 63% (Figure 2 and Table 3).

Figure 2
– Receiver operating characteristic (ROC) curves for in-hospital and 6-month post-discharge GRACE. AUC: area under ROC curve (C statistics). NSTEMI: non-ST segment myocardial infarction; STEMI: ST segment elevation myocardial infarction.

Table 3
– Predictive accuracy of in-hospital and 6-month post-discharge GRACE scores in NSTEMI and STEMI subgroups

		Area	Sensitivity (%)	Specificity (%)	p-value
In-hospital	NSTEMI	0.64 (0.29-0.98)	66.7	74.1	0.432
	STEMI	0.78 (0.49-1)	80	91.9	0.043
6-month post-discharge	NSTEMI	0.59 (0.25-0.93)	66.7	63	0.592
	STEMI	0.77 (0.57-0.98)	62.5	88.2	0.018

• Area under the receiver operating characteristic curve for in-hospital and 6-month post-discharge GRACE scores in the NSTEMI and STEMI subgroups, with sensitivity and specificity for their cutoff values.
• NSTEMI: Non ST-segment elevation myocardial infarction; STEMI: ST-segment elevation myocardial infarction.

In the ST segment elevation myocardial infarction (STEMI), which had a mortality rate of 11.9%, the in-hospital GRACE score had c² of 8.8 (p = 0.359) and an AUC of 0.78. The cutoff value was 179, with sensitivity of 80% and specificity of 91.9%. Conversely, the GRACE score 6 months after discharge, when cumulative mortality was 19%, had a c² of 7.99 (p = 0.435) and an AUC of 0.77. The cutoff value was 135, with sensitivity of 62.5% and specificity of 88.2% (Figure 2 and Table 3).

The hospital mortality rates for patients with predicted low, intermediate, and high risk according to the in-hospital GRACE score were 2.8% (2 deaths), 2% (1 death), and 13.9% (5 deaths) respectively. Conversely, for patients with predicted low, intermediate, and high risk according to the 6-month post-discharge GRACE score was 3% (3 deaths), 0% (0 death) and 22% (8 deaths), respectively (Figure 3 and Table 4).

Figure 3
– Comparison of mortality rates according to risk stratification by in-hospital and 6-month post-discharge GRACE scores.

Table 4
– Number of deaths according to risk classification

Death		Risk
	Low	Intermediate	High
In-hospital	2 (2.8%)	1 (2%)	5 (13.9%)
6-month post-discharge	3 (5%)	0 (0%)	8 (22.8%)

Discussion

The use of score risk for stratification and prognostic is recommended in the clinical practice by the national and international guidelines on NSTEMI and STEMI.³,⁵ The GRACE score includes quantitative and qualitative variables and has greater discriminative accuracy than other prognostic tools, such as the TIMI risk.³

In our validation study, the GRACE score showed a satisfactory discriminative power. The AUC was 0.76 for the in-hospital GRACE score and 0.78 for the 6-month post-discharge GRACE score. In the classical study of Eagle et al. with 17.142 patients, this score had a discriminative value of 0.81 in patients with ACS.⁶

With regard to the secondary outcome, the subgroup of patients with STEMI showed an in-hospital AUC of 0.78. Two Brazilian studies were conducted with patients with STEMI, one by Correia et al., who reported an AUC of 0.867 in a sample of 152 patients, and another by Sola et al., who shown an AUC of 0.803 in a cohort of 169 individuals from Salvador, state of Bahia.⁷,⁸ In these national studies, it was not possible to compare 6-month post-discharge outcomes, because they were not assessed.

International studies, such as that conducted by Bargos et al. with an Argentinean cohort, found results similar to those of our study for the in-hospital GRACE score, with an AUC of 0.76.⁹ Furthermore, a similar AUC (0.6) for in-hospital mortality was observed em patients with NSTEMI.

The Spanish study by Abu-Ass et al. validated the 6-month post-discharge GRACE score with an AUC of 0.861.¹⁰

In our setting, in-hospital mortality was 5.1%. However, it was 2.8%, 2% and 13.9% for patients with predicted low, moderate, and high mortality risk, respectively. In the Spanish study with 6997 participants conducted by Cordero et al., mortality rate was 5.33% and 0%, 0.6% and 9.6%, respectively.¹¹ Even with a smaller sample, our study found values similar to those obtained in the Spanish study.

The limitation of our study is the fact that there was no exploratory analysis either of the factors related to mortality in our sample or of the impact of the prescribed pharmacological and interventional treatments.

It is the first Brazilian study that showed the validity of the GRACE score beyond in-hospital prognosis.

Conclusion

The GRACE score was validated to predict in-hospital and 6-month post-discharge mortality in our setting in a non-selected sample of patients with ACS. The discriminative power of the score was found to be satisfactory, ratifying recent guidelines that recommend using the GRACE score in risk stratification and selection of intensive early treatment strategies, as well as in the watchful post-discharge follow-up.

References

• ¹
  World Health Organization. World health statistics 2018: monitoring health for the SDGs, sustainable development goals. Geneva: World Health Organization; 2018.
• ²
  Hammami R, Jdidi J, Mroua F, Kallel R, Hentati M, Abid L, et al. Accuracy of the TIMI and GRACE scores in predicting coronary disease in patients with non-ST-elevation acute coronary syndrome. Rev Port Cardiol. 2018;37(1):41-49. doi: 10.1016/j.repc.2017.05.012.
• ³
  Ibanez B, James S, Agewall S, Antunes MJ, Bucciarelli-Ducci C, Bueno H, et al. 2017 ESC Guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation: The Task Force for the management of acute myocardial infarction in patients presenting with ST-segment elevation of the European Society of Cardiology (ESC). Eur Heart J. 2018;39(2):119-177. doi: 10.1093/eurheartj/ehx3937.
• ⁴
  Nicolau JC, Timerman A, Marin-Neto JA, Piegas LS, Barbosa CJ, Franci A, et al. Guidelines of Sociedade Brasileira de Cardiologia for Unstable Angina and Non-ST-Segment Elevation Myocardial Infarction (II Edition, 2007) 2013-2014 Update. Arq Bras Cardiol. 2014;102(3 Suppl 1):1-61. doi: 10.5935/abc.2014S001.
• ⁵
  Collet JP, Thiele H, Barbato E, Barthélémy O, Bauersachs J, Bhatt DL, et al. 2020 ESC Guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation. Eur Heart J. 2021;42(14):1289-1367. doi: 10.1093/eurheartj/ehaa575.
• ⁶
  Eagle KA, Lim MJ, Dabbous OH, Pieper KS, Goldberg RJ, Van de Werf F, et al. A validated prediction model for all forms of acute coronary syndrome: estimating the risk of 6-month postdischarge death in an international registry. JAMA. 2004;291(22):2727-33. doi: 10.1001/jama.291.22.2727.
• ⁷
  Correia LC, Garcia G, Kalil F, Ferreira F, Carvalhal M, Oliveira R, et al. Prognostic value of TIMI score versus GRACE score in ST-segment elevation myocardial infarction. Arq Bras Cardiol. 2014;103(2):98-106. doi: 10.5935/abc.20140095.
• ⁸
  Solla DJF, Novais VON, Carvalho LGMC, Camara SFC, Teixeira LST, Correa LBC, et al. Evaluation of the prognostic value of GRACE score in STEMI in Salvador, Bahia, Brazil: Findings from RESISST. Resuscitation. 2012;83:e65. doi: 10.1016/j.resuscitation.2012.08.166.
• ⁹
  Burgos ML, Costabel JP, Cigalini I, Godoy CL, Zamora SG, Giordanino EF, et al. Validation of the GRACE score (global registry of acute coronary events) as a predictor of in-hospital mortality in patients with acute myocardial infarction in Argentina. Rev Fed Argent Cardiol. 2017;46(4):228-32.
• ¹⁰
  Abu-Assi E, García-Acuña JM, Peña-Gil C, González-Juanatey JR. Validation of the GRACE risk score for predicting death within 6 months of follow-up in a contemporary cohort of patients with acute coronary syndrome. Rev Esp Cardiol. 2010;63(6):640-8. doi: 10.1016/s1885-5857(10)70138-9.
• ¹¹
  Cordero A, Rodriguez-Manero M, García-Acuña JM, López-Palop R, Cid B, Carrillo P, et al. Additive value of the CRUSADE score to the GRACE score for mortality risk prediction in patients with acute coronary syndromes. Int J Cardiol. 2017;245:1-5. doi: 10.1016/j.ijcard.2017.07.095.

• Study Association
  This study is not associated with any thesis or dissertation work.
• Ethics approval and consent to participate
  This study was approved by the Ethics Committee of the Faculdade Meridional – IMED under the protocol number 2.531.453. All the procedures in this study were in accordance with the 1975 Helsinki Declaration, updated in 2013. Informed consent was obtained from all participants included in the study.

• Sources of Funding: There were no external funding sources for this study.

Publication Dates

• Publication in this collection
  16 July 2021
• Date of issue
  Mar-Apr 2022

History

• Received
  02 June 2020
• Reviewed
  19 Jan 2021
• Accepted
  07 Apr 2021