Application of Risks Scores in Acute Coronary Syndromes. How Does ProACS Hold Up Against Other Risks Scores?

Background Multiple risk scores (RS) are approved in the prediction of worse prognosis in acute coronary syndromes (ACS). Recently, the Portuguese Journal of Cardiology has proposed the ProACS RS. Objective Application of several validated RS, as well as ProACS in patients, admitted for ACS. Evaluation of each RS's performance in predicting in-hospital mortality and the occurrence of all-cause mortality or non-fatal ACS at one-year follow-up and compare them to the ProACS RS. Methods A retrospective study of ACS was performed. The following RS were applied: GRACE, ACTION Registry-GWTG, PURSUIT, TIMI, EMMACE, SRI, CHA2DS2-VASc-HS, C-ACS and ProACS. ROC Curves were created to determine the predictive power for each RS and then were directly compared to ProACS. Results The ProACS, ACTION Registry-GWTG and GRACE showed a c-statistics of 0.908, 0.904 and 0.890 for predicting in-hospital mortality, respectively, performing better in ST-segment elevation myocardial infarction patients. The other RS performed satisfactorily, with c-statistics over 0.750, apart from the CHA2DS2-VASc-HS and C-ACS which underperformed. All RS underperformed in predicting worse long-term prognosis revealing c-statistics under 0.700. Conclusion ProACS is an easily obtained risk score for early stratification of in-hospital mortality. When evaluating all RS, the ProACS, ACTION Registry-GWTG and GRACE RS showed the best performance, demonstrating high capability of predicting a worse prognosis. ProACS was able to demonstrate statistically significant superiority when compared to almost all RS. Thus, the ProACS has showed that it is able to combine simplicity in the calculation of the score with good performance in predicting a worse prognosis.


Introduction
Cardiovascular disease is the most common cause of death worldwide. 1,2 In the past three to four decades, studies have shown a significant reduction in acute and long-term mortality by acute coronary syndromes (ACS). [1][2][3] This is attributed to improvements in medical therapy and invasive strategies. [1][2][3] However, ACS represent a heterogeneous group, with varying risk of morbimortality. [1][2][3][4] Several risk stratification models have been developed to determine which patients carry a higher probability of worst outcome. [3][4][5][6][7][8][9][10][11][12][13] Early risk stratification is crucial to ensure a tailored approach to each individual patient, weighing both the risks and benefits of each treatment option. 1,2 Recently, there has been a systematic approach to risk assessment, with the creation of a myriad of risks scores (RS). [3][4][5][6][7][8][9][10][11][12][13] Perhaps, the Global Registry of Acute Coronary Events (GRACE) Risk Score 6 is the most widely recognized RS in ACS. According to the most recent European guidelines, the GRACE risk score is recommended when stratifying risk in ST-segment elevation myocardial infarction (STEMI) and non-ST-segment elevation myocardial infarction, and/or unstable angina (NSTEMI/UA). 1,2 However, there are several other known RS, such as: the Thrombolysis in Myocardial Infarction (TIMI) for STEMI; 7 the Platelet glycoprotein IIb/IIa in Unstable angina: Receptor Suppression Using Integrilin Therapy (PURSUIT); 5 the Simple Risk Index (SRI); 8 the Evaluation of the Methods and Management of Acute Coronary Events (EMMACE); 9 and more recently, the Canada Acute Coronary Syndrome (C-ACS), 10 the CHA 2 DS 2 -VASc-HS score, 11 and the ACTION (Acute Coronary Treatment and Intervention Outcomes Network) Registry-GWTG. 12 In 2016, the Portuguese Journal of Cardiology published a new risk score, formulated using the Portuguese Registry on Acute Coronary Syndromes. The Portuguese Registry of Acute Coronary Syndromes was established in 2002, 14 under the auspice of the Portuguese Society of Cardiology. It is an observational, multicentric, nationwide prospective study in which each hospital participates with data from all patients admitted for ACS. The group developed the simple but effective ProACS risk score for predicting in-hospital mortality, which can be easily applied even in a pre-hospital setting. 3 The objective of this article is to calculate all of the RS in patients admitted for ACS, in a single-centre study. The authors evaluate each RS's performance in predicting in-hospital mortality and compare them specifically to the ProACS RS. The authors also determine each RS's performance at predicting worse outcome in STEMI and NSTEMI/UA independently. Finally, the authors access each RS´s ability to predict mortality and recurring ACS at one-year follow-up.

Methods
This is a retrospective study of patients admitted for ACS to a Coronary Care Unit of a centralized hospital, from December 2006 to May 2016. Only patients presenting with a history of chest pain at rest or other symptoms suggestive of an ACS with or without new significant ST-segment or T-wave changes, new left bundle branch block or elevated biomarkers of myocardial damage were included. Of the 1714 patients included in the study period, 1452 were selected, with the remaining patients being excluded due to missing data. The population sample of this study was not included in the development cohort used to formulate the ProACS risk score, 3 although it has been included in the validation cohorts.
The following RS were calculated for all patients: GRACE, TIMI for STEMI, PURSUIT, SRI, EMMACE, C-ACS, CHA 2 DS 2 -VASc-HS, ACTION Registry-GWTG and ProACS. All RS were calculated using data from the initial clinical history, electrocardiogram and laboratory values collected on admission. All patients included were followed up for at least one year or until the occurrence of a major event.
The primary endpoint of this study was in-hospital mortality and the combination of all-cause mortality or non-fatal ACS at one-year follow-up.

Statistical analysis
Categorical variables were characterized by percentages. Group comparisons, with respect to these variables, were performed through chi-square or Fisher's exact test. Numeric continuous variables were expressed as mean ± standard deviation and RS as median with interquartile range, given their ordinal nature. Group comparisons were achieved through the Mann-Whitney test since the normality assumption was not satisfied for any of the studied numeric variables. Comparative analyses were carried out in relation to demographic variables, therapeutic strategies and general outcome parameters. The RS was evaluated by receiver operating characteristic (ROC) curves, and their area under the curve (AUC), with respect to their ability to differentiate patients with and without adverse clinical events, regarding in-hospital mortality and the combination of all-cause mortality or non-fatal ACS at one-year follow-up. The comparison of AUCs, for each RS with ProACS, was done by the method described by DeLong et al. 15 The Hosmer-Lemeshow Test 16 was used to evaluate the goodness of fit for each risk score. Two-sided p-values are reported and a p-value < 0.05 was considered statistically significant. Statistical analysis was performed with SPSS version 17.0 (SPSS Inc., Chicago, IL, USA) and MedCalc version 18.2.1 (MedCalc Software, Osted, Belgium).

Results
Baseline characteristics and univariate predictors of worse outcome A total of 1,452 patients were included in this study. The baseline characteristics are displayed in Table 1. Regarding in-hospital mortality, 6.5% of the patients died. At one-year follow-up, 9.9% of the patients either died or suffered a non-fatal ACS. Table 2 displays the univariate predictors for in-hospital mortality and for all-cause mortality and non-fatal ACS at one-year follow-up. Regarding in-hospital mortality, it is evident that older patients have higher mortality, with chronic kidney disease being associated with a worse prognosis. The clinical presentation also influences the outcome. Lower blood pressure and higher heart rate, as well as higher Killip-Kimball (KK) class, were linked to a higher mortality rate. It is also evident that lower haemoglobin and higher creatinine, troponin and brain natriuretic peptide values is associated with a worse prognosis, as well as a lower left ventricular ejection fraction. Regarding the occurrence of events at one-year, follow-up older and female patients tend to have higher mortality. NSTEMI/UA is associated with a worse prognosis. Lower diastolic blood pressure, a higher heart rate and KK class are associated with higher rate of events. Concerning past medical history, diabetes mellitus, chronic kidney disease and previous known coronary disease are associated with a worse outcome. A higher rate of events at 1-year follow-up was seen in patients medicated previously to the index event with statin, renin-angiotensin-aldosterone system inhibitors, beta-blocker and antiplatelet therapy. All RS scored significantly higher in the groups with worse outcome, both in in-hospital mortality and at one-year follow-up.
Predictive accuracy of the risk scores Table 3, Table 4 and Table 5 describe the predictive accuracy and goodness of fit of the RS at predicting in-hospital mortality globally, at predicting in-hospital mortality in the specific group of STEMI and NSTEMI/UA patients individually, and at predicting occurrence of all-cause mortality and non-fatal ACS at one-year follow-up, respectively. The last column of each table show how the other RS compare to the ProACS score. Figure 1 displays the ROC curves regarding the RS and in-hospital mortality. Figure 2 shows in-hospital mortality in the STEMI and NSTEMI group individually. The long-term prognosis is demonstrated in Figure 3.
The majority of the RS showed a good discriminatory accuracy to predict in-hospital mortality, as demonstrated by c-statistics consistently over 0.700. Notably, three RS outperformed the others, namely the GRACE, ACTION  All RS consistently showed better discriminatory accuracy at predicting in-hospital mortality in STEMI patients. In this population, the ACTION Registry-GWTG and ProACS RS performed incredibly well, with c-statistics of over 0.900. Almost all RS revealed an adequate fit, except for the SRI (p = 0.011), the C-ACS (p = 0.005) and a trend from PURSUIT (p = 0.075). In STEMI patients, the ACTION Registry-GWTG (p = 0.882) and ProACS RS (p = 0.821) showed good fit. The ProACS RS demonstrated statistically significant superior discriminatory accuracy when compared to all other RS, except for ACTION Registry-GWTG (p = 0.2248).

Registry-GWTG and
In the NSTEMI population, the RS performed slightly worse when compared to STEMI patients. ProACS, ACTION Registry-GWTG and GRACE scores were the RS with the highest discriminatory accuracy at predicting in-hospital mortality, with c-statistics of 0.898, 0.895 and 0.878 respectively. ProACS also demonstrated significant superiority, except when compared to the aforementioned RS. Regarding all-cause mortality and non-fatal ACS at one-year follow-up, all RS underperformed, with c-statistics consistently under 0.700. ProACS was only statistically superior to the C-ACS RS, which showed particularly poor discriminatory accuracy (c-statistic 0.550). Most RS revealed model lack of fit.

Development of risks scores in acute coronary syndromes
Advances in medical therapy and the development of invasive strategies has had a significant impact on prognosis in ACS. [1][2][3] Risk stratification has become an essential part of the establishment of a personalized treatment strategy in patients with ACS, weighing the risks and benefits of an early invasive approach. 1,2 In STEMI patients, primary percutaneous coronary intervention is the standard approach, thus early risk stratification is less important. 1,3 However, risk stratification in STEMI still plays an important role in predicting which patients are at higher risk for mortality or recurrent ACS, thus warranting a more aggressive medical therapy. 1  with early risk stratification playing a more central role in deciding which patients benefit more from an early invasive strategy. [2][3][4][5]13 Several risk score have been formulated in the last 20 years, attempting to best predict which patients are at a higher risk for a worse outcome. [3][4][5][6][7][8][9][10][11][12][13] The simple TIMI risk score for STEMI 7 and for NSTEMI/UA 13 was developed from large clinical trials, with controlled and selected populations. The TIMI RS for STEMI 7 was formulated from the InTIME II trial which enrolled a total of 15,078 patients, all were candidates for fibrinolytic therapy. This risk score performed well at identifying high risk patients (c-statistics for predicting in-hospital mortality and in the first 24 hours after admission was 0.784 and 0.813, respectively). 7 The TIMI for NSTEMI/UA was developed using the database of the TIMI 11B trial, with a total of 3910 patients, satisfactorily predicting all-cause mortality, myocardial infarction or urgent revascularization at 14 days. 13 However, since it underperformed in our population, the authors decided not to use the RS. The SRI risk score was also calculated from the InTIME trial, using a cohort of 13,253 STEMI patients. This risk score satisfactorily predicted in-hospital death (c-statistic 0.79). The PURSUIT risk score was developed through the Platelet glycoprotein IIb/IIIa in Unstable angina: Receptor Suppression Using Integrilin (eptifibatide) Therapy trial using a NSTEMI population of 9,461 patients, with a Table 3    c-statistic of 0.814. These RS are simple and intuitive, however, derivation from large trial databases tend to overlook specific high-risk patients. [3][4][5] The GRACE risk score was developed using an international registry, much more representative of real-world patients, with a total of 11,389 patients enrolled. 6 The GRACE risk score outperformed previous RS which tended to use clinical trial data. GRACE showed good predictive capacity for in-hospital mortality and at 6-month follow-up. 6,17 This risk score was updated using a cohort of 48,023 patients 18 and has become the most widely used risk score both in STEMI and NSTEMI/UA. 1,2 The EMMACE risk score was also developed from patients admitted for ACS over a 3-month period in 1995, compiling a total of 2,135 patients. 9 A mathematical formula only comprising 3 variables (age, heart rate and systolic blood pressure) was formulated and revealed good performance at predicting mortality at 30 days (c-statistics of 0.76 to 0.79). This risk score is simple and reproducible. 9  The Journal of the American College of Cardiology published in 2016 a new risk score. 11 This risk score was developed using data from the ACTION Registry-GWTG, which included a total of 145,952 patients from more than 300 hospitals from the United States of America admitted for both STEMI and NSTEMI. 11 The ACTION Registry-GWTG risk score performed well in the general population (c-statistic 0.88), as well as in specific subsets of patients. 11 This score appeared to be a good alternative to the GRACE score.

-Predictive accuracy and goodness of fit of the scores at predicting in-hospital mortality and comparation with the ProACS risk score
Finally, in 2017, the Portuguese Journal of Cardiology presented a new and simple risk score. 3 The ProACS risk score, formulated by Timóteo et al. 3 was developed using the Portuguese Registry of Acute Coronary Syndromes. The risk model was developed from the data of 17,380 patients. Internal and external validation of the score was done using 12,701 and 8,532 patients, respectively. 3 Timóteo et al. 3 built a simple risk score with only 4 variables, age, systolic blood pressure, Killip class and ST-segment elevation (information easily obtainable even in a pre-hospital setting). The score performed well in predicting in-hospital mortality, both in STEMI and NSTEMI (c-statistics ranging from 0.785 to 0.809). This risk score was formulated similarly to the C-ACS, a score formulated by a Canadian group and published in the American Heart Journal. The C-ACS developed a simple score with 4 variables (age ≥75, Killip class >I, systolic blood pressure < 100 mmHg and heart rate > 100 beats/min. 10 The score was derived from the Acute Myocardial Infarction in Quebec (AMI-QUEBEC) and Canada ACS-1 registries, compiling a total of 6,182 patients. 10 This score performed well at predicting worse outcome both in short-term (c-statistics ranging from 0.73 to 0.75) and in long-term mortality (c-statistics ranging from 0.73 to 0.76). 10 It was the authors´ objective to test several RS, which have been validated in the setting of ACS, to determine which one fared better at balancing a good predictive capability, combined with simple and intuitive use. The authors decided to apply the aforementioned RS in a single-centre population of patients admitted for ACS and compare each score to the ProACS.

Risk scores and in-hospital mortality
Almost all RS performed well. However, the CHA 2 DS 2 -VASc-HS and the C-ACS scores underperformed in this population (c-statistics of 0.600 and 0.619, respectively), even though both have been validated for the prediction of short-term mortality. The TIMI for STEMI, PURSUIT, SRI and EMMACE RS performed moderately well, with a c-statistics of 0.744, 0.775, 0.732 and 0.749, respectively. Of all the RS, three outperformed the other, achieving extremely good c-statstics, namely the ProACS, GRACE and ACTION Registry-GWTG RS. All of the RS predict short-term mortality. However, not all are equally efficient. The ProACS, GRACE and ACTION Registry-GWTG RS performed incredibly well when determining short-term mortality. A c-statistics of 0.908, 0.904 and 0.890 was calculated for each respective score. These results demonstrate a greater efficiency that that shown in previous studies. 3,6,12,18 The ProACS demonstrated impressive results. It was significantly better than all the other RS, apart from the ACTION Registry-GWTG and GRACE RS. The only setback was a HL-test value of under 0.05 in both the ProACS and ACTION Registry-GWTG, indicating model lack of fit. This resulted from the presence of NSTEMI patients in the study.
All RS performed better at predicting in-hospital mortality in a STEMI setting. Again, the ProACS, ACTION Registry-GWTG and GRACE RS were the more accomplished RS, attaining c-statistics of 0.923, 0.911 and 0.899, respectively. These numbers are especially impressive since they outperformed each of their derivation and validation cohorts. 3,12,18 ProACS demonstrated statistical superiority when compared to all others RS, apart from the ACTION Registry-GWTG and only marginal superiority when compared to the GRACE RS. In STEMI patients, these three RS revealed good fit. Once more, the TIMI for STEMI, PURSUIT, SRI and EMMACE RS had a satisfactory performance (c-statistics of 0.785, 0.809, 0.781 and 0.795, respectively). The CHA 2 DS 2 -VASc-HS and the C-ACS RS performed disappointingly, with c-statistics of 0.674 and 0.620, respectively.
Concerning NSTEMI, the ProACS, GRACE and ACTION Registry-GWTG achieved a good predictive power, with c-statistics of 0.898, 0.878 and 0.895. In this particular population, both the ProACS and the ACTION Registry-GWTG showed lack of fit, thus interfering with the goodness of fit in the general population in these RS. The PURSUIT and EMMACE RS performed moderately good, with c-statistics of 0.742 and 0.702. It is impressive that the PURSUIT RS performed better at predicting a worse outcome in STEMI when compared to NSTEMI, since it is based upon NSTEMI patients. 4,5 The TIMI for STEMI and SRI predictably underperformed (c-statistics of 0.696 and 0.682), since both were developed for STEMI patients. 7 Again, the C-ACS revealed poor discriminatory accuracy (c-statistic 0.618) and the CHA 2 DS 2 -VASc-HS was unable to predict in-hospital mortality in NSTEMI patients (c-statistic of 0.534, p = 0.453).

Risk scores and long-term prognosis
The majority of the RS evaluated were developed solely for prediction of short-term prognosis. [3][4][5][6][7][8][9][10][11][12][13] In this population, all the RS underperformed when predicting all-cause mortality and non-fatal ACS at one-year follow-up (c-statistics < 0.7). Almost all the RS presented with a c-statistic ranging from 0.622 to 0.690, without a statistically significant difference when compared with the ProACS. Notably, the C-ACS was unable to predict the worst long-term prognosis (c-statistic 0.550, p = 0.057), even though it was validated for long-term prognosis prediction. 10 More studies are needed to develop RS with better discriminatory accuracy for predicting long-term prognosis in ACS patients.

Limitations
This is a single-centre retrospective, observational study of a small population. The analysis of the parameters was based on nonrandomized data. The population sample was relatively small and was composed by the sequential patients admitted in a single centralized hospital, thus it might represent a biased sample.

Conclusions
In this population, several RS showed good discriminatory accuracy at predicting short-term mortality. The ProACS, GRACE and ACTION Registry-GWTG RS performed incredibly, with c-statistics around 0.90. This revealed great predictive capability both in STEMI and NSTEMI patients. The TIMI for STEMI, PURSUIT, SRI and EMMACE RS performed moderately well. However, the CHA 2 DS 2 -VASc-HS 20 and the C-ACS underperformed, perhaps due to differences between the cohort from which they were based on and the population sample of this study. In this real-world population, it is evident that RS developed from databases of large registries, such as the GRACE, ProACS and the ACTION Registry-GWTG, seem to fare better than those derived from clinical trials. RS developed from clinical trials tend to include skewed populations which avoid high-risk patients. None of the RS performed well at predicting long-term prognosis. This is understandable given they were intended for the prediction of short-term mortality.
The ProACS risk score proved to be an effective risk model, which performed incredibly well in this population, in both STEMI and NSTEMI patients. It is an intuitive risk score that requires only four easily obtainable variables. Its simplicity is rivalled only by the C-ACS, which has significantly underperformed in every aspect. The authors believe that ProACS is an appropriate and simple method to obtain adequate risk stratification regarding short-term prognosis that applies well to the Portuguese population.

Potential Conflict of Interest
No potential conflict of interest relevant to this article was reported.

Sources of Funding
There were no external funding sources for this study.

Study Association
This study is not associated with any thesis or dissertation work.

Ethics approval and consent to participate
This article does not contain any studies with human participants or animals performed by any of the authors.