Validation of German Aortic Valve Score in a Multi-Surgeon Single Center

Mehmet Kalender Ahmet Nihat Baysal Okay Guven Karaca Kamil Boyacioglu Nihan Kayalar About the authors

Abstract

OBJECTIVE:

Risk assessment for operative mortality is mandatory for all cardiac operations. For some operation types such as aortic valve repair, EuroSCORE II overestimates the mortality rate and a new scoring system (German AV score) has been developed for a more accurate assessment of operative risk. In this study, we aimed to validate German Aortic Valve Score in our clinic in patients undergoing isolated aortic valve replacement.

METHODS:

A total of 35 patients who underwent isolated open aortic valve replacement between 2010 and 2013 were included. Patients with concomitant procedures and transcatheter aortic valve implantation were excluded. Patients' data were collected and analyzed retrospectively. Patients' risk scores EuroSCORE II were calculated online according to criteria described by EuroSCORE taskforce, Aortic Valve Scores were also calculated.

RESULTS:

The mean age of patients was 61.14±13.25 years (range 29-80 years). The number of female patients was 14 (40%) and body mass index of 25 (71.43%) patients was in range of 22-35. Mean German Aortic Valve Score was 1.05±0.96 (min: 0 max: 4.98) and mean EuroSCORE was 2.30±2.60 (min: 0.62, max: 2.30). The Aortic Valve Score scale showed better discriminative capacity (AUC 0.647, 95% CI 0.439-0.854). The goodness of fit was x2HL=16.63; P=0.436). EuroSCORE II scale had shown less discriminative capacity (AUC 0.397, 95% CI 0.200-0.597). The goodness of fit was good for both scales. The goodness of fit was x2HL=30.10; P=0.610.

CONCLUSION:

In conclusion, German AV score applies to our population with high predictive accuracy and goodness of fit.

Keywords:
Aortic Valve; Risk Assessment; Adult; Risk Grade

INTRODUCTION

The assessment of operative mortality risk is mandatory for all cardiac operations. Patients need to be informed preoperatively about the risk factors. Some risk scoring systems are used to compare and standardize the results of the operations. The European System for Cardiac Operation Risk Evaluation (EuroSCORE) is a risk model published in 1999[11 Roques F, Nashef SA, Michel P, Gauducheau E, de Vincentiis C, Baudet E, et al. Risk factors and outcome in European cardiac surgery: analysis of the EuroSCORE multinational database of 19030 patients. Eur J Cardiothorac Surg. 1999;15(6):816-23.]. For more than a decade, this risk model had been used widely and validated in innumerable papers demonstrating wonderful goodness of fit[22 Roques F, Michel P, Goldstone A, Nashef SA. The logistic EuroSCORE. Eur Heart J. 2003;24(9):881-2.,33 Siregar S, Groenwold RH, de Heer F, Bots ML, van der Graaf Y, van Herwerden LA. Performance of the original EuroSCORE. Eur J Cardiothorac Surg. 2012;41(4):746-54.]. Current requirements necessitated an update to scoring systems which ended up developing EuroSCORE II which was published on May 2010[22 Roques F, Michel P, Goldstone A, Nashef SA. The logistic EuroSCORE. Eur Heart J. 2003;24(9):881-2.]. EuroSCORE II also demonstrated a discriminative capacity similar to EuroSCORE (AUC EuroSCORE II=0.81 vs. AUC EuroSCORE=0.78), and good calibration (x2HL=15.48; P=0.0505)[44 Nashef SA, Roques F, Sharples LD, Nilsson J, Smith C, Goldstone AR, et al. EuroSCORE II. Eur J Cardiothorac Surg. 2012;41(4): 734-44.]. On the other hand, for specific operation types such as aortic valve repair, EuroSCORE II overestimates the mortality rate[55 Gummert JF, Funkat A, Osswald B, Beckmann A, Schiller W, Krian A, et al. EuroSCORE overestimates the risk of cardiac surgery: results from the national registry of the German Society of Thoracic and Cardiovascular Surgery. Clin Res Cardiol. 2009;98(6):363-9.

6 Biancari F, Kangasniemi OP, Aliasim Mahar M, Rasinaho E, Satomaa A, Tiozzo V, et al. Changing risk of patients undergoing coronary artery bypass surgery. Interact Cardiovasc Thorac Surg. 2009;8(1):40-4.
-77 Nilsson J, Algotsson L, Höglund P, Lührs C, Brandt J. Comparison of 19 pre-operative risk strati?cation models in open-heart surgery. Eur Heart J. 2006;27(7):867-74.] which resulted in development of a new scoring system. Some of these new scoring systems emerged nation based such as Ambler, Guaragna and German Aortic Valve score (formerly named AKL-score)[88 Ambler G, Omar RZ, Royston P, Kinsman R, Keogh BE, Taylor KM. Generic, simple risk strati?cation model for heart valve surgery. Circulation. 2005;112(2):224-31.

9 Guaragna JC, Bodanese LC, Bueno FL, Goldani MA. Proposed preoperative risk score for patients candidate to cardiac valve surgery. Arq Bras Cardiol. 2010;94(4):541-8.
-1010 Kötting J, Schillerb W, Beckmann A, Schäfer E, Döbler K, Hamm C, et al. German Aortic Valve Score: a new scoring system for prediction of mortality related to aortic valve procedures in adults. Eur J Cardiothorac Surg. 2013;43(5):971-7.]. German Aortic Valve Score was described by Kötting et al.[1010 Kötting J, Schillerb W, Beckmann A, Schäfer E, Döbler K, Hamm C, et al. German Aortic Valve Score: a new scoring system for prediction of mortality related to aortic valve procedures in adults. Eur J Cardiothorac Surg. 2013;43(5):971-7.] in 2013 with a study in which 1147 isolated aortic valve surgery and transcatheter aortic valve implantation (TAVI) patients were enrolled. German aortic valve score has 15 risk factors (Table 1). Two of them (body mass index - BMI - and no sinus rhythm) are different from EuroSCORE II. EuroSCORE II differs in five parameters comparing to German Aortic Valve score (hand poor mobility, diabetes on insulin, Canadian Cardiovascular Society class 4 angina, weight of the intervention and thoracic aorta surgery) - Table 2.

Table 1
Patients' characteristics. German Aortic Valve Score.
Table 2
Patients' characteristics. EuroSCORE II.

In this study, we aimed to validate German Aortic Valve Score by comparing it with original the EuroSCORE II risk scoring system in patients with isolated open aortic valve replacement.

METHODS

Patients who underwent isolated open aortic valve replacement between May 2010 and June 2013 were included in the study. Those with concomitant procedures, isolated bioprosthesis replaced patients and TAVI were excluded. Patients' data were collected and analyzed retrospectively. Primary end point was observed in hospital mortality. Patients' risk scores EuroSCORE II were calculated online according to criteria described by EuroSCORE taskforce[1111 EuroSCORE. Available at http://www.euroscore.org /Accessed March 21, 2014.
http://www.euroscore.org...
]. Aortic Valve Scores were calculated according to criteria described by Kötting et al.[1010 Kötting J, Schillerb W, Beckmann A, Schäfer E, Döbler K, Hamm C, et al. German Aortic Valve Score: a new scoring system for prediction of mortality related to aortic valve procedures in adults. Eur J Cardiothorac Surg. 2013;43(5):971-7.].

Sensitivity and specificity was assessed by the use of receiver operating characteristic (ROC) curve and the calibration of German Aortic Valve Score was assessed by Hosmer-Lemeshow (HL) test[1212 Hosmer DW, Lemeshow S. A goodness-of-?t test for the multiple logistic regression model. Commu in Stat. 1980;A10:1043-69.]. Calibration was considered to be poor if the test was significant. The discrimination measures the capacity of a model (in this case German Aortic Valve Score and EuroSCORE II) to differentiate the individuals of a sample that suffer an event (in this case, death) and those that do not. The discriminative capacity of the analyzed event was estimated by mean of ROC curve[1313 Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: developing a prognostic model. BMJ. 2009;338:b604.]. For the analysis, the statistical package SPSS® 15.0 (SPSS, Inc., Chicago, IL, USA) for Windows® was used. A P-value <0.05 was considered significant.

RESULTS

We evaluated 35 isolated aortic valve replacement operations in adult patients for this study. The mean age of patients was 61.14±13.25 years (range 29-80 years). The number of female patients was 14 (40%). Patients' characteristics are shown in Tables 1 and 2.

Mean German Aortic Valve Score was 1.05±0.96 (min: 0, max: 4.98) and mean EuroSCORE was 2.30±2.60 (min: 0.62, max: 2.30). The Aortic Valve Score scale showed better discriminative capacity (AUC 0.647, 95% CI 0.439-0.854) (Figure 1). The goodness of fit was x2HL=16.63; P=0.436) (Table 3). EuroSCORE II scale had shown less discriminative capacity (AUC 0.397, 95% CI 0.200-0.597) (Figure 2). The goodness of fit was good for both scales. The goodness of fit was x2HL=30.10; P=0.610 (Table 4).

Table 3
Contingency table for Hosmer–Lemeshow test (German Aortic Valve Score).

Fig. 1
The receiver operating characteristic (ROC) curve of German aortic valve score.

Fig. 2
The receiver operating characteristic (ROC) curve of EuroSCORE II.

Table 4
Contingency table for Hosmer–Lemeshow test (EuroSCORE II).

DISCUSSION

Risk scoring systems are valuable for benchmarking of institution results, however, several risk scoring systems have been developed and used. EuroSCORE II is a new updated scoring system with better mortality score and goodness of fit. But some statistical questions have been raised recently[1414 Sergeant P, Meuris B, Pettinari M. EuroSCORE II, illum qui est gravitates magni observe. Eur J Cardiothorac Surg. 2012;41(4):729-31.,1515 Collins GS, Altman DG. Design flaws in EuroSCORE II. Eur J Cardiothorac Surg. 2013;43(4):871.]. Moreover, parallel to our opinion there are papers advocating that one scoring system for all patient groups, cardiac diseases and therapies can certainly be misleading[1010 Kötting J, Schillerb W, Beckmann A, Schäfer E, Döbler K, Hamm C, et al. German Aortic Valve Score: a new scoring system for prediction of mortality related to aortic valve procedures in adults. Eur J Cardiothorac Surg. 2013;43(5):971-7.,1616 Hannan EL, Wu C, Bennett EV, Carlson RE, Culliford AT, Gold JP, et al. Risk index for predicting in-hospital mortality for cardiac valve surgery. Ann Thorac Surg. 2007;83(3):921-9.

17 Zheng Z, Li Y, Zhang S, Hu S, Chinese CABG Registry Study. The Chinese coronary artery bypass grafting registry study: how well does the EuroSCORE predict operative risk for Chinese population? Eur J Cardiothorac Surg. 2009;35(1):54-8.
-1818 Howell NJ, Head SJ, Freemantle N, van der Meulen TA, Senanayake E, Menon A, et al. The new EuroSCORE II does not improve prediction of mortality in high-risk patients undergoing cardiac surgery: a collaborative analysis of two European centres. Eur J Cardiothorac Surg. 2013;44(6):1006-11.]. EuroSCORE II was also based on a data set consisting mainly of coronary procedures. Therefore, we believe that there is a requirement for a new scoring system more adaptive for aortic valve procedures. There are also papers reporting the requirement of a new scoring system for aortic valve procedures[88 Ambler G, Omar RZ, Royston P, Kinsman R, Keogh BE, Taylor KM. Generic, simple risk strati?cation model for heart valve surgery. Circulation. 2005;112(2):224-31.,1010 Kötting J, Schillerb W, Beckmann A, Schäfer E, Döbler K, Hamm C, et al. German Aortic Valve Score: a new scoring system for prediction of mortality related to aortic valve procedures in adults. Eur J Cardiothorac Surg. 2013;43(5):971-7.,1919 Florath I, Rosendahl UP, Mortasawi A, Bauer SF, Dalladaku F, Ennker IC, et al. Current determinants of operative mortality in 1400 patients requiring aortic valve replacement. Ann Thorac Surg. 2003;76(1):75-83.

20 Jin R, Grunkemeier GL, Starr A, Providence Health System Cardiovascular Study Group. Validation and refinement of mortality risk models for heart valve surgery. Ann Thorac Surg. 2005;80(2):471-9.
-2121 Nowicki ER, Birkmeyer NJ, Weintraub RW, Leavitt BJ, Sanders JH, Dacey LJ, et al.; Northern New England Cardiovascular Disease Study Group and the Center for Evaluative Clinical Sciences, Dartmouth Medical School. Multivariable prediction of in-hospital mortality associated with aortic and mitral valve surgery in Northern New England. Ann Thorac Surg. 2004;77(6):1966-77.]. Kotting et al.[1010 Kötting J, Schillerb W, Beckmann A, Schäfer E, Döbler K, Hamm C, et al. German Aortic Valve Score: a new scoring system for prediction of mortality related to aortic valve procedures in adults. Eur J Cardiothorac Surg. 2013;43(5):971-7.] described a new sco ring system for aortic valve procedures based on German Registry.

Former predictive models were developed for specific locations , but global need made EuroSCORE and STS popular and they were used widely. As Casalino et al.[2222 Casalino R, Tarasoutchi F, Spina G, Katz M, Bacelar A, Sampaio R, et al. EuroSCORE models in a cohort of patients with valvular heart disease and a high prevalence of rheumatic fever submitted to surgical procedures. PLoS One. 2015;10(2):e0118357.] reported in their study that German Aortic Valve Score best fits in German population, but in our opinion it may be applicable to our population as well. Our results showed a high quality of discrimination AUC 0.647 and Hosmer-Lemeshow method exhibited sufficient concordance in the predicted and observed mortality (x2HL=16.63; P=0.436).

Non-randomized and retrospectively design, single institution setting, multi-surgeon operations and small sample size were the major limitations of our study.

CONCLUSION

In conclusion, German Aortic Valve score applies to our population with high predictive accuracy and goodness of fit.

  • This study was carried out at the Konya Education Research Hospital, Cardiovascular Surgery Department, Konya, Turkey.
  • No financial support.

ACKNOWLEDGEMENTS

Special thanks to Necla Yildiz and Cemile Cevik for their enormous help during the writing process.

REFERENCES

  • 1
    Roques F, Nashef SA, Michel P, Gauducheau E, de Vincentiis C, Baudet E, et al. Risk factors and outcome in European cardiac surgery: analysis of the EuroSCORE multinational database of 19030 patients. Eur J Cardiothorac Surg. 1999;15(6):816-23.
  • 2
    Roques F, Michel P, Goldstone A, Nashef SA. The logistic EuroSCORE. Eur Heart J. 2003;24(9):881-2.
  • 3
    Siregar S, Groenwold RH, de Heer F, Bots ML, van der Graaf Y, van Herwerden LA. Performance of the original EuroSCORE. Eur J Cardiothorac Surg. 2012;41(4):746-54.
  • 4
    Nashef SA, Roques F, Sharples LD, Nilsson J, Smith C, Goldstone AR, et al. EuroSCORE II. Eur J Cardiothorac Surg. 2012;41(4): 734-44.
  • 5
    Gummert JF, Funkat A, Osswald B, Beckmann A, Schiller W, Krian A, et al. EuroSCORE overestimates the risk of cardiac surgery: results from the national registry of the German Society of Thoracic and Cardiovascular Surgery. Clin Res Cardiol. 2009;98(6):363-9.
  • 6
    Biancari F, Kangasniemi OP, Aliasim Mahar M, Rasinaho E, Satomaa A, Tiozzo V, et al. Changing risk of patients undergoing coronary artery bypass surgery. Interact Cardiovasc Thorac Surg. 2009;8(1):40-4.
  • 7
    Nilsson J, Algotsson L, Höglund P, Lührs C, Brandt J. Comparison of 19 pre-operative risk strati?cation models in open-heart surgery. Eur Heart J. 2006;27(7):867-74.
  • 8
    Ambler G, Omar RZ, Royston P, Kinsman R, Keogh BE, Taylor KM. Generic, simple risk strati?cation model for heart valve surgery. Circulation. 2005;112(2):224-31.
  • 9
    Guaragna JC, Bodanese LC, Bueno FL, Goldani MA. Proposed preoperative risk score for patients candidate to cardiac valve surgery. Arq Bras Cardiol. 2010;94(4):541-8.
  • 10
    Kötting J, Schillerb W, Beckmann A, Schäfer E, Döbler K, Hamm C, et al. German Aortic Valve Score: a new scoring system for prediction of mortality related to aortic valve procedures in adults. Eur J Cardiothorac Surg. 2013;43(5):971-7.
  • 11
    EuroSCORE. Available at http://www.euroscore.org /Accessed March 21, 2014.
    » http://www.euroscore.org
  • 12
    Hosmer DW, Lemeshow S. A goodness-of-?t test for the multiple logistic regression model. Commu in Stat. 1980;A10:1043-69.
  • 13
    Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: developing a prognostic model. BMJ. 2009;338:b604.
  • 14
    Sergeant P, Meuris B, Pettinari M. EuroSCORE II, illum qui est gravitates magni observe. Eur J Cardiothorac Surg. 2012;41(4):729-31.
  • 15
    Collins GS, Altman DG. Design flaws in EuroSCORE II. Eur J Cardiothorac Surg. 2013;43(4):871.
  • 16
    Hannan EL, Wu C, Bennett EV, Carlson RE, Culliford AT, Gold JP, et al. Risk index for predicting in-hospital mortality for cardiac valve surgery. Ann Thorac Surg. 2007;83(3):921-9.
  • 17
    Zheng Z, Li Y, Zhang S, Hu S, Chinese CABG Registry Study. The Chinese coronary artery bypass grafting registry study: how well does the EuroSCORE predict operative risk for Chinese population? Eur J Cardiothorac Surg. 2009;35(1):54-8.
  • 18
    Howell NJ, Head SJ, Freemantle N, van der Meulen TA, Senanayake E, Menon A, et al. The new EuroSCORE II does not improve prediction of mortality in high-risk patients undergoing cardiac surgery: a collaborative analysis of two European centres. Eur J Cardiothorac Surg. 2013;44(6):1006-11.
  • 19
    Florath I, Rosendahl UP, Mortasawi A, Bauer SF, Dalladaku F, Ennker IC, et al. Current determinants of operative mortality in 1400 patients requiring aortic valve replacement. Ann Thorac Surg. 2003;76(1):75-83.
  • 20
    Jin R, Grunkemeier GL, Starr A, Providence Health System Cardiovascular Study Group. Validation and refinement of mortality risk models for heart valve surgery. Ann Thorac Surg. 2005;80(2):471-9.
  • 21
    Nowicki ER, Birkmeyer NJ, Weintraub RW, Leavitt BJ, Sanders JH, Dacey LJ, et al.; Northern New England Cardiovascular Disease Study Group and the Center for Evaluative Clinical Sciences, Dartmouth Medical School. Multivariable prediction of in-hospital mortality associated with aortic and mitral valve surgery in Northern New England. Ann Thorac Surg. 2004;77(6):1966-77.
  • 22
    Casalino R, Tarasoutchi F, Spina G, Katz M, Bacelar A, Sampaio R, et al. EuroSCORE models in a cohort of patients with valvular heart disease and a high prevalence of rheumatic fever submitted to surgical procedures. PLoS One. 2015;10(2):e0118357.

Publication Dates

  • Publication in this collection
    Mar-Apr 2017

History

  • Received
    02 Oct 2016
  • Accepted
    29 Dec 2016
Sociedade Brasileira de Cirurgia Cardiovascular Rua Afonso Celso, 1178 Vila Mariana, CEP: 04119-061 - São Paulo/SP Brazil, Tel +55 (11) 3849-0341, Tel +55 (11) 5096-0079 - São Paulo - SP - Brazil
E-mail: bjcvs@sbccv.org.br