CARDIAC SURGERY
Validation of EuroSCORE II risk model for coronary artery bypass surgery in high-risk patients

Introduction

Determining operative mortality risk is mandatory for all cardiac operations. Patients have to be informed preoperatively about the risk factors. Some risk scoring systems are used to compare and standardize the results of the operations. The European System for Cardiac Operation Risk Evaluation (EuroSCORE) is a risk model described in 1999 [1]. For more than a decade, this risk model has been used widely and validated innumerable times, demonstrating wonderful goodness of fit [2, 3]. Although there are many risk models used globally, risk scoring systems are relatively outdated. Therefore, the update of scoring systems was required, so EuroSCORE II was published on May 2010 [4]. EuroSCORE II has demonstrated a discriminative capacity similar to EuroSCORE (AUCEuroSCORE II = 0.81 vs. AUCEuroSCORE = 0.78), and good calibration (x2HL [EuroSCORE II] = 15.48; p = 0.0505) [5].
In this study, we aimed to validate EuroSCORE II in comparison with the original EuroSCORE in a group of octogenarian patients with high preoperative risk who underwent isolated coronary artery bypass grafting (CABG).

Material and methods

In this study we included only octogenarian high-risk patients who underwent CABG from January 2000 to January 2010. Redo and concomitant procedures were excluded. Patients’ data were collected and analyzed retrospectively. Cardiovascular risk score of all patients was calculated by additive and logistic EuroSCORE and EuroSCORE II according to the criteria described by the EuroSCORE taskforce [6]. Patients were classified in three groups by additive EuroSCORE. All patients had a minimum score of 5 due to their age. So patients with an additive EuroSCORE of 5 to 8 were considered to have low risk, 8 to 10 moderate risk and higher than 10 high risk. We compared the observed mortality with the expected mortality according to logistic EuroSCORE and EuroSCORE II, which was calculated online [7].
Patient characteristics are shown in Table I. Additive EuroSCORE, logistic EuroSCORE and EuroSCORE II models were compared based on sensitivity and specificity. Sensitivity and specificity were assessed by receiver operating characteristic (ROC) analysis and the calibration of both scales was assessed by the Hosmer-Lemeshow (HL) test. Calibration was considered to be poor if the HL test was significant. The discrimination measures the capacity of a model (in this case additive and logistic EuroSCORE and EuroSCORE II) to differentiate the individuals of a sample who suffer an event (in this case death) and those who do not. The discriminative capacity of the three scales was estimated by means of ROC curves [8]. For the statistical analysis, the Statistical Package for Social Sciences (SPSS) version 15.0 (SPSS, Inc., Chicago, IL, USA) for Windows was used. A p-value < 0.05 was considered significant.

Results

We considered 105 CABG operations on high-risk octogenarian patients for this study from January 2000 to January 2010. The mean (standard deviation; SD) age of the patients was 81.43 ± 2.21 (range: 80-89) years; 39 (37.1%) of them were female.
The two scales showed good discriminative capacity in the global patient sample, with the area under the ROC curve (AUC) being higher for EuroSCORE II (0.772, 95% CI: 0.673-0.872) (Fig. 1). The goodness of fit was good for both scales (Table II). In the low-risk subgroup all scales had good discriminative capacity with EuroSCORE II still being better than others (AUC: 0.774; 0.776; 0.816). However, in the moderate- and high-risk subgroups all scales showed poor discriminative capacity (Figs. 2-4).
Benchmarking of our institutional mortality rates revealed worse prediction upon EuroSCORE II scoring compared to EuroSCORE (Fig. 5).

Discussion

The development of EuroSCORE II eliminated insufficiencies observed in EuroSCORE such as low prevalence of octogenarians and valve surgery. Additionally, due to the progress in cardiac surgery, the impact of renal function on mortality decreased. Finally, EuroSCORE II was capable of predicting hospital mortality after major cardiac surgery with an excellent discriminative capacity (AUC = 0.81, 95% CI: 0.78-0.83) [5]. Alcazar et al. validated EuroSCORE II on 3798 patients, concluding that EuroSCORE II was a good discriminative method but with poor calibration [9]. Nashef et al. also advocated this conclusion with 5553 cases [5]. Howell and colleagues reported EuroSCORE II to be a model with poor calibration (p = 0.035) and original EuroSCORE to have a statistically significantly better model fit (the difference in AIC was –5.66; p = 0.017) in high-risk patients [10].
By applying both logistic models on the whole group, no statistically significant differences were observed comparing AUCEuroSCORE and AUCEuroSCORE II (Fig. 1). We compared the patients grouped according to additive EuroSCORE, and finally neither model did well, with statistically insignificant AUC results (Figs. 2-4). But our subgroups were statistically different and the numbers were small. On the other hand, when ROC analysis was applied to the whole study group, both models did well (Fig. 1), and also we observed that EuroSCORE II had better discriminative values. Parallel to our results, Chalmers et al. validated EuroSCORE II with 5576 subjects and concluded that EuroSCORE II has good discriminative capacity and good calibration (C-statistic 0.87 and HL p = 0.6) [11]. Also Akgul et al. reported a good C-statistic value of EuroSCORE II compared to the original EuroSCORE (0.992 [95% CI: 0.977-0.998] for logistic EuroSCORE and 0.990 [95% CI: 0.975-0.997] for EuroSCORE II) and in the subgroup of high risk (additive EuroSCORE > 6) they found that again EuroSCORE II was better (0.857 [95% CI: 0.691-0.954] for logistic EuroSCORE and 0.961 [95% CI: 0.829-0.998] for EuroSCORE II) [12].
In our study, we observed that the original EuroSCORE overestimates compared to EuroSCORE II, but we had high mortality rates compared to STS (Society of Thoracic Surgeons) results (20% and 6.8% respectively) and both risk models (Fig. 1). Chalmers et al. claim that EuroSCORE II has better calibration for cumulative sum survival (CUSUM) curves [11]. In the medical literature, there are papers supporting the results of the original EuroSCORE for the Turkish population, but no study specifically analyzed high-risk patients [13-16]. At this point EuroSCORE II needs to be validated in more cases nationally and subgroups of low prevalence and high-risk patients.
This study was conducted in a single center with multi-surgeon operations. Analysis of a single institution’s results has limitations and may not represent national and international practice and outcomes. Also the study was designed to collect data retrospectively, and was conducted on a small population with particular properties.

Conclusions

We consider that EuroSCORE II has a better AUC (area under the ROC curve) compared to the original EuroSCORE but both scales showed good discriminative capacity and goodness of fit on octogenarian patients undergoing isolated coronary artery bypass grafting.

Disclosure

The authors report no conflict of interest.

References

1. Roques F, Nashef SA, Michel P, Gauducheau E, de Vincentiis C, Baudet E, Cortina J, David M, Faichney A, Gabrielle F, Gams E, Harjula A, Jones MT, Pintor PP, Salamon R, Thulin L. Risk factors and outcome in European cardiac surgery: analysis of the EuroSCORE multinational database of 19030 patients. Eur J Cardiothorac Surg 1999; 15: 816-823.
2. Roques F, Michel P, Goldstone A, Nashef SAM. The logistic EuroSCORE. Eur Heart J 2003; 24: 881-882.
3. Siregar S, Groenwold RHH, de Heer F, Bots ML, van der Graaf Y, van Herwerden LA. Performance of the original EuroScore. Eur J Cardiothorac Surg 2012; 41: 746-754.
4. Poullis M, Fabri B, Pullan M, Chalmers J. Sampling time error in EuroSCORE II. Interact CardioVasc Thorac Surg 2012; 14: 640-641.
5. Nashef SA, Roques F, Sharples LD, Nilsson J, Smith C, Goldstone AR, Lockowandt U. EuroSCORE II. Eur J Cardiothorac Surg 2012; 41: 1-12.
6. EuroSCORE; available at http://www.euroscore.org/; accessed on June 10, 2013.
7. EuroSCORE II calculator; available at http://www.euroscore.org/calc.html; accessed on June 10, 2013.
8. Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: developing a prognostic model. BMJ 2009; 338: b604.
9. Carnero-Alcázar M, Silva Guisasola JA, Reguillo Lacruz FJ, Maroto Castellanos LC, Cobiella Carnicer J, Villagrán Medinilla E, Tejerina Sánchez T, Rodríguez Hernández JE. Validation of EuroSCORE II on a single-center 3800 patient cohort. Interact Cardiovasc Thorac Surg 2013; 16: 293-300.
10. Howell NJ, Head SJ, Freemantle N, van der Meulen TA, Senanayake E, Menon A, Kappetein AP, Pagano D. The new EuroSCORE II does not improve prediction of mortality in high-risk patients undergoing cardiac surgery: a collaborative analysis of two European centers. Eur J Cardiothorac Surg 2013; 44: 1006-1011.
11. Chalmers J, Pullan M, Fabri B, McShane J, Shaw M, Mediratta N, Poullis M. Validation of EuroSCORE II in a modern cohort of patients undergoing cardiac surgery. Eur J Cardiothorac Surg 2013; 43: 688-694.
12. Akgül A, Gürsoy M, Bakuy V, Bal Polat E, Kömürcü IG, Kavala AA, Türkyilmaz S, Cağlar IM, Tekdöş Y, Atay M, Altun Ş, Gulmaliyev C, Memmedov S. Comparison of standard Euroscore, logistic Euroscore and Euroscore II in prediction of early mortality following coronary artery bypass grafting. Anadolu Kardiyol Derg 2013; 13: 425-431 [Article in Turkish].
13. Akar AR, Kurtcephe M, Sener E, Alhan C, Durdu S, Kunt AG, Güvenir HA; Group for the Turkish Society of Cardiovascular Surgery and Turkish Ministry of Health. Validation of the EuroSCORE risk models in Turkish adult cardiac surgical population. Eur J Cardiothorac Surg 2011; 40: 730-735.
14. Findik O, Haberal I, Akyildiz M, Aksoy T, Erturk E, Zorman Y, Zengin M. A comparison of the sensitivity and specificity of the EuroSCORE, Cleveland, and CABDEAL risk stratification systems in the Turkish population. Turk Gogus Kalp Damar Cerrahisi Dergisi 2012; 20: 458-466.
15. Soyal T, Ozeren M, Kar M, Gokaslan G, Erdem H, Dolgun A, Sarigul A, Yucel E. Retrospective risk analysis of mortality and morbidity in coronary artery bypass graft reoperation candidates using EuroSCORE. Turk Gogus Kalp Damar Cer Derg 2004; 12: 241-245.
16. Kaplan M, Kut M.S, Cimen S, Demirtas MM. Applicability of EuroSCORE (European System for Cardiac Operative Risk Evaluation) risk scoring system in Turkish patients. Turk Gogus Kalp Damar Cer Derg 2003; 11: 147-158.