Subjective versus statistical model assessment of mortality risk in open heart surgical procedures

Background. The aim of this study was to compare the predictive accuracy for open heart surgical mortality between a statistical model based on collection of clinical data and surgeons’ subjective risk assessment. Methods. Predictive discrimination of both risk assessments (surgeons’ and model) was compared through the area under the receiver operating characteristic curve. Logistic regression analysis was used to assess the relation between surgeons’ and model predictions to actual outcomes. Calibration of the subjective estimates was evaluated with a χ2 test. Results. Overall, the area under the receiver operating characteristic curve was 0.76 for the statistical model and 0.70 for the subjective assessment. Logistic regression analysis showed that the statistical model remained significant after accounting for the subjective assessment. Calibration of subjective mortality predictions was poor. Conclusions. Surgeons’ risk assessment tends to cluster in the middle ranges of risk. Subjective assessment seems accurate in identifying the two extremes of risk but is inaccurate for intermediate risk levels. A multivariate statistical model improves the accuracy of subjective predictions.