Validated physiologic scoring systems are inadequate for predicting in-hospital mortality among critically ill emergency department patients

Study objectives: The New Simplified Acute Physiology Score (SAPS II), Morbidity Probability Models (MPM II), and Logistic Organ Dysfunction System (LODS) have all demonstrated high accuracy (>0.80 for area under the receiver operating characteristic [ROC] curve) for predicting mortality in ICU populations. We examined the prognostic utility of these 3 instruments in a cohort of prospectively studied critically ill emergency department (ED) patients. Methods: We performed a secondary analysis of 190 patients enrolled in a randomized controlled trial from January 2002 to October 2003. Inclusion criteria were age older than 18 years, no history of trauma, arterial hypotension (systolic blood pressure <100 mm Hg), and presence of a sign and a symptom of shock by consensus of 2 independent physicians who used printed criteria. All patients who were admitted to the ICU were included in this preplanned observational subanalysis. Data were prospectively collected and stored in an electronic database and were composed of historical, physiologic, and diagnostic information available in the ED to the emergency physician at the patient encounter. Using published formulae for each of the 3 instruments, the raw scores and probability of inhospital death were calculated post hoc for each patient. The main outcome was inhospital mortality. The area under the ROC curve was used to evaluate prognostic accuracy. Results: Ninety-one (47%) of 190 patients were admitted to the ICU and analyzed in this study. The mean age was 56±16 years, 42% were female patients, the mean initial systolic blood pressure was 84±13 mm Hg, and the average length of stay in the ED was 4.2±2.0 hours. The inhospital mortality rate was 21%. The average SAPS II score was 40±14 points, the average LODS score was 5±3 points, and the average MPM II (using MPM0) score was −1.058±1.242. The area under the ROC curve for calculated probability of inhospital mortality for SAPS II was 0.72 (95% confidence interval [CI] 0.57 to 0.87), for MPM0 was 0.69 (95% CI 0.54 to 0.84), and for LODS was 0.60 (95% CI 0.45 to 0.76). Conclusion: Using variables available in the ED, 3 previously validated ICU scoring systems (SAPS II, MPM II, and LODS) demonstrated moderate accuracy. The lower limits of the CIs do not exclude poor prognostic accuracy (area under the curve <0.60) for any system. These results suggest inadequate accuracy of these ICU scoring systems for prognostic purposes among our cohort of critically ill adult ED patients. These results highlight the need to develop a scoring system using information available in the ED for risk stratification of critically ill patients.