Evaluation of a knowledge-based decision-support system for ventilator therapy management

Evaluation of knowledge-based systems differs from that of conventional systems in terms of verification and validation techniques. Furthermore, evaluating medical decision-support systems is difficult because the field is thus far comparatively unexplored. This paper presents an evaluation of a medical knowledge-based system called VentEx that supports decision-making in the management of ventilator therapy. Real patient data from 1300 hours of patient care involving 12 patients with 6 diagnoses are used to validate the knowledge base. The results range from 4.5% to 15.6% disagreement between the setting recommendations produced by VentEx and a gold standard, and 22.2% disagreement for recommendations for weaning. A comparison between the standard and two physicians showed that VentEx produced advice of the same quality as the physicians.