An audit method suited for decision support systems for clinical environment

We present a novel on-line method to audit predictive models using a Bayesian perspective. The auditing model has been specifically designed for decision support systems (DSSs) suited for clinical environments. Taking as starting point the proposed diagnosis supplied by the clinician, our method compares and evaluates the predictive skills of those models able to answer to that diagnosis. Our approach consists in calculating the posterior odds of a model through the composition of three different odds: prior, static and dynamic. To do so, our method estimates the posterior odds from the cases that the comparing models had in common during the design stage and from the cases already viewed by the DSS after deployment in the clinical site. In addition, if an ontology of the classes is available, our method can audit models answering related questions, which offers a reinforcement to the decisions the user already took and gives orientation on further diagnostic steps. The main technical novelty of this approach lies in the design of an audit model adapted to suit the decision workflow of a clinical environment. The audit model allows deciding which is the classifier that best suits each particular case under evaluation and allows the detection of possible misbehaviours due to population differences or systematic errors in the clinical site. We show the efficacy of our method for the problem of brain tumour diagnosis with magnetic resonance spectroscopy.