Preliminary Validation of a Method to Measure Information Value in Clinical Documentation

Well-established electronic medical record (EMR) systems accumulate large volumes of documentation (notes, reports and summaries) of highly variable purpose, scope and quality. To assist treatment decisions, caregivers review electronic charts, but do so in a constrained display space. In this study we investigate whether tf·idf, a computable relevance statistic, exhibits a valid correspondence with user judgments of EMR document quality and usefulness. 110 administrators, nurses, and practitioners rated progress notes on a user-perceived EMR document quality scale. Quality ratings were compared to tf·idf computed for each document evaluated. Over the entire group, several quality dimensions significantly correlated with tf·idf, but practitioner ratings were most strongly correlated. Enhancing tf.idf via stemming and stopword preprocessing produced stronger correlations than the baseline implementation. Tf·idf relevance shows promise as a valid proxy of practitioner-perceived document quality, with the potential to aid developing a relevance filter that allows EMR users to review documents more efficiently.