Methodological Approaches to Measuring the Effects of Implementation of Health Information Technology (HIT)

The research design, evaluation methodology, and statistical analysis of the clinical efficacy of healthcare information technology (HIT) implementation can be a challenging task. Much of the research to date involves weakly designed studies. We discuss some of the more rudimentary experimental designs and analytical approaches that have typically been used. Approaches to strengthen a research design include: adding a matched control unit or hospital; using multiple observations before and after the HIT implementation; making observations across a number of hospitals that are implementing the same HIT application; comparing the changes in these hospitals to a matched set that have not yet implemented the HIT application; and applying statistical approaches that permit changes in trends over time to be examined. Here we report our use of linear piecewise spline mixed effects models and compare our models to other methodological approaches that could be used for this evaluation.