A design framework for evaluating changes in clinical communication systems

Poor communication in critical care settings has the potential to adversely impact patient outcomes and clinical team performance. Hence, improvements to clinical communication systems are continually sought. However, evaluating these modifications and assessing the risk of degraded performance are complex issues, which are typically addressed qualitatively. These approaches typically examine individual perceptions and can fail to capture the actual performance of the modified processes in the communication system. In order to avoid these issues and effectively analyze the changes in a given system, a Monte Carlo simulation methodology is proposed using data extracted from Electronic Medical Records (EMR) compiled by the Huron Consulting Group to precisely measure modified system performance. We couple a risk analysis, using control charts, with a temporal distribution analysis to determine whether to halt using a new system due to failures or determine if more observations are needed to make an adequate assessment. Specifically, we use EMR data to apply this methodology and show the tradeoff between sample size and the likelihood that the data converges to the current state of communication. This methodology will enable researchers to expedite the assessment of the impact of a technology, which is critical in a complex system such as the hospital.