Identification of cardio-pulmonary resuscitation (CPR) scenes in video simulating medical crises

Medical simulations, where uncommon clinical situations can be replicated, have proved to provide a more comprehensive training. Simulations involve the use of patient simulators, which are lifelike mannequins. After each session, the physician must manually review and annotate the recordings and then debrief the trainees. The physician responsible for the simulation has recorded over 100 videos, and has realized that this process can be tedious and that retrieval of specific video segments should be automated. In this paper, we propose a machine learning based approach to detect and classify scenes that involve CPR scenes. It provides answers to queries that are of interest to the physician supervising the training sessions such as: show me all the scenes that have a CPR action from a given video simulation training, or ”retrieve time specific data about such critical events as elapsed time between failure of circulation and the initiation of CPR, a measure clearly association with patient outcome”. Our approach consists of two main steps: The first one segments the video into shots, selects one keyframe for each shot, and identifies regions with skin-like colors in each keyframe. Each skin region is then represented by sequence of observations that encode its motion in the different frames within the shot boundary. The second step consists of using motion-based features in a discrete HMM classifier to identify the skin-like regions that involve CPR activities. The proposed approach was evaluated and validated using a 30 min video simulation session. We show that the HMM classifier can detect 80% of the CPR scenes with only 10% false alarms.