Identification of cardio-pulmonary resuscitation (CPR) scenes in medical simulation videos using spatio-temporal gradient orientations

In this work, we present the application of spatio-temporal three dimensional gradients to detect and classify scenes that involve localized actions like CPR in medical simulation videos. Medical simulations provide a more feasible and comprehensive training to avoid human errors during uncommon clinical situations. Life-like mannequins that can simulate emergency patient conditions are used for this purpose. The physician responsible for these simulations, records each session after which, he manually reviews and annotates the recordings, and then debriefs the trainees. With the increasing number of video recordings, automatic retrieval of specific video segments became necessary. Here we propose an automatic scene retrieval system which can detect and classify scenes into CPR and non-CPR scenes. We use a simple linear SVM classifier for the classification. It provides answers to queries that are of interest to the physician supervising the training sessions such as: "show me all the scenes that have a CPR action from a given video simulation training", or "retrieve time specific data about such critical events as elapsed time between failure of circulation and the initiation of CPR, a measure clearly associated with patient outcome". Our system has the following two main advantages over other existing systems: (1) It does not require video shot segmentation (2) It uses one algorithm and need not be coupled with any other algorithms like image segmentation, skin detection etc. The proposed approach was evaluated and validated using ~30 min video simulation sessions. We show that the proposed approach out performs the state of the art by being able to correctly classify the CPR scenes with an error rate of only 10%.