Synchronizing disparate video streams from laparoscopic operations in simulation-based surgical training

In this paper, we propose a novel approach for synchronizing multiple videos captured from common laparoscopic operations in simulation-based surgical training. The disparate video sources include two hand-view sequences and one tool-view sequence that does not contain any visual overlap with the hand views. The synchronization of the video is essential for further visual analysis tasks. To the best of our knowledge, there is no prior work dealing with synchronization of completely different visual streams capturing different aspects of the same physical event. In the proposed approach, histograms of dominant motion (HoDM) are extracted and used as features for each frame. Multi-view sequence correlation (MSC), computed as accumulated products of pairwise correlations of HoDM magnitudes and co-occurrence rate of pairwise HoDM orientation patterns, is proposed for ranking possible configurations of temporal alignment. The final relative shifts for synchronizing the videos are determined by maximizing both the overlap length of all sequences and the MSC scores through a coarse-to-fine search procedure. Experiments were performed on 41 groups of videos of two laparoscopic operations, and the performance was compared to start-of-the-art method, demonstrating the effectiveness of the proposed approach.