Moving camera human activity localization and recognition with motionplanes and multiple homographies

Camera motion is often present in videos, due to shaking, jitter or ego-motion. This creates the need for a reliable motion compensation algorithm that will lead to accurate spatial activity localization and activity recognition. We detect background areas using SLIC superpixels and multiple homographies, and introduce motionplanes, defined as background areas that are undergoing diverse camera motions. Dense trajectories are sampled from these regions and histograms of appearance and motion are extracted to build a representation scheme to be used for activity recognition. Experiments on two publicly available unconstrained video datasets of human activities recorded with a moving camera show that our algorithm not only outperforms most activity localization state-of-the-art algorithms but also leads to high activity recognition accuracy rates.