Relative pose problem for non-overlapping surveillance cameras with known gravity vector

We present a method for estimating the relative pose of two calibrated or uncalibrated non-overlapping surveillance cameras from observing a moving object. We show how to tackle the problem of missing point correspondences heavily required by SfM pipelines and how to go beyond this basic paradigm. We relax the non-linear nature of the problem by accepting two assumptions which surveillance scenarios offer, i.e. the presence of a moving object and easily estimable gravity vector. By those assumptions we cast the problem as a Quadratic Eigenvalue Problem offering an elegant way of treating nonlinear monomials and delivering a quasi closed-form solution as a reliable starting point for a further bundle adjustment. We are the first to bring the closed form solution to such a very practical problem arising in video surveillance. Results in different camera setups demonstrate the feasibility of the approach.