The generalization, optimization, and information-theoretic justification of filter-based and autocovariance-based motion estimation

We discuss the theoretical foundations of measuring motion in video data, and relate this strongly to statistical estimation theory. A very general class of motion estimation methods is characterized by determining second order moments of filter bank outputs. These moments are represented in tensors, and motion estimation boils down to analyzing their eigensystems. An alternative approach is to directly estimate and analyze the autocorrelation of the given signal. We provide motivation for developing these approaches further towards directional entropy rate criteria rather than rely on conventional directional smoothness criteria. This pa- per emphasizes that prior knowledge on the video signal (e.g. spatial autocovariance, distribution of expected motion speed, noise spectrum,...) should be integrated into the motion estimation procedure. Relations between different classes of motion algorithms (differential, tensor-based, steerable filters...) are discussed and perspectives for a unification and enhancement of such procedures are presented.