Rigid body segmentation and shape description from dense optical flow under weak perspective

We present an algorithm for identifying and tracking independently moving rigid objects from optical flow. The proposed method uses the fact that each distinct object has a unique epipolar constraint associated with its motion. This is in contrast to using local optical flow information for segmentation. Thus motion discontinuities based on self-occlusion are distinguished from those due to separate objects. The use of epipolar geometry allows for the determination of individual motion parameters for each object as well as the recovery of relative depth for each point on the object. The segmentation problem is formulated as a scene partitioning problem and a statistic-based algorithm which uses only nearest neighbor interactions and a finite number of iterations is developed. A Kalman filter based approach is used for tracking motion parameters with time. The algorithm assumes an affine camera where perspective effects are limited to changes in overall scale. No camera calibration parameters are required