3D Target Scale Estimation and Motion Segmentation for Size Preserving Tracking in PTZ Video

In size preserving video tracking, the camera’s focal length (zoom) is adjusted automatically to compensate for the changes in the target’s image size caused by the relative motion between the camera and the target. The accurate estimation of these changes is paramount to the system performance. The existing method of choice for real-time target scale estimation applies structure from motion (SFM) based on the weak perspective projection model [1]. We design a target scale estimation algorithm with linear solution based on the more advanced paraperspective projection model. Another key problem in SFM based algorithms is the separation between foreground and background features (image corners), especially when composite camera and target motions are involved. This paper also addresses a fast foreground/background separation algorithm, the affine shape method. The resulting segmentation automatically adapts to the target’s 3D geometry and motion. Experimental results illustrate the effectiveness of the proposed scale estimation and segmentation algorithms in tracking translating and rotating objects with a PTZ camera while preserving their sizes.