Joint motion and geometry modeling with quad-tree leaf merging

Quad-tree structures are often used to model motion between frames of a video sequence. However, a fundamental limitation of the quad-tree structure is that it can only capture horizontal and vertical edge discontinuities at dyadically related locations. To address this limitation recent work has focused on the introduction of geometry information to nodes of tree structured motion representations. Recent research into quad-tree structures have also demonstrated the importance of leaf merging. In this paper we create a general quadtree structure, well suited to jointly representing geometry and motion information. We then explore tree pruning and leaf merging, where the geometry information is treated as an equal partner with motion. To achieve an efficient joint representation we improve on the estimation algorithm that detects boundary geometry and introduce polynomial motion models. Experimental results show that the approach taken in this paper provides significant improvement over previous quad-tree based motion representation schemes.