Algorithms and systems for modeling moving scenes

In this paper I describe the application of machine-vision techniques to video coding in order to create what my research group calls object-oriented television, where moving scenes are represented in terms of objects (as recovered by analysis methods). Beyond data compactness, such a representation offers the ability to add new degrees of freedom to content creation and display. I discuss some of the scene analysis problems (particularly 2-D and 3-D model-fitting and object segmentation) and the algorithmic approaches my group has taken to solve them; suggest computational strategies for compact, powerful, programmable decoding hardware (particularly stream-based computing combined with automatic resource management); and demonstrate some of the applications we have developed.