Mid-level vision: new directions in vision and video

Human vision, machine vision, and image coding, all have to deal with the problem of finding representations that are useful and efficient. The best-known techniques are based on low-level processing, using signal processing concepts such as filters, transforms, and simple non-linearities. Low-level concepts are at the heart of standard vision systems for computing optic flow, texture, etc. Low-level image coding techniques include DCTs, pyramids, wavelets, etc. To advance to a new generation of image coding architectures one must work with new image representations that involve such concepts as surfaces, lighting, transparency, etc. These representations fall in the domain of “mid-level” vision. By representing images with these more sophisticated vocabularies one can increase the flexibility and efficiency of the vision and image coding systems. The authors decompose an image sequence into a set of overlapping layers, rather like the “cels” used by a traditional animator. These layers are ordered in depth, sliding over one another and being combined according to the rules of transparency and occlusion. For some test sequences the authors achieve data compression far better than is possible with standard techniques such as MPEG