A pyramid-based front-end processor for dynamic vision applications

Real-time vision tasks such as autonomous driving require prodigious computing power yet practical vision systems need to be compact and low cost. I suggest that such systems can be partitioned into two computing stages, for "front-end processing" and "high-level interpretation," respectively, and that each of these stages can be implemented as a single integrated circuit or a small number of such circuits. The two stages differ in data representation and computing architecture: The front-end stage operates on sampled image data and its computations are performed on a processor optimized for signal level processing. The high-level stage operates on abstract and symbolic image data and its computations ate performed on a general-purpose microprocessor. In this paper I describe a "segmented pipeline" architecture for front-end processing and a chip level processor implementation. This vision front-end processor is designed to support early vision functions, such as feature enhancement and motion and stereo analysis, for a broad range of dynamic vision applications. The approach makes systematic use of a multiresolution pyramid framework to achieve high computational efficiency, robustness, and precision.