Machine vision using combined frame-based and event-based vision sensor

Conventional synchronous imaging sensor provides frame-based video with a relatively high degree of temporal redundancy. On the other hand, activity-driven, event-based imaging sensor provides low resolution, monochromatic video feeds with low latency. This paper aims to integrate the output from both camera systems to leverage on the strengths of both imaging sensors. We describe and demonstrate various video processing applications achieved using the combined camera system. The applications include a novel video-compression scheme, foveated imaging on the moving objects, object tracking and velocity estimation. All demonstrations are achieved through the integration of data outputs from the Dynamic Vision Sensor (DVS128) and conventional frame-based QVGA (320×240) PS3-Eye webcam, in the jAER software.