RGB-Z: mapping a sparse depth map to a high resolution RGB camera image

The video stream from a commercial RGB camera is mapped to a semi-synchronous video stream of a time-of-flight Z-depth camera to produce a composite RGB-Z video stream. The combined image is rendered to show both the color and depth effects of the scene. Each pixel of the composite image provides both RGB encoding and Z depth in real-world coordinates. The data is useful for both machine vision and consumer imaging applications. The RGB brightness data provides detailed image of the scene including textures, colors and ambient light information. The time-of-flight depth camera provides additional information about the relative location and size of objects independent of texture and ambient-light condition. The demo shows several 3D rendering technique including wire-frame and color shading.