Three-dimensional video generation using foreground separation and disocclusion detection

In this paper, we present a multi-depth camera system for 3-D video contents generation. We combine five video cameras and three TOF depth cameras to capture a 3-D scene in real time. By taking advantages of both active and passive sensor based depth acquisition methods, we can obtain accurate multi-view depth maps. After several preprocessing operations, each depth map is warped to the video camera position and used as initial depth values for refinement. To reduce mismatched depth values along object boundaries, we separate the foreground and background regions using moving objects detection and extract occlusion and disocclusion areas with the initial depth information. By applying the stereo matching method, we can improve multi-view depth maps. Intermediate view image sequences are then synthesized using both the multi-view color and depth information.