Visual echo cancellation in a projector-camera-whiteboard system

We propose to incorporate a whiteboard into a projector-camera system. The whiteboard serves as the writing surface (input) as well as the projecting surface (output). The ability to write and draw on top of computer-projected content opens up many new opportunities for real-time collaborations between people located on-site and remotely. Such applications inevitably require extracting handwritings from video images that contain both handwritings and the projected content. By analogy with echo cancellation in audio conferencing, we call this problem visual echo cancellation. This paper presents one approach to accomplish the task. Our visual echo cancellation algorithm estimates the incident light and derives the surface albedo based on both incident light and reflection. By estimating the albedo, we can extract the writings and recover their colors. Our approach includes two basic components of projector-camera systems: geometric calibration and color calibration. The first one solves the mapping between the position in the camera view and the position in the projector screen, while the second one solves the mapping between the actual color of the projected content and that seen by the camera.