Enabling genuine eye contact and accurate gaze in remote collaboration

Conventional telepresence systems allow remote users to see one another and interact with shared media and documents, but users cannot make eye contact, and gaze awareness with respect to shared media and documents is lost. In this paper we describe a remote collaboration system based on a see-through display to create an experience where local and remote users are seemingly separated only by a vertical sheet of glass. Users can see each other and media displayed on the shared surface. Face detectors on the local and remote video streams are used to introduce an offset in the video display so as to bring the local user´s face, the local camera, and the remote user´s face image into collinearity. This ensures that when the local user looks at the remote user´s image, the camera behind the see-through display captures an image with the `Mona Lisa effect´, where the eyes of an image appears to follow the viewer. Experiments show that our system is capable of capturing and delivering realistic, genuine eye contact as well as accurate gaze awareness with respect to shared media.