Spatially varying radiometric calibration for camera-display messaging

Modern society has ubiquitous electronic displays including billboards, signage and kiosks. The concurrent prevalence of handheld cameras creates a novel opportunity to use cameras and displays as communication channels. The electronic display in this channel serves a twofold purpose: to display an image to humans while simultaneously transmitting hidden bits for decoding by a camera. Unlike standard digital watermarking, the message recovery in camera-display systems requires physics-based modeling of image formation in order to optically communicate hidden messages in real world scenes. By modeling the photometry of the system using a camera-display transfer function (CDTF), we show that this function depends on camera pose and varies spatially over the display.We devise a radiometric calibration to handle the nonlinearities of both the display and the camera, and we use this method for recovering video messages hidden within display images. Results are for 9 different display-camera systems for messages with 4500 bits. Message accuracy improves significantly with calibration and we achieve accuracy near 99% in our experiments, independent of the type of camera or display used.