Same frame rate IR to enhance visible video conference lighting

Professional studios produce good lighting by using well-positioned, bright studio lights to illuminate the subjects, while casual desktop video conferencing often suffers from uneven and unreliable lighting. Using bright lights is not an option - they annoy and distract non-professional subjects. In this work, we illuminate the subjects using invisible near infrared (IR) lights and capture the subject simultaneously in IR and visible light. The IR video is used as a reference to enhance the visible video. The advantages of IR illumination for video conferencing have previously been shown, but the prior works require 4 to 8 times higher IR frame rate, with concomitant complexity and bandwidth increases. The enhancement developed here performs well with low IR frame rates equal to the visible frame rate. The technique works well even when uneven lighting causes bright visible light pixels to clip. After describing the technique, comparisons with prior methods are provided.