Sensor-Assisted Video Encoding for Mobile Devices in Real-World Environments

In this paper, we present a comprehensive study on sensor-assisted video encoding (SaVE) schemes for video capturing on mobile devices in real-world environments. Our purpose is to reduce the computational complexity of video encoding by leveraging sensors that are increasingly available on mobile devices, e.g., accelerometers and digital compasses. Motion estimation is a key component of video encoding. In this paper, SaVE calculates the rotational movement of a camera (on mobile devices) and then infers the global motion in the camera imager. SaVE subsequently employs the estimated global motion as predictors to simplify motion estimation algorithms for state-of-the-art H.264/AVC video coding. We have constructed a prototype of SaVE and evaluated its performance with a pair of accelerometers, a digital compass, and their combination. Our experimental results show that SaVE can significantly reduce the computations of motion estimation while achieving equal or better video quality. Our results also show that SaVE has a strong noise-resistant capability. Therefore, it can be practically employed in real-world environments.