Real-time panorama canvas of natural images

This paper deals with a real-time panorama algorithm for mobile camera systems. The proposed system generates the panoramic images and shows the intermediate results simultaneously on the mobile display. The proposed panorama system consists of feature point extraction, feature tracking, rotation matrix estimation, and image warping onto cylindrical surface. Feature points are extracted by fast Hessian detector in the low resolution and corner detector in the high resolution. Then, the feature points are tracked by matching between input frame and synthesized panorama image. The camera motion is modeled as a rotation matrix which is estimated using the tracked feature points. For realtime operation of panoramic image synthesis, we propose a method to estimate the rotation matrix using non-iterative least square method. We project the feature points of image onto the unit sphere surface in the world coordinates. This enables us to estimate the parameters of rotation matrix as the linear and non-iterative problem. Finally, we project the input frames onto panorama surface using the rotation matrix. We also implement a real-time display interface to show the intermediate panorama results while panorama image is being generated. Thus, the proposed panorama system paints the panorama canvas with real images. According to the experiments in the mobile systems such as mobile phone and tablet PC, the proposed system works well to generate the panoramic images in real-time.