Registration of camera captured documents under non-rigid deformation

Document registration is a problem where the image of a template document whose layout is known is registered with a test document image. Given the registration parameters, layout of the template image is superimposed on the test document. Registration algorithms have been popular in applications, such as forms processing where the superimposed layout is used to extract relevant fields. Prior art has been designed to work with scanned documents under affine transformation. We find that the proliferation of camera captured images makes it necessary to address camera noise such as non-uniform lighting, clutter, and highly variable scale/resolution. The absence of a scan bed also leads to challenging non-rigid deformations being seen in paper images. Prior approaches in point pattern based registration like RANdom SAmple Consensus (RANSAC), and Thin Plate Spline-Robust Point Matching (TPS-RPM)form the basis of our work. We propose enhancements to these methods to enable registration of cell phone and camera captured documents under non-rigid transformations. We embed three novel aspects into the framework: (i) histogram based uniformly transformed correspondence estimation, (ii) clustering of points located near the regions of interest (ROI) to select only close by regions for matching, (iii) validation of the registration in RANSAC and TPS-RPM algorithms for non-rigid registration. We consider Scale Invariant Feature Transform (SIFT) and Speeded-Up Robust Features (SURF) as our features. Results are reported as comparing prior art with our method on a dataset that will be made publicly available.