Principal component analysis with missing data and its application to object modeling

Observation-based modeling can reduce the cost and effort of model constructions for tasks such as virtual reality environment. Object modeling from a sequence of range images has been formulated as a problem of principal component analysis with missing data (PCAMD), which can be generalized as a weighted least square (WLS) minimization problem. After all visible regions appeared over the whole sequence are segmented and tracked, a normal measurement matrix of surface normals and a distance measurement matrix of normal distances to the origin are constructed respectively. These two measurement matrices, with possibly many missing elements due to occlusion and mismatching, enable us to formulate multiple view merging as a combination of two WLS problems. The solution to the first WLS problem, which employs the quaternion representation of the rotation matrix, yields surface normals and rotation matrices. Subsequently the normal distances and translation vectors are computed by solving the second WLS problem. Experiments using synthetic data and real range images show that our approach is robust against noise and mismatch because it produces a statistically optimal object model by making use of redundancy from multiple views. A toy house model from a sequence of real range images is presented