Title :
Facial feature tracking and pose estimation in video sequences by factorial coding of the low-dimensional entropy manifolds due to the partial symmetries of faces
Author :
Mandel, E.D. ; Penev, P.S.
Author_Institution :
Lab. of Comput. Sci., Rockefeller Univ., New York, NY
Abstract :
In a number of practical scenarios, such as video conferencing and visual human/computer interaction, objects that belong to a well defined class are segmented, normalized, and encoded, after which they are stored and/or transmitted, and subsequently reconstructed. The Karhunen-Loeve transform (KLT) optimally concentrates the signal power in a relatively small number of uncorrelated coefficients. Nevertheless, it implicitly assumes a multidimensional Gaussian probability model, which is typically not correct. Here we show that, in the context of video sequences of human heads, the segmentation and normalization steps result in partial symmetries which force the KLT coefficients to lie close to low-dimensional manifolds in suitably chosen high-dimensional KLT subspaces. We show how this fact can be used to track the faces robustly, and to estimate their pose. We use vector quantization to discover those manifolds, and to build a factorial code that has a substantially lower dimensionality than KLT
Keywords :
Karhunen-Loeve transforms; entropy; feature extraction; image reconstruction; image segmentation; image sequences; parameter estimation; teleconferencing; tracking; transform coding; vector quantisation; video coding; KLT coefficients; Karhunen-Loeve transform; encoded object; facial feature tracking; factorial coding; human heads; low-dimensional entropy manifolds; multidimensional Gaussian probability model; normalized object; object reconstruction; object segmentation; partial symmetries; pose estimation; signal power; uncorrelated coefficients; vector quantization; video conferencing; video sequences; visual human/computer interaction; Entropy; Face detection; Facial features; Humans; Karhunen-Loeve transforms; Parameter estimation; Robustness; Statistics; Video sequences; Videoconference;
Conference_Titel :
Acoustics, Speech, and Signal Processing, 2000. ICASSP '00. Proceedings. 2000 IEEE International Conference on
Conference_Location :
Istanbul
Print_ISBN :
0-7803-6293-4
DOI :
10.1109/ICASSP.2000.859311