A new frame interpolation scheme for talking head sequences

Author

Chen, Tsuhan ; Wang, Yao ; Graf, Hans Peter ; Swain, Cassandra

Author_Institution

AT&T Bell Labs., Holmdel, NJ, USA

Volume

2

fYear

1995

fDate

23-26 Oct 1995

Firstpage

591

Abstract

A video codec typically-skips frames to satisfy the bit rate constraint. This results in a jerky motion and a loss of lip synchronization in sequences of talking persons. We propose a frame interpolation technique to solve these problems. The main components of this technique include: foreground/background segmentation, mesh-based frame interpolation, and image analysis/synthesis for mouth movements. This technique generates smooth head motion and renders lip motion that is synchronized with the voice of the person

Keywords

image segmentation; image sequences; interpolation; motion estimation; synchronisation; video codecs; bit rate constraint; foreground/background segmentation; frame interpolation; image analysis/synthesis; lip synchronization loss; mesh-based frame interpolation; mouth movements; smooth head motion; synchronised lip motion; talking head sequences; video codec; voice; Bandwidth; Bit rate; Decoding; Head; Image motion analysis; Image sequence analysis; Interpolation; Mouth; Rendering (computer graphics); Video codecs;

fLanguage

English

Publisher

ieee

Conference_Titel

Image Processing, 1995. Proceedings., International Conference on

Conference_Location

Washington, DC

Print_ISBN

0-8186-7310-9

Type

conf

DOI

10.1109/ICIP.1995.537548

Filename

537548