Real-time realistic 3D facial expression cloning for smart TV

This paper suggests a novel technology that can clone a user´s facial expression to his avatar in 3D virtual worlds realistically using only one color camera on a smart TV. To do this, the user´s 3D head movement information and 3D position of facial feature points are needed in real time. We propose two novel approaches to achieve this. Firstly, we use a personalized 3D and 2D facial expression model to deal with head movement and various expressions. Secondly, we use a facial muscle model to generate natural motion of facial feature points located in cheeks and forehead which are difficult to be tracked using a camera. Experimental results demonstrate that the proposed method would be an efficient technique to perform realistic 3D facial expression cloning.