Multimodal person authentication system using features of utterance

In this paper, we propose a multimodal biometrics authentication method using features of an utterance. The proposed authentication method authenticates persons using image and voice signals. Hence, the proposed method can be realized with only a camera and microphone to extract the lip area and voice without the special equipment used in other personal authentication methods and can easily change the registration data. Moreover, the proposed authentication method can provide a key function to the registered phrase of the utterance. In the proposed method, the edges and texture in the mouth are used as image features, and pitch and spectrum envelope are used as voice features. Authentication is realized by classifiers generated by AdaBoost, classifiers are generated for the voice- and image-processing parts. Moreover, each classifier is weighted according to the corresponding confidence and then the final authentication score is calculated. Hence, the proposed method can provide valid authentication results in various environments. Experimental results demonstrate that multimodal processing in the proposed method is more effective than monomodal (only image or voice) processing.