Speech Recognition for a Humanoid with Motor Noise Utilizing Missing Feature Theory

Automatic speech recognition (ASR) is essential for a human-humanoid communication. One of the main problems with ASR is that a humanoid inevitably generates motor noises. These noises are easily captured by the humanoid´s microphones because the noise sources are closer to the microphones than the target speech source. Thus, the signal-to-noise ratio (SNR) of input speech becomes quite low (sometimes less than 0 dB). However, it is possible to estimate these noises by using information about the humanoid´s own motions and gestures. In this paper we propose a method to improve ASR for a humanoid with motor noises by utilizing the information about the humanoid´s motions/gestures. The method consists of psychologically-inspired noise suppression and missing-feature-theory-based ASR (MFT-ASR). The proposed noise suppression technique adds white noise after noise suppression which does not improve SNR, but it is suitable for MFT-ASR. This is inspired by the fact that noise addition sometimes helps human perception as described in Gestalt psychology. MFT-ASR improves ASR by masking unreliable acoustic features in the input sound. The information obtained on motion/gesture is used for estimating reliability of acoustic features in MFT-ASR. We evaluated the proposed method with noisy speech recorded by Honda ASIMO in a room with reverberation. The noise data contained 32 kinds of noises: motor noises without motions, gesture noises, walking noises, and so on. The experimental results show that the proposed method outperforms the conventional multi-condition training technique.