Multi-stream processing using context-independent and context-dependent hybrid systems

Multi-stream processing provides a successful approach to enhance the generalization capability of a recognizer and can, moreover, be combined with other robust techniques, such as spectral subtraction and/or robust features, HMM/MLP hybrid systems, and others. The question usually arises at which point the different streams are to be recombined, i.e. at the feature or at the probability level. Feature and probability combination are often seen as alternative approaches. We show here how a sensitive combination of both renders this decision obsolete and improves recognition as compared to each approach carried out on its own. The study has been carried out on the digits and numbers part of the Portuguese SPEECHDAT corpus. This corpus includes a large number of speakers and channel conditions and is, thus, well suited to test the described multi-stream systems under realistic conditions. Results are presented for both context-independent and context-dependent models used in an HMM/MLP hybrid system.