A reliability guided sensor fusion model for optimal weighting in multimodal systems

For intelligent sensory systems, it is highly desirable to develop assessment methods that can continuously evaluate the reliability of potential sensory strategies taking into consideration changes in observation conditions. This relies on measuring a set of complementary features from multiple sensors and combining these features in an "intelligent" way that maximizes information gather and minimizes the impact of noise coming from the individual sensors. In this work, we formulate a statistical assessment method for estimating the reliability of observation conditions and propose an optimal mapping into weighting measures using genetic algorithms. Our approach is particularly beneficial for multimodal systems such as audio-visual speech recognition (AVSR).