Score-Level Fusion of Face and Voice Using Particle Swarm Optimization and Belief Functions

We propose an efficient particle swarm optimization (PSO) technique that weights the belief assignments of voice and face classifiers. The belief assignment is computed from the score of each modality using Denœux and Appriou models. The fusion of the weighted belief assignments is then performed by using Dempster-Shafer (DS) theory and proportional conflict redistribution (PCR5) combination rules. Experiments are conducted on the scores of XM2VTS and BANCA multimodal databases. A comparative study is achieved using our method, several existing PSO-based fusion techniques, DS theory, and PCR5 combination rules. Experimental studies show that the proposed approach improves the error equal rate compared with the well-established methods on BANCA multimodal database since it contains controlled, degraded, and adverse data.