Decentralized Learning of a Gaussian Mixture with Variational Bayes-based Aggregation

A distributed statistical estimation technique is presented, with its motivation from, and application to, multimedia content-based indexing. The contribution is a scheme for estimating a multivariate probability density, in the case where this density takes the form of a Gaussian mixture model. They have of broad applicability for multimedia feature modelling. Assuming independently estimated mixtures, we propagate their parameters in a decentralized fashion (gossip) in a network, and aggregate GMMs from connected nodes, to improve estimation. As an improvement through a change of principle over previous work, aggregation is achieved through Bayesian modelling of the GMM component grouping problem and solved using a variational Bayes technique, applied at component level. This determines, through a single, low-cost yet accurate process, assignments of components that should be aggregated and the number of components in the mixture after aggregation. Because only model parameters are exchanged on the network, computational and network load remain very moderate. The scheme is demonstrated on the task of speaker recognition.