Spectral covariance in prior distributions of non-negative matrix factorization based speech separation

This paper proposes an algorithm for modeling the covariance of the spectrum in the prior distributions of non-negative matrix factorization (NMF) based sound source separation. Supervised NMF estimates a set of spectrum basis vectors for each source, and then represents a mixture signal using them. When the exact characteristics of the sources are not known in advance, it is advantageous to train prior distributions of spectra instead of fixed spectra. Since the frequency bands in natural sound sources are strongly correlated, we model the distributions with full-covariance Gaussian distributions. Algorithms for training and applying the distributions are presented. The proposed methods produce better separation quality that the reference methods. Demonstration signals are available at www.cs.tut.fi/~tuomasv.