Discriminative algorithm for compacting mixture models with application to language recognition

In this paper we explore a discriminative algorithm for compacting large order mixture models. Several studies investigated efficient algorithms for finding a reduced-order model that best approximates a referenced model using only the original mixture parameters. Recently, a discriminative approach named maximum correct association (MCA) was introduced to efficiently construct a set of compact models for improved classification. In this paper we suggest a two stage procedure that applies the MCA algorithm after initially obtaining a compact model through the variational-EM method (which is a non-discriminative algorithm). The proposed method is validated in a language recognition task where large order mixture models are compacted into low order models. Experiments showed that the MCA-refined models performed consistently better than reduced models derived with the non-discriminative methods including boosting performance over the standard maximum-likelihood trained from the original data.