Stream derivation and clustering scheme for subspace distribution clustering hidden Markov model

Bocchieri and Mak (Proc. Eurospeech, vol. 1, p. 107-10, 1997) introduced a novel subspace distribution clustering hidden Markov model (SDCHMM) as an approximation to a continuous-density HMM (CDHMM). Deriving SDCHMMs from CDHMMs requires a definition of multiple streams and a Gaussian clustering scheme. Previously, we have tried 4 and 13 streams, which are common but ad hoc choices. In this paper, we present a simple and coherent definition for streams of any dimension: the streams comprise the most correlated features. The new definition is shown to give better performance in two speech recognition tasks. The clustering scheme of Bocchieri and Mak is an O(n²) algorithm which can be slow when the number of Gaussians in the original CDHMMs is large. Now, we have devised a modified k-means clustering scheme using the Bhattacharyya distance as the distance measure between Gaussian clusters. Not only is the new clustering scheme faster but, when combined with the new stream definitions, we now obtain SDCHMMs which perform at least as well as the original CDHMMs (with better results in some cases)