Relax frame independence assumption for standard hmms by state dependent auto-regressive feature models

We propose a new type of frame-based hidden Markov models (HMMs), in which a sequence of observations are generated using state-dependent autoregressive feature models. Based on this correlation model, it can be proved that expressing the probability of a sequence of observations as a product of probabilities of decorrelated individual observations does not require the assumption of frame independence. Under the maximum likelihood (ML) criteria, we also derived re-estimation formulae for the parameters (mean vectors, covariance matrix, and diagonal regression matrix) of the new HMMs using an expectation maximization (EM) algorithm. From the formulae, it is interesting to see that the new HMMs have extended the standard HMMs by relaxing the frame independence limitation. The initial experiment conducted on WSJ20K task shows an encouraging performance improvement with only 117 additional parameters in all