A Markov Chain Model with High-Order Hidden Process and Mixture Transition Distribution

The hidden Markov model (HMM) and the high order Markov model have higher prediction accuracy than the first order Markov model, and then widely used in pattern recognition such as speech, handwriting and gesture recognition. In the high-order Markov chain, the number of parameters grows exponentially with respect to the order, and hampers the parameter estimation. To solve these problems, Raftery introduced the mixture transition distribution (MTD) model in 1985 as a parsimonious model for high-order Markov chains. However, the parameter estimation of MTD model is still not easy when using the EM algorithm. In this paper we propose a new Markov model with high-order hidden process and MTD. We show that, by assuming that the latent process follows a second-order Markov chain, the class of high-order Markov models can be generalized in an advisable way. This combination generalizes not only the MTD model, but also the HMM. To reduce some unnecessary errors in parameter estamation, we use the scaling procedure. Moreover, an application using an impulsive noise sequence shows that the generalization can lead to better results than its nested models.