Title :
From Finite-System Entropy to Entropy Rate for a Hidden Markov Process
Author :
Zuk, Or ; Domany, Eytan ; Kanter, Ido ; Aize, Michael
Author_Institution :
Dept. of Phys. of Complex Syst., Weizmann Inst. of Sci.
Abstract :
A recent result presented the expansion for the entropy rate of a hidden Markov process (HMP) as a power series in the noise variable epsi. The coefficients of the expansion around the noiseless (epsi=0) limit were calculated up to 11th order, using a conjecture that relates the entropy rate of an HMP to the entropy of a process of finite length (which is calculated analytically). In this letter, we generalize and prove the conjecture and discuss its theoretical and practical consequences
Keywords :
entropy; hidden Markov models; HMP; Hidden Markov process; entropy rate; finite-system entropy; power series; Entropy; Hidden Markov models; Information theory; Markov processes; Physics; Signal processing; Signal to noise ratio; Speech recognition; Stochastic resonance; Taylor series; Entropy; Taylor series; hidden Markov process (HMP);
Journal_Title :
Signal Processing Letters, IEEE
DOI :
10.1109/LSP.2006.874466
