Context tree estimation for not necessarily finite memory processes, via BIC and MDL

Author

Csiszár, Imre ; Talata, Zsolt

Author_Institution

Renyi Inst. of Math., Hungarian Acad. of Sci., Budapest

fYear

2005

fDate

4-9 Sept. 2005

Firstpage

755

Lastpage

759

Abstract

The concept of context tree, usually defined for finite memory processes, is extended to arbitrary stationary ergodic processes (with finite alphabet). These context trees are not necessarily complete, and may be of infinite depth. The familiar BIC and MDL principles are shown to provide strongly consistent estimators of the context tree, via optimization of a criterion for hypothetical context trees of finite depth, allowed to grow with the sample size n as o(log n). Algorithms are provided to compute these estimators in O(n) time, and to compute them on-line for all i les n in o(nlog n) time

Keywords

Bayes methods; computational complexity; optimisation; stochastic processes; trees (mathematics); BIC principle; Bayesian information criterion; MDL principle; O(n) time; arbitrary stationary ergodic processes; context tree estimation; criterion optimization; finite alphabet; hypothetical context trees; infinite depth; minimum description length; not necessarily finite memory processes; o(nlog n) time; strongly consistent estimators; Bayesian methods; Computational complexity; Context modeling; Mathematics; Maximum likelihood estimation; Probability; Stochastic processes; Upper bound;

fLanguage

English

Publisher

ieee

Conference_Titel

Information Theory, 2005. ISIT 2005. Proceedings. International Symposium on

Conference_Location

Adelaide, SA

Print_ISBN

0-7803-9151-9

Type

conf

DOI

10.1109/ISIT.2005.1523438

Filename

1523438