Constrained Complexity Generalized Context-Tree Algorithms

In this paper, we present a generalization of the context tree weighting algorithm that can address limitations imposed by the tree structure of traditional context-tree algorithms. By allowing a more general graphical structure, we demonstrate how a greatly increased class of models can be compactly represented using a context graph. Furthermore, through a judicious choice of this context graph, we show that a modified version of the weighting algorithm exists with computational complexity that remains linear in the context-graph depth. Although we present this method specifically in the context of universal prediction and focus on a particular context graph, the method is generally applicable and can be used to trade off algorithmic complexity with modeling power.