The Holodex: Integrating Summarization with the Index

In this paper we introduce the Holodex, a `holistic index´ for databases that includes a facility for statistics and aggregate-like computations. The Holodex is an integration of the conventional index and summarization over traversals of the index. It can store customized summaries in its data structure, and in this way it can maintain, and provide fast access to, summarized information. The Holodex rests on the summary-traversal architecture - a customizable summarization scheme for tree indexes. An important property of the summary-traversal architecture is that index structures defining an ordering on data can be augmented to provide extra summary information as well. For example, both tree indexes (such as the B+-tree) and tree-hash hybrids (e.g., multi-level trie hashing and interpolation search tree) define an ordering, and they can be naturally extended to include summary information. This combination of indexing and summarization has a variety of uses, including computation of aggregate functions, rollups, bulk computation, and a variety of kinds of statistics, particularly those that are in some way related to order. More specifically, it is useful for computing non-parametric statistics - including rank statistics and order statistics - as well as direct implementation of queries like basic statistical tests on sample distributions