Generalized Expansion Dimension

In this paper we propose a framework for modeling the intrinsic dimensionality of data sets. The models can be viewed as generalizations of the expansion dimension, which was originally proposed for the analysis of certain similarity search indices using the Euclidean distance metric. Here, we extend the original model to other metric spaces: vector spaces with the L_p or vector angle (cosine similarity) distance measures, as well as product spaces for categorical data. We also provide a practical guide for estimating both local and global intrinsic dimensionality. The estimates of data complexity can subsequently be used in the design and analysis of algorithms for data mining applications such as search, clustering, classification, and outlier detection.