Recognition capacity versus search speed in noisy databases

The tradeoff between the number of distinguishable objects and search speed in a data management system is investigated in an information-theoretic framework. In the discussed scenario, incoming high-dimensional (and noisy) data vectors are enrolled in (possibly multiple) clusters to be accessed later. Upon receiving a random query, which is the noisy version of an enrolled vector, the search engine retrieves only a subset of the clusters to compare against the query. This creates tension between the search speed (determined by the expected number of retrieved entries) and recognition capacity (maximum possible number of entries that can be reliably recognized). A single-letter achievable rate region is characterized and it is shown with examples that search can be performed much faster in the discussed scenario than in non-clustered linear scan without compromising maximum recognition capacity.