Bucketing Coding and Information Theory for the Statistical High-Dimensional Nearest-Neighbor Problem

The problem of finding high-dimensional approximate nearest neighbors is considered when the data is generated by some known probabilistic model. A large natural class of algorithms (bucketing codes) is investigated, Bucketing information is defined, and is proven to bound the performance of all bucketing codes. The bucketing information bound is asymptotically attained by some randomly constructed bucketing codes. The example of n Bernoulli(1/2) very long (length d → ∞) sequences of bits is singled out. It is assumed that n - 2m sequences are completely independent, while the remaining 2m sequences are composed of m dependent pairs. The interdependence within each pair is that their bits agree with probability 1/2 <; p ≤ 1. It is well known how to find most pairs with high probability by performing order of n^log₂2/p comparisons. It is shown that order of n^1/p+∈comparisons suffice, for any ∈ > 0. A specific 2-D inequality (proven in another paper) implies that the exponent 1/p cannot be lowered. Moreover, if one sequence out of each pair belongs to a known set of n^(2p-1)2 sequences, pairing can be done using order n^1+∈ comparisons!