Incremental Updating of Nearest Neighbor-Based High-Dimensional Entropy Estimation

We present an algorithm for estimating entropy from high-dimensional data based on Kozachenko-Leonenko nearest neighbor estimator. The problem of finding all nearest neighbors is approximately solved using a best-bin first (BBF) bottom-up k-D tree traversal. Our main application is evaluating higher-order mutual information (MI) image similarity criteria that, unlike standard scalar MI, are directly usable for vector (e.g. color) images and can take into account neighborhood information. As during the optimization the MI criterion is often evaluated for very similar images, it is advantageous to update the k-D tree incrementally. We show that the resulting algorithm is fast and accurate enough to be practical for the image registration application