Nonproduct Data-Dependent Partitions for Mutual Information Estimation: Strong Consistency and Applications

Author

Silva, Jorge ; Narayanan, Shrikanth

Author_Institution

Dept. of Electr. Eng., Univ. of Chile, Santiago, Chile

Volume

58

Issue

7

fYear

2010

fDate

7/1/2010 12:00:00 AM

Firstpage

3497

Lastpage

3511

Abstract

A new framework for histogram-based mutual information estimation of probability distributions equipped with density functions in (R^d,B(R^d)) is presented in this work. A general histogram-based estimate is proposed, considering nonproduct data-dependent partitions, and sufficient conditions are stipulated to guarantee a strongly consistent estimate for mutual information. Two emblematic families of density-free strongly consistent estimates are derived from this result, one based on statistically equivalent blocks (the Gessaman´s partition) and the other, on a tree-structured vector quantization scheme.

Keywords

information theory; statistical distributions; vector quantisation; density free strongly consistent estimation; density function; histogram based mutual information estimation; mutual information estimation; nonproduct data dependent partition; probability distribution; tree-structured vector quantization scheme; Asymptotically sufficient partitions; Vapnik–Chervonenkis inequality; data-dependent partitions; histogram-based estimation; mutual information; tree-structured vector quantization;

fLanguage

English

Journal_Title

Signal Processing, IEEE Transactions on

Publisher

ieee

ISSN

1053-587X

Type

jour

DOI

10.1109/TSP.2010.2046077

Filename

5433027