Kullback-Leibler information measure for studying convergence rates of densities and distributions

Author

Meyer, M. Eugene ; Gokhale, D.V.

Author_Institution

Dept. of Math. & Stat., California State Univ., Chico, CA, USA

Volume

39

Issue

4

fYear

1993

fDate

7/1/1993 12:00:00 AM

Firstpage

1401

Lastpage

1404

Abstract

The Kullback-Leibler (KL) information measure l(f₁:f₂) is proposed as an index for studying rates of convergence of densities and distribution functions. To this end, upper bounds in terms of l(f ₁:f₂) for several distance functions for densities and for distribution functions are obtained. Many illustrations of the use of this technique are given. It is shown, for example, that the sequence of KL information measures converges to zero more slowly for a normalized sequence of gamma random variables converging to its limiting normal distribution than for a normalized sequence of largest order statistics from an exponential distribution converging to its limiting extreme value distribution. Furthermore, a sequence of KL information measures for log-normal random variables approaching normality converges more slowly to zero than for a sequence of normalized gamma random variables

Keywords

convergence; information theory; statistical analysis; Kullback-Leibler information measure; convergence rates; densities; distance functions; distribution functions; exponential distribution; extreme value distribution; gamma random variables; largest order statistics; log-normal random variables; normal distribution; normalized sequence; upper bounds; Convergence; Density measurement; Distribution functions; Extraterrestrial measurements; Gaussian distribution; Mathematics; Random variables; Sampling methods; Statistical distributions; Upper bound;

fLanguage

English

Journal_Title

Information Theory, IEEE Transactions on

Publisher

ieee

ISSN

0018-9448

Type

jour

DOI

10.1109/18.243456

Filename

243456