DocumentCode :
933472
Title :
Maximum entropy and conditional probability
Author :
Van Campenhout, Jan M. ; Cover, Thomas M.
Volume :
27
Issue :
4
fYear :
1981
fDate :
7/1/1981 12:00:00 AM
Firstpage :
483
Lastpage :
489
Abstract :
It is well-known that maximum entropy distributions, subject to appropriate moment constraints, arise in physics and mathematics. In an attempt to find a physical reason for the appearance of maximum entropy distributions, the following theorem is offered. The conditional distribution of X_{l} given the empirical observation (1/n)\\sum ^{n}_{i}=_{l}h(X_{i})=\\alpha , where X_{1},X_{2}, \\cdots are independent identically distributed random variables with common density g converges to f_{\\lambda }(x)=e^{\\lambda ^{t}h(X)}g(x) (Suitably normalized), where \\lambda is chosen to satisfy \\int f_{\\lambda }(x)h(x)dx= \\alpha . Thus the conditional distribution of a given random variable X is the (normalized) product of the maximum entropy distribution and the initial distribution. This distribution is the maximum entropy distribution when g is uniform. The proof of this and related results relies heavily on the work of Zabell and Lanford.
Keywords :
Entropy functions; Density functional theory; Density measurement; Entropy; Frequency; Helium; Mathematics; Physics; Random variables; Statistical distributions; Uncertainty;
fLanguage :
English
Journal_Title :
Information Theory, IEEE Transactions on
Publisher :
ieee
ISSN :
0018-9448
Type :
jour
DOI :
10.1109/TIT.1981.1056374
Filename :
1056374
Link To Document :
بازگشت