Estimating a probability using finite memory

Author

Leighton, F. Thomson ; Rivest, Ronald L.

Volume

32

Issue

6

fYear

1986

fDate

11/1/1986 12:00:00 AM

Firstpage

733

Lastpage

742

Abstract

Let ${X_{i}}_{i=1}^{\\infty }$ be a sequence of independent Bernoulli random variables with probability $p$ that $X_{i} = 1$ and probability $q=1-p$ that $X_{i} = 0$ for all $i \\geq 1$ . Time-invariant finite-memory (i.e., finite-state) estimation procedures for the parameter p are considered which take $X_{1}, \\cdots$ as an input sequence. In particular, an n-state deterministic estimation procedure is described which can estimate p with mean-square error $O(\\log n/n)$ and an $n$ -state probabilistic estimation procedure which can estimate $p$ with mean-square error $O(1/n)$ . It is proved that the $O(1/n)$ bound is optimal to within a constant factor. In addition, it is shown that linear estimation procedures are just as powerful (up to the measure of mean-square error) as arbitrary estimation procedures. The proofs are based on an analog of the well-known matrix tree theorem that is called the Markov chain tree theorem.

Keywords

Estimation; Probability; Computer errors; Computer science; Counting circuits; Estimation error; Laboratories; Probability; Random variables; State estimation; Statistics; Tail;

fLanguage

English

Journal_Title

Information Theory, IEEE Transactions on

Publisher

ieee

ISSN

0018-9448

Type

jour

DOI

10.1109/TIT.1986.1057250

Filename

1057250