Value of information in the control of stochastic systems

This paper discusses the control of nonlinear stochastic systems and, in particular, linear systems with unknown parameters. The stochastic nature of the problem leads to the probing and caution properties of the control. Explicit expressions of the probing and caution terms in a stochastic control problem are presented. These terms are obtained by a closed-loop approximation of the stochastic dynamic programming equation. An approximate value of information can be evaluated and the benefit to be derived from probing (experimentation)can be traded off against its cost. The interplay between caution and probing is discussed.