Adaptive sensing for search with continuous actions and observations

The problem of allocating sensing energy over a field arises in many applications. When this allocation is over space and time and observations of sensing outcomes are available, this becomes a stochastic control problem with a very large state space. In this paper, we study a two-stage sensor energy allocation problem with constraints on the total energy available and continuous-valued state and decision spaces. We develop a stochastic control formulation of this problem and establish lower bounds on the optimal cost. We use a lower bound as a surrogate cost and solve the associated stochastic control problem using dynamic programming combined with Lagrangian relaxation. Subsequently, we use the computed solutions to obtain near-optimal adaptive energy allocation policies. Numerical experiments establish that our approach yields superior performance to approaches proposed previously and can generate solutions two orders of magnitude faster than previous approaches.