Measurement of a Markov jump process: When is it too costly?

The problem of scheduling measurements of a continuous-time finite-state Markov jump process is considered. This model is relevant to determining the optimal frequency for performing medical exams, inventories, polls, and for many other applications. Unlike the LQG case, where the Kalman filter gain is independent of the measured signals, for this system, the optimal measurement schedule depends on the actual measurements obtained. The feedback rule for generating a measurement schedule is optimized to trade-off between uncertainty about the state of the system and the cost of measurement. Analysis of this trade-off leads to the determination of a critical cost-per-observation above which measurements are too costly, and the best policy is never to observe the system.