On the optimal thresholds in remote state estimation with communication costs

In this paper, we consider a remote sensing system that consists of a sensor and an estimator. A sensor observes a first order Markov source and must communicate it to a remote estimator. Communication is noiseless but expensive. At each time, based on the history of its observations and decisions, the sensor chooses whether to transmit or not. If the sensor does not transmit, then the estimator must estimate the Markov process using its past observations. It was shown by Lipsa and Martins, 2011 and by Nayyar et al, 2013 that the optimal strategy has the following structure. The optimal estimation strategy is Kalman-like and the optimal communication strategy is to communicate when the estimation error is greater than a threshold. We derive closed form expressions for infinite horizon discounted cost version of the problem. Our solution approach is inspired by the idea of calibration used in multi-armed bandits. We identify the value of the communication cost for which one is indifferent between two consecutive threshold based strategies. Using these values, we characterize the optimal thresholds as a function of the communication cost. Lastly, we present an example of birth-death Markov chain to illustrate our results.