Average cost optimal threshold strategies for remote estimation with communication cost

In this paper, we consider a remote sensing system that consists of a sensor and an estimator. A sensor observes a first order Markov source and must communicate it to a remote estimator. Communication is noiseless but expensive. At each time, based on the history of its observations and decisions, the sensor chooses whether to transmit or not. If the sensor does not transmit, then the estimator must estimate the Markov process using its past observations. We study the average cost problem in the light of vanishing discount approach. The problem was studied previously by Lipsa and Martins, IEEE TAC, 2011 and by Nayyar et al, IEEE TAC, 2013, where it was shown that the optimal estimation policy is Kalman-like and the optimal communication policy is to communicate when the estimation error is greater than a threshold. In the discounted set-up, we had earlier characterized the optimal policy and the optimal thresholds as a function of communication cost. The average cost problem is investigated as the limiting case of the discounted cost problem as the discount factor approaches one. The average cost and the optimal values of the thresholds are provided in terms of the communication cost. Lastly, we present an example of birth-death Markov chain to illustrate our results.