Optimal Node Visitation in Stochastic Digraphs

The optimal node visitation (ONV) problem addressed in this paper concerns the visitation of a subset of nodes in a stochastic graph a specified number of times, while minimizing the expected visits to another node in this graph. The presented results first provide a formulation of the ONV problem as a stochastic shortest path problem, and subsequently they develop a suboptimal policy that is computationally tractable and asymptotically optimal. In particular, it is established that the ratio of the expected performance of this policy to the expected performance of an optimal policy converges to one, as the underlying visitation requirements are scaled uniformly to infinity. Furthermore, it is shown that under some stronger assumptions, the divergence of the performance of this policy from the performance of the optimal policy remains uniformly bounded by a constant, as the visitation requirements are scaled to infinity. Finally, it is shown that, for certain problem structures, the considered policy admits a closed-form characterization of its performance, which subsequently enables its optimized parameterization and its efficient integration into adaptive control schemes of even higher efficiency.