Simulated-annealing type Markov chains and their order balance equations

Generalized simulated-annealing-type Markov chains are considered where the transition probabilities are proportional to powers of a vanishingly small parameter. It is possible to associate with each state an order of recurrence which quantifies the asymptotic behavior of the state occupation probability. These orders of recurrence satisfy a fundamental balance equation across every edge-cut in the graph of the Markov chain. Moreover, the Markov chain converges in a Cesaro sense to the set of states having the largest recurrence orders. The authors provide graph-theoretic algorithms to determine the solutions of the balance equations. By applying these results to the problem of optimization by simulated annealing, they show that the sum of the recurrence order and the cost is a constant for all states in a certain connected set, whenever a weak reversibility condition is satisfied