Potential based sensitivity analysis of Markov chains

We propose a new approach to the single-sample-path-based sensitivity analysis of Markov chains. The approach is based on a fundamental concept: performance potentials. Like the potential energy in physics, only the differences between potentials are important. We show that the differences of potentials can be determined by using the group inverse of I-P, where I is the identity matrix and P the transition matrix of the Markov chain. Potentials reflect the system performance in transient periods and can be used to determine the performance sensitivity with respect to a change of the transition matrix. The results provide a general and efficient approach to the single-sample-path-based sensitivity analysis for many engineering systems, for which the standard perturbation analysis does not work well