Title :
Perturbation realization, potentials, and sensitivity analysis of Markov processes
Author :
Cao, Xi-Ren ; Chen, Han-Fu
Author_Institution :
Hong Kong Univ. of Sci. & Technol., Kowloon, Hong Kong
fDate :
10/1/1997 12:00:00 AM
Abstract :
Two fundamental concepts and quantities, realization factors and performance potentials, are introduced for Markov processes. The relations among these two quantities and the group inverse of the infinitesimal generator are studied. It is shown that the sensitivity of the steady-state performance with respect to the change of the infinitesimal generator can be easily calculated by using either of these three quantities and that these quantities can be estimated by analyzing a single sample path of a Markov process. Based on these results, algorithms for estimating performance sensitivities on a single sample path of a Markov process can be proposed. The potentials in this paper are defined through realization factors and are shown to be the same as those defined by Poisson equations. The results provide a uniform framework of perturbation realization for infinitesimal perturbation analysis (IPA) and non-IPA approaches to the sensitivity analysis of steady-state performance; they also provide a theoretical background for the PA algorithms developed in recent years
Keywords :
Banach spaces; Markov processes; perturbation techniques; probability; sensitivity analysis; Markov processes; Poisson equations; infinitesimal generator; infinitesimal perturbation analysis; perturbation realization; potentials; sensitivity analysis; steady-state performance; Algorithm design and analysis; Computational modeling; Discrete event systems; Helium; Markov processes; Optimization; Performance analysis; Poisson equations; Sensitivity analysis; Steady-state;
Journal_Title :
Automatic Control, IEEE Transactions on