Title :
A consistent algorithm for derivative estimation of Markov chains
Author_Institution :
Dept. of Syst. Sci. & Math., Washington Univ., St. Louis, MO, USA
Abstract :
A consistent algorithm for derivative estimation of finite-state, discrete-time Markov chains is presented. The basic idea is to simulate original Markov chains with modified performance measures that can be estimated by extra simulation. The computational load of the extra simulation at each step is bounded. The algorithm attains the best possible rate of convergence as the simulation time goes to infinity. A connection between the algorithm and solutions to Poisson equations is also revealed
Keywords :
Markov processes; differentiation; discrete time systems; finite automata; identification; simulation; stochastic processes; Poisson equations; computational load; derivative estimation; finite-state discrete-time Markov chains; modified performance measures; simulation; Computational modeling; Computer simulation; Cost function; Discrete event simulation; H infinity control; Poisson equations; Sensitivity analysis; State estimation; State-space methods; Stationary state;
Conference_Titel :
Decision and Control, 1994., Proceedings of the 33rd IEEE Conference on
Conference_Location :
Lake Buena Vista, FL
Print_ISBN :
0-7803-1968-0
DOI :
10.1109/CDC.1994.411084
