On the sensitivity analysis of the expected accumulated reward

This paper deals with the sensitivity computation of the expected accumulated reward of Markov models. Very often, we are facing the problem of the computation time, especially when the Markov process is stiff. We consider the standard uniformization method for which the global error may be easily bounded. Because the time complexity of this method becomes large when the stiffness increases, we then suggest an ordinary differential equations method, the third-order implicit Runge–Kutta method. After providing a new way of writing the system of equations to be solved, we apply this method with a stepsize choice different from the classical one in order to accelerate the algorithm execution. This method is interesting for stiff Markov models but unfortunately, it is difficult to control the global error. We propose a new approach based on the uniformized power technique. This method will save computation time if the mission time is long and the state space is not too large. Moreover, this method integrates an efficient error control mechanism. The time complexities of the three methods are compared via a concrete example.