Computation of absorption probability distributions of continuous-time Markov chains using regenerative randomization

Randomization is a popular method for the transient solution of continuous-time Markov models. Its primary advantages over other methods (i.e., ODE solvers) are robustness and ease of implementation. It is however well-known that the performance of the method deteriorates with the “stiffness” of the model: the number of required steps to solve the model up to time t tends to Λt for Λt→∞, where Λ is the maximum output rate. For measures like the unreliability Λt can be very large for the t of interest, making the randomization method very inefficient. In this paper we consider such measures and propose a new solution method called regenerative randomization which exploit the regenerative structure of the model and can be far more efficient. Regarding the number of steps required in regenerative randomization we prove that (1) it is smaller than the number of steps required in standard randomization when the initial distribution is concentrated in a single state, (2) for Λt→∞, it is upper bounded by a function O(log(Λt/ε)), where ε is the desired approximation error bound. Using a reliability example we analyze the performance and stability of the method