The analysis of discrete transient events in Markov games

The evolution of a system from the transient phase into a steady-state or asymptotic phase is an important area of study in engineering and the mathematical sciences. While analytic methods exist for determining the steady-state behavior of a system, the transient analysis is typically more difficult. Transient analysis is often approached in either an ad hoc, case-by-case manner or is performed by simulation. In this paper we explore the transient analysis of absorbing Markov chains by counting discrete-time events. We derive a closed-form expression for the expectation of these events and give some examples. We then show how several single-agent systems may be combined into a multi-agent system where the interactions between agents can be analyzed. This affords a model for analyzing competition. For example, we can determine advantages to specific players and determine the expected number of lead changes. After developing these ideas we present simulation results to verify our methods.