Markov decision processes for train run curve optimization

We propose three computationally efficient methods for finding optimal run curves of electrical trains, all based on the idea of approximating the continuous dynamics of a moving train by a Markov Decision Process (MDP) model. Deterministic continuous train dynamics are converted to stochastic transitions on a discrete model by observing the similarity between the properties of convex combinations and those of probability mass functions. The resulting MDP uses barycentric coordinates to effectively represent the cost-to-go of the approximated optimal control problem. One of the three solution methods uses equal-distance steps, as opposed to the usual equal-time steps, to avoid self transitions of the MDP, which allows very fast computation of the cost-to-go in one pass only.