Engine maintenance policy optimization with succinct value function representation

Due to the large economic and environmental impact and pervasive application, engine maintenance policy optimization has attracted the research interest in the past several decades. Markov decision process (MDP) provides a general framework for this problem, but the large state space makes it difficult to apply the traditional value iteration and policy iteration in practice. How to approximate the value function is an important issue in approximate dynamic programming but lack of systematic approach so far. For an engine maintenance problem with n safety-critical components, each of which has l-day lifetime, we develop a succinct representation of the value function, which reduces the required memory space from lⁿ to nl^n-1. Specifically, for the two-component case we develop an approximate value function which further reduces the time complexity for the calculation of the value function from O(l) to O(1). Numerical results demonstrate that the approximate value iteration usually obtains the optimal policy.