The measure space structure of logical Markov decision processes

There has been much progress from reinforcement learning towards relational reinforcement learning and many new algorithms are now presented. Many of these approaches are upgrades of propositional representations towards the use of relational or computational logic representations. In this paper, we present a novel mathematic structure in which the underlying Markov decision process (MDP) is built on both the ground and the logical measure space structure. We also combine the ground space with the logical space by using the conception of conditional expectation. This framework will not only bring a stochastic and intelligent style for reinforcement learning, but also provide a sound basis for verifying the validity of logical Markov decision process theory.