Optimal control of partially observable discrete time stochastic hybrid systems for safety specifications

This paper describes a theoretical framework for the design of controllers to satisfy probabilistic safety specifications for partially observable discrete time stochastic hybrid systems. We formulate the problem as a partial information stochastic optimal control problem, in which the objective is to maximize the probability that the state trajectory remains within a given safe set in the hybrid state space, using observations of the history of inputs and outputs. It is shown that this optimal control problem, which has a multiplicative payoff structure, is equivalent to a terminal payoff problem when the state space is augmented with a binary random variable capturing the safety of past state evolution. This allows us to derive a sufficient statistic for the probabilistic safety problem as a set of Bayesian filtering equations updating a conditional distribution on the augmented state space, as well as an abstract dynamic programming algorithm for computing the maximal probability of safety and an optimal control policy.