Mission planning in unstructured environments: A reinforcement learning approach

Mission planning scenarios in robotics typically involve one or more semi-autonomous agents and a human operator. The high-level goal is to find an optimal allocation of agents to tasks. Each individual task or collection of tasks may have subgoals, such as search, localization, or information gathering. Each of these lower-level goals are their own topics, and the high-level goal of task accomplishment can be categorized as a decision problem. The focus of this work is on solving decision problems through reinforcement learning. Tasks are completely parameterized by a minimal set of basis constraints—spatial, temporal, and (agent-task) coupling—which produces a single cost for each agent-task pairing. The cost completely captures the ability of a specific agent to perform a specific task. A novel state representation is presented that mitigates exponential growth of the state space by scaling independently of the spatial dimension. The decision problem is cast as an MDP and an optimal policy is found using Q-learning. Simulation results are presented for a two-agent two-task example, showing convergence of the value function and improved learning over time. To highlight the scalability of the learning algorithm, additional simulations compare the learned policy to a hand-coded greedy policy for varying number of tasks. The learned policy is shown to reliably allocate agents to tasks with minimal parameter tuning and is robust to low-level changes to agent dynamics as well as random agent motion.