Departure MANagement with a Reinforcement Learning Approach: Respecting CFMU Slots

This paper considers how existing Reinforcement Learning (RL) techniques can be used to model and learn solutions for large scale Multi-Agent Systems (MAS). The large scale MAS of interest is the context of the movement of departure flights in big airports, commonly known as the Departure MAN-agement (DMAN) problem. A particular DMAN subproblem is how to respect Central Flow Management Unit (CFMU) take-off time windows, which are time windows planned by flow management authorities to be respected for the take-off time of departure flights. A RL model to handle this problem is proposed including the Markov Decision Process (MDP) definition, the behavior of the learning agents and how the problem can be modeled using RL ranging from the simplest to the full RL problem. Several experiments are also shown that illustrate the performance of the machine learning algorithm, with a comparison on how these problems are commonly handled by airport controllers nowadays. The environment in which the agents learn is provided by the Fast Time Simulator (FTS) AirTOp and the airport case study is the John F. Kennedy International Airport (KJFK) in New York City, USA, one of the busiest airports in the world.