An indirect reinforcement learning approach for ramp control under incident-induced congestion

Incident-induced congestion is one of the main causes for delays on motorways. Strategies for managing such congestion using traffic control technologies can be classified into model-based and model-free methods. Both methods possess their own merits but also have drawbacks. Dyna-Q architecture is a method that can combine model-free learning and model-based planning together to obtain the benefits from both sides. Based on the Dyna-Q architecture, an indirect reinforcement learning (IRL) approach is derived in this study. The new method is compared with two other methods, namely DRL and ALINEA. Simulation experiment results show that, with suitable weight values, IRL can achieve a superior performance in many scenarios. Moreover, compared with DRL, IRL has a much faster learning speed.