Motorway ramp-metering control with queuing consideration using Q-learning

The standard reinforcement learning algorithms have proven to be effective tools for letting an agent learn from its experiences generated by its interaction with an environment. Among others, reinforcement learning algorithms are of interest because they require no explicit model of the environment beforehand and learning happens through trial and error. This property makes them suitable for real control problems like traffic control. Especially when considering the performance of a network where for instance a local ramp-metering controller needs to consider the performance of the network, since limitations needs to be considered, like the maximum permissible queue length, reinforcement learning algorithms are of interest. Here, a local ramp-metering control problem with queuing consideration is taken up and the performance of standard Q-learning algorithm as well as a newly proposed multi-criterion reinforcement learning algorithm is investigated. The experimental analysis confirms that the proposed multi-criterion control approach has the capability to decrease the state-space size and increase the learning speed of controller while improving the quality of solution.