Distributed urban traffic control based on locally observable cell occupancies

A distributed reinforcement strategy for controlling the traffic lights in an urban area is proposed to minimize the average waiting time of the vehicles in the network. The network is assumed to follow the dynamics of the cell transmission model (CTM). Unlike many other CTM-based approaches, the proposed strategy only uses those cell occupancies that are observable from the flow measurements. A sufficient condition for the observability of a cell occupancy from its inflows and outflows is presented, given the congestion status of the flows. Those cells not satisfying this observability condition are heavily penalized in order to push the traffic agents to take actions that make them observable in future. Using distributed value functions, it is shown that each traffic agent takes into account the waiting time of all the intersections in choosing its action given that the graph of the network is connected. The efficacy of the proposed strategy is validated by simulation.