Learning a pricing strategy in multi-domain DWDM networks

In today´s Internet the commercial aspect of routing is gaining more and more importance. Commercial agreements between ISPs (i.e. transit and peering agreements) influence the inter-domain routing policies which are now driven by monetary aspects as well as global resource and performance optimization. To allow scalability and protect business critical topology information, hierarchical routing and topology aggregation became a fundamental issue in modern inter-domain networks. In this paper, we introduce a pricing mechanism that takes into account the effects of load dependent internal costs on the domain income and allows ISPs to set link prices during the topology aggregation process. We adapt a Continuous Action Reinforcement Learning Automata (CARLA), to operate in this framework as a tool used by ISP operators to learn the best price according to the network state. The reinforcement signal is proportional only to the domain´s utility (i.e. profit) and thus does not need any central authority or sensitive information exchange among domains. Simulation results show that one ISP using CARLA can significantly improve its utility compared to other ISPs that statically choose their link prices. When two ISPs employ the same CARLA they can reach an equilibrium strategy while still improving their utilities.