A Reinforcement Learning Framework for Path Selection and Wavelength Selection in Optical Burst Switched Networks

Optical burst switching (OBS) is a promising technology that exploits the benefits of optical communication and supports statistical multiplexing of data traffic at a fine granularity making it a suitable technology for the next generation Internet. Contention among the bursts that arrive simultaneously at a core node leads to burst loss which affects the throughput of higher layer traffic. Development of efficient algorithms for path selection and wavelength selection is crucial to minimize the burst loss probability (BLP) in OBS networks. In this paper, we formulate path selection and wavelength selection in OBS networks as a multi-armed bandit problem and discuss the difficulties to solve them optimally. We propose algorithms based on Q-learning to solve these problems near-optimally. At an egress node, the path selection algorithm evaluates the Q values for a set of precomputed paths and chooses a path that corresponds to minimum BLP. Similarly, Q-learning algorithm for wavelength selection selects a wavelength in a pre-routed path such that the BLP is minimized. We do not assume wavelength conversion and buffering at the core nodes and hence, selection of path and wavelength is done only at the edge nodes. We simulate the proposed algorithms under dynamic load to demonstrate that they reduce the BLP compared to the other adaptive algorithms available in the literature.