Distributed flow scheduling in an unknown environment

Flow scheduling is crucial in the next-generation network but hard to address due to fast changing link states and tremendous cost to explore the global structure. In this paper, we first design a distributed virtual game to solve the optimization of flow scheduling problem assuming the priori knowledge of the distribution of edge cost as a random variable. In our virtual game, we use incentives to stimulate selfish users to reach a Nash Equilibrium Point which is suboptimum based on the analysis of the `Price of Anarchy´. This algorithm is then generalized into the situation with unknown cost distribution, where the ultimate goal is to minimize the cost in the long run. In order to achieve a reasonable tradeoff between exploration of cost distribution and exploitation with limited information, we model this problem as a Multi-armed Bandit Game and combine the newly proposed DSEE with our virtual game design. Armed with these powerful tools, we find a totally distributed algorithm to ensure the logarithmic growing of regret with time, which is optimum in classic Multi-armed Bandit problem. Theoretical proof and simulation results both confirm the effectiveness of our algorithm. To the best of our knowledge, this is the first work to combine multi-armed bandit with distributed flow scheduling.