Deep Blue: A Fuzzy Q-Learning Enhanced Active Queue Management Scheme

Although RED has been widely used with TCP, however it has several known drawbacks [1]. The BLUE algorithm that benefits from a different structure has tried to compensate some of them in a successful way [2]. A quick review on active queue management algorithms from the very beginning indicates that most of them tried to improve classic algorithms. Some of them use network traffic history to achieve more flexibility and prediction ability while others use algorithms such as fuzzy logic to address scalability problem and high input load. Our proposed approach benefits from both: Using fuzzy logic to deal with high input load and embedding expert knowledge into the algorithm while optimizing router decisions with reinforcement learning fed by network traffic history. We call this approach "DEEP BLUE" as is consist of an improved version of BLUE algorithm. Derived from BLUE, our algorithm uses packet drop rate and link idle events to manage congestion. Our experiments using OPNET simulator shows that this scheme works faster and more efficient than original BLUE.