A Scalable Approach to Avoid Incast Problem from Application Layer

With the development of distributed computing technology and cloud computing technology, the amount of data is rising dramatically, as well as the number of datacenters, where we keep these data for long periods. In these datacenters, TCP protocol is still widely used in most of the network traffic. On the other hand, TCP protocol can also result in severe goodput collapse in a high bandwidth and low latency datacenter environment. One of the problems is called incast problem. In order to efficiently utilize the bandwidth of distributed systems such as datacenter environments, some traffic control mechanisms should be very necessary. In this paper, we discussed about an application layer control mechanism to improve the performance of certain traffic pattern which may cause incast problem. The main idea is to make the data flows be staggered and transferred in serialized manner. We conducted both simulation-based and real-machine-based experiments. In the real-machine-based experiments, we conducted both small scale test and large scale test. We find out that the proposed approach, staggered flows, is able to avoid incast problem and to make the performance better in most cases. Also, this paper shows the potential of scalability that this approach can keep the performance well with the increment of node quantity.