Evaluation of Linux I/O Schedulers for Big Data Workloads

Big data is receiving more and more attention as an increasingly large number of institutions turn to big data processing for business insights and customer personalization. Most of the research in big data has focused on how to distribute a given workload over a set of computing nodes to achieve good performance. The operating system at each node uses an I/O scheduling algorithm to retrieve disk blocks and load them into main memory. Achieving good performance in big data applications is therefore inherently dependent on the efficiency of the I/O scheduler used in the OS of the nodes. In this paper, we evaluate the impact of different Linux I/O schedulers on a number of big data workloads. The objective of the study is to determine whether specific schedulers (or specific configurations of those schedulers) are superior to others in terms of supporting big data workloads.