Integrated caching and tiering according to use and QoS requirements

In this paper we consider the management of a tiered storage system consisting of disk and flash drive storage and a DRAM cache, with the challenge of taking into account heterogeneous quality of service (QoS) requirements. We integrate and adjust methods which control the use of fast resources such as flash drives and cache, according to the user access patterns of different data extents and the QoS requirements of the extents, which we developed in previous work. Using traces from real production systems, we show that the benefits of the integrated system are substantially larger than that provided by each method alone. Our method is able to substantially improve the performance of data with high QoS demands, with little or no damage to data with low QoS demands. Thus we are able to exploit the resources of the storage system to the advantage of all data types. we show improvements in the range of 9%-71% for datasets with the highest QoS requirements, and 0%-70% response time improvement overall, compared to a QoS optimized system which took only disk drive resource allocation into consideration. In workloads where cache is useful we obtain large gains, showing that it is important to integrate back-end (drives) and front-end (cache) optimization.