Aggrecode: Constructing route intersection for data reconstruction in erasure coded storage

Node failures often occur in large-scale data centers today. Erasure coded storage system provides high data reliability via data reconstruction. Existing work can improve reconstruction performance, while considering the transmission of recovery data as the main source of reconstruction overheads. Transmission costs are highly related with network topology, which is unfortunately overlooked. An ideal connected topology assumes that two nodes in a data center has a direct link. The unmatching design between the network model and the practical topology may lead to an underestimated transmission costs. In this paper, we propose an erasure coded storage system for data reconstruction, which uses the practical network topology to minimize the reconstruction transmission costs. First, we identify the aggregation feature of erasure coding reconstruction and propose Aggregation Decoding, which splits the decoding process into several sub-decoding operations during reconstruction routing to reduce overall recovery data to be transmitted. We further improve Aggrecode to construct efficient route basing on the location of participating nodes to exploit the aggregation feature of Aggregation Decoding. We formulate this routing problem as a relaxed Steiner Tree problem. We design two heuristic routing algorithms based on ant-colony optimization specialized for two failure recovery cases, e.g., node recovery and degraded read. Our analytical results demonstrate the important properties of Aggrecode. These properties are evaluated by extensive experiments deployed on popular data center topologies, such as Torus, Fat-tree, DCell and BCube. The results show that Aggrecode can reduce data transmission costs by at least 37.12% for all settings.