Existence and construction of capacity-achieving network codes for distributed storage

In a distributed storage system based on erasure coding, when a storage node fails, repairing the erasure code incurs some network traffic. Previous work has characterized the fundamental tradeoff between storage efficiency and repair network bandwidth. This was done via a cut-based analysis on a graph that models the evolution of information flow in the storage system subject to arbitrary sequences of node failures/repairs. This paper presents techniques for constructing network codes that achieve the optimal tradeoff between storage efficiency and repair network bandwidth. The challenge here is that network coding is applied over an unbounded graph with an unbounded number of receivers. It is shown in this paper that optimal codes can be constructed over a finite field whose size depends only on the maximum number of nodes at any instant, but independent of how many failures/repairs can happen. Key to the code construction is a ldquopath-weavingrdquo procedure that leads to an inductive existence proof/code construction.