A peer-to-peer replica management service for high-throughput grids

Future high-throughput grids may integrate millions or even billions of processing and data storage nodes. Services provided by the underlying grid infrastructure may have to be able to scale to capacities not even imaginable today. In this paper we concentrate on one of the core components of the data grid architecture - the replica location service - and evaluate a redesign of the system based on a structured peer-to-peer network overlay. We argue that the architecture of the currently most widespread solution for file replica location on the grid is biased towards high-performance deployments and cannot scale to the future needs of a global grid. Structured peer-to-peer systems can provide the same functionality, while being much more manageable, scalable and fault-tolerant. However, they are only capable of storing read-only data. To this end, we propose a revised protocol for distributed hash tables that allows data to be changed in a distributed and scalable fashion. Results from a prototype implementation of the system suggest that grids can truly benefit from the scalability and fault-tolerance properties of such peer-to-peer algorithms.