Author_Institution :
Comput. & Inf. Sci. Dept., Univ. of Pennsylvania, Philadelphia, PA, USA
Abstract :
The sciences, business confederations, and medicine urgently need infrastructure for sharing data and updates among collaborators\´ constantly changing, heterogeneous databases. The ORCHESTRA system addresses these needs by providing data transformation and exchange capabilities across DBMSs, combined with archived storage of all database versions. ORCHESTRA adopts a peer-to-peer architecture in which individual collaborators contribute data and compute resources, but where there may be no dedicated server or compute cluster. We study how to take the combined resources of Orchestra\´s autonomous nodes, as well as PCs from "cloud" services such as Amazon EC2, and provide reliable, cooperative storage and query processing capabilities. We guarantee reliability and correctness as in distributed or cloud DBMSs, while also supporting cross-domain deployments, replication, and transparent failover, as provided by peer-to-peer systems. Our storage and query subsystem supports dozens to hundreds of nodes across different domains, possibly including nodes on cloud services. Our contributions include (1) a modified data partitioning substrate that combines cluster and peer-to-peer techniques, (2) an efficient implementation of replicated, reliable, versioned storage of relational data, (3) new query processing and indexing techniques over this storage layer, and (4) a mechanism for incrementally recomputing query results that ensures correct, complete, and duplicate-free results in the event of node failure during query execution. We experimentally validate query processing performance, failure detection methods, and the performance benefits of incremental recovery in a prototype implementation.
Keywords :
distributed databases; groupware; query processing; relational databases; ORCHESTRA system; cloud DBMS; cloud services; collaborative data sharing systems; data exchange; data partitioning substrate; data transformation; database management systems; distributed DBMS; heterogeneous databases; indexing techniques; peer-to-peer architecture; query processing; query subsystem; relational data; storage subsystem; Clouds; Collaboration; Computer architecture; Databases; Indexing; Information science; Peer to peer computing; Personal communication networks; Prototypes; Query processing;