Optimizing Distributed RDF Triplestores via a Locally Indexed Graph Partitioning

Semantic web techniques based on RDF are a promising approach to help make sense of the growing deluge in scientific data and conceptually integrate separately administered datasets. However, storing large datasets entirely on a single machine is not scalable, which has led to the concept of distributed triple stores. Existing triple stores, however, fail to take advantage of the non-uniform nature of semantic web data, leading to inefficient data allocation. In this paper, we extend our previous work on uniform graph partitioning to include a local index which is used to filter intermediate results and optimize sub-querying, and a new system design which can better exploit graph partitioned datasets. This also allows us to measure the total communication cost in our simulation. As shown in our experiments, our approach can effectively reduce the communication cost of query-processing messages compared with other approaches, balance the size of partitions compared with other approaches, and enhanced parallelism through independent sub-querying.