Comparative evaluation of native RDF stores

The semantic web popularity is basis for the development of RDF stores. Benchmarks are being used to validate the performance claim of RDF stores. The existing studies measure the stores´ performance only in terms of time and do not cover resource utilization, particularly the main memory, for data manipulation operations of RDF stores. Therefore, their provided information on semantic stores is limited. To cope with the deficiencies of the existing benchmarks, we have proposed an evaluation metric that considers both time and main memory to judge the performance of semantic stores on CRUD operations. The evaluation methodology has been applied on the existing semantic stores using the Barton Library dataset. We deduce that time and memory required for any operation, are interdependent. The evaluation results show that the proposed evaluation metric provides better cost estimation than the existing benchmarks.