Thump storage: A management and analysis system for structured big data

Structured big data presents a great challenge to the current platform and technology. It is a crossing field of two technological fields, namely the distributed-storage and parallel-processing of massive data, and relation-oriented database. In this paper, we present a management and analysis system for structured big data called ThumpStorage by integrating the bottom distributed structure of Hadoop distributed file system (HDFS) and the partitioning and scheduling technology of the massive parallel processing (MPP) database. This system shows high efficiency, low latency and high scalability. Finally, we test ThumpStorage by single table query, multi table query and concurrent jobs under different cluster node numbers, and compare with the test result of Hive.