An efficient two-level bitmap index for cloud data management

A Cloud may be seen as a type of flexible computing infrastructure consisting of many compute nodes, where resizable computing capacities can be provided to different customers. To fully harness the power of the Cloud, efficient data management is needed to handle huge volumes of data and support a large number of concurrent end users. To achieve that, a scalable and high-throughput indexing scheme is generally required. Such an indexing scheme must support parallel search to improve scalability. In this paper, we present a bitmap based indexing scheme for efficient data processing in the Cloud. Our approach can be summarized as follows. First, we build a local bitmap index for each compute node which only indexes data residing on the node. Second, we organize the compute nodes as a structured overlay and each node maintains a portion of the global index for the whole different data. The global index is also bitmap index to indicate the node each data resides in. Third, all bitmaps are compressed by adopting run-length coding for reducing storage requirement. We conduct extensive experiments on a LAN, and the results demonstrate that our indexing scheme is dynamic, efficient and scalable.