• DocumentCode
    3728172
  • Title

    Efficient Indexing and Querying of Massive Astronomical Data Using Compressed Word-Aligned Hybrid Bitmap

  • Author

    Ying-Bo Liu;Hui-Mei Dai;Feng Wang;Wei Dai;Hui Deng

  • Author_Institution
    Comput. Technol. Applic. Key Lab. of Yunnan Province, Kunming Univ. of Sci. &
  • fYear
    2015
  • Firstpage
    1567
  • Lastpage
    1572
  • Abstract
    New Vacuum Solar Telescope (NVST) possesses a world-class solar telescope which observes the sun by high resolution image instruments and multi-wave spectrometer in visible and near infrared bands. NVST has gathered millions of Flexible Image Transport System (FITS) files since it has been for the routine observation and thousands of files are generated in a day. To search efficiently for a subset of these extremely large and linearly increasing FITS files with low latency and response time goes beyond traditional database systems. In this paper, We present an NVST indexing and querying framework (NIQF) based on an implementation of word-aligned hybrid bitmaps to deal with the issue, and design several simple and useful APIs, which are convenient for integrating with NVST data archiving system and meet the requirements of fast data indexing of massive data processing in NVST. We study the performance of NIQF by comparing with MySQL. The results demonstrate that the performance of NIQF is more efficient than MySQL in a huge amount of solar data set. This study indicates potential performance and feasibility of NIQF for large astronomical data indexing and querying in NVST.
  • Keywords
    "Indexing","Metadata","Servers","Telescopes","Data mining"
  • Publisher
    ieee
  • Conference_Titel
    Systems, Man, and Cybernetics (SMC), 2015 IEEE International Conference on
  • Type

    conf

  • DOI
    10.1109/SMC.2015.277
  • Filename
    7379409