Multi-sourced Data Storage and Index Construction for Equipment Condition Assessment

With constant promotion and deepening of smart power grids, the data volume of power grid operation and equipment monitoring gain exponential growth and the environment of big data in electric system forms. Many platforms are deployed to meet different demands of equipment operation and maintenance. Since these platforms are independent and cannot be used in coordination, multi-sourced heterogeneous data become mainstream. Integrated storage and performance optimization of multi-sourced heterogeneous data are critical for condition assessment of power transmission and transformation equipment and security operation of power systems. In the paper, it focuses on the integration and optimization of multi-sourced data and design a storage schema of big data which can distributedly process and analyze data. It presents the storage schema of multi-sourced data based on HBase and a description method of metadata, which achieve the lossless and scalable integration of multi-sourced data. To solve the problem of low query efficiency due to the nonsupport of HBase for secondary index, it also proposes a constructing approach of secondary index for the attributes which are often used in the query tree of condition assessment. Extensive experiments show that the proposed storage schema of multi-sourced data and secondary index constructing approach have better query performance and index updating efficiency.