A Scalable Data Warehouse Model Based on Complex Semantic Event Processing in Distributed Systems

Data model in traditional data warehouse is always predefined and difficult to extend because of its rigid schema. This paper proposes a scalable data extraction model in data warehouse via Complex Semantic Event (CSE). CSE is an intelligent process to integrate data extraction, aggregation and transformation. The schema of CSE is automatically synchronized and easy to scale for changing requirement. High level aggregated data is stored on the server, while lower level data could be stored in distributed hosts since they are rarely queried. This CSE based approach also promotes the idea of implementing data warehouse aggregation in distributed network or computing cloud. This method decreases the load on servers and reduces network traffic. We implemented this CSE based model in our monitoring system, where data warehouse is used as the data source for trend analysis and prediction. It has been observed that this approach can improve system performance and is helpful in resource load balancing.