Towards a Multi-cluster Analytical Engine for Transportation Data

In the new digital age, the pace and volume of growing transportation related data is exceeding our ability to manage and analyze it. In this position paper, we present a data engine, Godzilla, to ingest real-time traffic data and support analytic and data mining over traffic data. Godzilla is a multi-cluster approach to handle large volumes of growing data, changing workloads and varying number of users. The data originates at multiple sources, and consists of multiple types. Meanwhile, the workloads belong to three camps, namely batch processing, interactive queries and graph analysis. Godzilla support multiple language abstractions from scripting to SQL-like language.