Data-centric HPC for Numerical Weather Forecasting

Modern day High Perfocmance Computing (HPC) systems are now capable of running detailed models that can accurately predict severe weather events like typhoons and floods. However, as more computing power becomes available to run Numerical Weather Prediction (NWP) models, scientists are now observing that performing subsequent data processing and analysis are increasingly becoming the major bottlenecks to effectively utilizing the HPC systems. In this paper, we propose a practical and workflow-conscious solution to improve the end-to-end process involved in NWP from the perspective of how data is acquired, processed and analyzed in the HPC system. Our work highlights the integration of data-centric middleware components including streams processing, data warehouse and business intelligence in an HPC environment to provide a scalable and robust data analytics platform suitable for the development of NWP models.