HiPerData: An autonomous large-scale model building and management platform for big data analytics

Data mining is a difficult task that relies on an exploratory and analytic process of processing large quantities of data in order to discover meaningful patterns for valuable insights. The increasing heterogeneity and complexity of data requires expert knowledge on how to combine multiple data mining techniques to process and analyze the data in an effective and efficient way. This paper presents a distributed architecture, HiPerData, for automated data processing and mining using large-scale computational resource management, model building and selection, and predictive and inference analysis. We illustrate two data mining tasks in which we automate the data mining knowledge flow construction based on the use of standards that have been defined in both data mining and automated-planning communities.