D3G: novel approaches to data statistics, understanding and preprocessing on the grid

Relocating the code for data preprocessing (DPP) closer towards the data source is the overall task of the D³G framework (data statistics, data understanding, data preprocessing on the grid), developed within a joint project of the University of Vienna, the Vienna University of Technology and the Czech Technical University. This paper presents the data service side architecture to gather data statistics on-the-fly and use them in remote DPP methods on query results as well as an approach to gather exact continuous data statistics for whole tables in a database on the grid. The performance results of our prototype implementation are showing low running costs for the continuous data statistics inside the database and also the feasibility of our proposed data service side functionality.