Beyond Mining: Characterizing the Data Generation Process

Data in databases is the outcome of an underlying data generation process (dgp), which is a complex function of unknown number of parameters. A few of these parameters may be known and understood by the domain expert or the end-user. The trends hidden in the evolving databases evolve with time and are manifestations of the underlying dgp. We believe that one of the objectives of the KDD technology is to discover the unknown parameters of the dgp and enable its characterization. This is a pre-requisite for developing capabilities to understand and possibly control the dgp. In this paper we draw the attention of the data mining research community towards the need to look beyond mining. The community is already paying serious attention to the issue of changes in the patterns discovered from evolving data repositories. We identify the commanalities in these works and show that they fit in a common framework where the discovered knowledge is consolidated and deviations are quantified. We extend this framework by introducing abstraction levels facilitation characterization of the underlying data generation process.