Exploring Data Streams with Nonparametric Estimators

A variety of real-world applications requires a meaningful online analysis of transient data streams. An important building block of many analysis tasks is the characterization of the underlying data distribution. Sophisticated techniques from the area of nonparametric statistics provide a well-defined estimation of continuous data distributions. The analysis of data streams may gain advantage of these techniques, however, the rigid processing requirements of streams render a direct application impossible. In our work, we tackle the adaptation of nonparametric techniques to streaming data. We concentrate on density estimation as it provides a convenient basis for the exploration of an unknown continuous data distribution. Specifically, we have developed kerneland wavelet-based density estimators for data streams in compliance with their processing requirements. Both techniques are incorporated into PIPES, our Java library for advanced data stream processing and analysis. In the demonstration, we present our nonparametric density estimators over data streams and show their performance for a variety of heterogeneous data streams from different real-world application scenarios. We also present the implementation of further analysis tasks on top of our estimators by means of illustrative use cases