Adaptive Multimedia Mining on Distributed Stream Processing Systems

We present an application for distributed semantic concept detection in multimedia streams. The streams are mined using Support Vector Machine based concept detectors (classifiers) deployed on a distributed stream processing system. We organize the classifiers into a hierarchical topology based on semantic relationships between the concepts of interest, and use the system resource manager to place the topology across a set of processing nodes. We then develop distributed game theoretic optimization strategies for dynamic adaptation of individual classifier operating characteristics in order to maximize end-to-end application utility under varying resource availability. As part of this paper, we will demonstrate the principles behind large-scale multimedia stream mining, and showcase the design, development, deployment, and distributed adaptation of such applications on a large scale cluster. A video demonstration of the system can be found at: http://childman.bol.ucla.edu/ICDM/demovideoicdm2009.swf.