Exploiting Unsupervised Learning in Publish Subscribe System Design

Skewed ness in popularity among subscriptions and events is an inherent property in publish/subscribe systems. In this paper, we propose to exploit the popularity skew by utilizing unsupervised machine learning techniques in the design of publish/subscribe systems. The design comprises three main ideas. First, similar subscriptions are clustered together using unsupervised machine learning methods, and the resulting cluster membership information is then distributed to all the brokers in the system. Secondly, the brokers apply unsupervised learning methods again to partition the published events they receive. By grouping similar events together, the events can be delivered in batches, instead of single event-based delivery as commonly employed in publish/subscribe systems. Thirdly, the event batches are delivered either as a single message using traditional publish/subscribe system delivery process, or using application level multicast trees maintained by the system. We explore the traditional delivery process case in this paper. Our design admits a spectrum of possible accuracy and efficiency choices in the design of publish/subscribe systems. Using an extensive set of experiments, our proposal is shown to deliver events using the collective resources of the overlay network effectively, while achieving reasonable accuracy.