Clustering to Improve Microblog Stream Summarization

Microblogging has shown a massive increase in use over the past couple of years. According to recent statistics, Twitter (the most popular microblogging platform) has over 340 million posts per day coming from its 140 million active users. In order to help users manage this information overload or to assess the full information potential of such microblogging streams (sequences of posts), a few summarization algorithms have been proposed. However, they are designed to work on a stream of posts filtered on a particular keyword, whereas most streams suffer from noise or have posts referring to more than one topic. Because of this, the generated summary is incomplete and even meaningless. We approach the problem of summarizing a stream and propose adding a layer of text clustering as a preprocessing step. We show how, by clustering posts into related groups and then applying a summarization algorithm, the quality of the summary improves.