WS2F: A weakly supervised framework for data stream filtering

In this paper we present a weakly supervised framework for relevant content filtering from social media platforms such as Twitter. Social media platforms are a rich source of information these days. However of all the available information, there is only a small fraction of which is of general interest. Most of the other information pertains to personal events, and is very specific to the users who are contributing that. It is therefore usually not of general interest. In this paper, we present a framework to filter out the topic-specific relevant information from the irrelevant information in the stream of text provided by social media platforms. Our framework does not depend on any labeled data, however it is capable of using domain knowledge in the form of rules and guidelines provided by domain experts. It is therefore easily extensible for new topics and events. The proposed framework is built keeping the streaming nature of social media platforms in mind, i.e., it is able to discover the content relevant to a specific event as it evolves in the text stream. Because of its adaptive nature, it is not only able to filter the relevant content, but also able to generate event story lines as the event evolves. We experiment on a dataset provided by TREC, and show that the framework not only filters relevant content for an event but also generates its story line effectively.