Towards an efficient platform for social big data analytics

With the development of social networks, people can easily share their feelings and express their opinions on interesting topics. Thus, social media mining is becoming an important research topic. However, huge volume of social Web data in various forms are rapidly generated around the world in a much faster speed than those of any other media. This can lead to the difficulties in social data analysis: noisy data and efficiency. In this paper, we propose a distributed data analytics platform for social media. First, data from various sources are collected and stored in a distributed index for efficient retrieval. Then, a distributed analytics framework is built from memory-based cluster computing based on the MapReduce paradigm. Finally, statistical analysis is performed and integrated for presentation. In the experiment, we built the platform by open source projects Hadoop and Spark, and implemented combinations of map and reduce operations. We compared the efficiency and scalability of the platform on various check-in datasets. Further evaluation is needed to verify the performance in different types of operations.