Collaborative filtering recommendation algorithm based on Hadoop and Spark

The aim of this work was to develop and compare recommendation systems which use the item-based collaborative filtering algorithm, based on Hadoop and Spark. Data for the research were gathered from a real social portal the users of which can express their preferences regarding the applications on offer. The Hadoop version was implemented with the use of the Mahout library which was an element of the Hadoop ecosystem. The authors original solution was implemented with the use of the Apache Spark platform and the Scala programming language. The applied similarity measure was the Tanimoto coefficient which provides the most precise results for the available data. The initial assumptions were confirmed as the solution based on the Apache Spark platform turned out to be more efficient.