Top-k computations in MapReduce: A case study on recommendations

Top-k is a well-studied problem in the literature, due to its wide spectrum of applications, like information retrieval, database querying, Web search and data mining. In the big data era, the volume of the data and their velocity, call for efficient parallel solutions that overcome the restricted resources of a single machine. Our motivating application is recommenders, which typically deal with big numbers of users and items, but other applications might benefit as well, like keyword search. In this paper, we propose a parallel top-k MapReduce algorithm that, unlike existing MapReduce solutions, manages to handle cases in which the k results do not fit in memory.