A service integrity assurance framework for cloud computing based on MapReduce

MapReduce is a highly efficient parallel large data sets (greater than 1TB) processing model, widely used in cloud computing environment. The current mainstream cloud computing service providers have adopted Hadoop, the open source MapReduce implementation, to build their cloud computing platform. However, like all open distributed computing frameworks, MapReduce suffers from the service integrity assurance vulnerability: it takes merely one malicious worker to render the overall computation result useless. It is very important to efficiently detect the malicious workers in cloud computing environment. Existing solutions are not effective enough in defeating the malicious behaviour of non-collusive and collusive workers. In this paper, we focus on the mappers, which typically constitute the majority of workers. On the basis of the existing frameworks, we make the master manage the computing workers based on the security levels, and introduce the trusted verifier worker and caching mechanism. According to the system analysis, the service integrity assurance framework suggested in this paper is more efficient and accurate to detect the malicious workers in the cloud computing environment based on MapReduce.