Incentive mechanism and protocol design for crowdsourcing systems

Crowdsourcing systems such as Amazon Mechanical Turk, Yahoo!Answers, and Google Helpouts have attracted extensive attention over the past few years. In a crowdsourcing system, a large group of “workers” solve the tasks outsourced by “requesters”. To make a crowdsourcing system sustainable, it is vital to attract users (both requesters and workers) to participate, and incentivize high-quality solutions. To achieve this objective, we design an effective incentive mechanism and reputation protocol. Our design incorporates various important elements of a crowdsourcing system such as workers having heterogeneous skill sets (i.e., some are “experts” while others are “novices”), and task assignment process, rating system, etc. Our incentive mechanism is composed of a rating system and a reward dividing scheme, and requires the system administrator to divide the reward based on requesters´ rating on the solution quality. We derive the minimum reward needed so that “expert” workers are guaranteed provide high-quality solutions. We show that “novice” workers provide low-quality solutions, and our reputation protocol eliminates this undesirable behavior by tracking a worker´s solution history and penalizing him when his reputation is poor. We apply repeated game-theoretic frameworks to quantify the impact of this reputation protocol on requesters´ cost in guaranteeing high quality solutions.