An Incentive Mechanism to Elicit Truthful Opinions for Crowdsourced Multiple Choice Consensus Tasks

Crowdsourcing is a rapidly growing technology to harness human intelligence to solve problems that are not suitable for automation. It is especially suitable for consensus tasks which collect opinions from human workers to gain insight into real-world phenomena. In these tasks, motivating workers to provide their truthful opinions is a challenging problem. Most existing incentive mechanisms proposed to address this problem assume that workers share a common prior. However, this assumption is not always valid in practice, resulting in that the existing approaches may discourage workers to provide their truthful opinions. In this paper, we propose a novel incentive mechanism -- iMET -- to elicit truthful opinions for crowdsourced multiple-choice consensus tasks. By incorporating the concepts of worker credibility and similarity, iMET rewards workers for providing truthful opinions without assuming a common prior. Through extensive simulations on the basis of a collected real-world dataset, iMET has been demonstrated to outperform another two widely used incentive mechanisms in eliciting truthful opinions, especially when diverse truthful opinions are held by workers.