Towards an Objective Assessment of Centrality Measures in Reputation Systems

Reputation systems facilitate cooperation in open environments. Centrality-based reputation systems use centrality measures to rank individuals, i.e., compute their reputation based on feedback. However, despite the large number of such measures that have been proposed recently, an objective comparison of their effectiveness in reputation systems is missing. This paper features such a comparison. Based on a reference ranking and a behavior model, we generate the underlying feedback data explicitly. While the behavior model may be arbitrarily complex, simple ones already provide interesting insights. We have observed that existing distance measures for rankings have certain limitations in our context, and we propose a new, more appropriate one. Further, most centrality measures cannot be directly applied on the feedback data. While one can transform the data to another representation, the impact on the quality of the ranking is unclear. One of our findings is that the transformation techniques and their parameter values are indeed important, more than we had anticipated.