Creating a consensus ranking of proposals from reviewers’ partial ordinal rankings

Peer review of research proposals and articles is an essential element in R&D processes worldwide. In most cases, each reviewer evaluates a small subset of the candidate proposals. The review board is then faced with the challenge of creating an overall “consensus” ranking on the basis of many partial rankings. In this paper we propose a branch-and-bound model to support the construction of an aggregate ranking from the partial rankings provided by the reviewers. In a recent paper we proposed ways to allocate proposals to reviewers so as to achieve the maximum possible overlap among the subsets of proposals allocated to different reviewers. Here, we develop a special branch-and-bound algorithm that utilizes the overlap generated through our earlier methods to enable discrimination in ranking the competing proposals. The effectiveness and efficiency of the algorithm is demonstrated with small numerical examples and tested through an extensive simulation experiment.