On the use of the analytic hierarchy process in the aggregation of expert judgments

Expert judgments are involved in many aspects of scientific research, either formally or informally. In order to combine the different opinions elicited, simple aggregation methods have often been used with the result that expert biases, interexpert dependencies and other factors which might affect the judgments of the experts are often ignored. A more comprehensive approach, based on the analytic hierarchy process, is proposed in this paper to account for the large variety of factors influencing the experts. A structured hierarchy is constructed to decompose the overall problem in the elementary factors that can influence the expertʹs judgements. The importance of the different elements of the hierarchy is then assessed by pairwise comparison. The overall approach is simple, presents a systematic character and offeres a good degree of flexibility. The approach provides the decision maker with a tool to quantitatively measure the significance of the judgments provided by the different experts involved in the elicitation. The resulting values can be used as weights in an aggregation scheme such as, for example, the simple weighted averaging scheme. Two applications of the approach are presented with reference to case studies of formal expert judgment elicitation previously analyzed in literature: the elicitation of the pressure increment in the containment building of the Sequoyah nuclear power plant following reactor vessel breach, and the prediction of the future changes in precipitation in the vicinity of Yucca Mountain.