Combining Experts’ Judgments: Comparison of Algorithmic Methods using Synthetic Data
Expert judgment (or expert elicitation) is a formal process for eliciting judgments from subject-matter experts about the value of a decision-relevant quantity. Judgments in the form of subjective probability distributions are obtained from several experts, raising the question how best to combine information from multiple experts. A number of algorithmic approaches have been proposed, of which the most commonly employed is the equal-weight combination (the average of the experts’ distributions). We evaluate the properties of five combination methods (equal-weight, best-expert, performance, frequentist, and copula) using simulated expert-judgment data for which we know the process generating the experts’ distributions. We examine cases in which two well-calibrated experts are of equal or unequal quality and their judgments are independent, positively or negatively dependent. In this setting, the copula, frequentist, and best-expert approaches perform better and the equal-weight combination method performs worse than the alternative approaches.
|Date of creation:||Mar 2012|
|Date of revision:|
|Publication status:||Published in Risk Analysis, vol.�33, n°1, janvier 2013, p.�109-120.|
|Contact details of provider:|| Phone: (+33) 5 61 12 86 23|
Web page: http://www.tse-fr.eu/
More information through EDIRC
When requesting a correction, please mention this item's handle: RePEc:tse:wpaper:25761. See general information about how to correct material in RePEc.
If references are entirely missing, you can add them using this form.