Pair-wise comparisons of multiple models
Often research in judgment and decision making requires comparison of multiple competing models. Researchers invoke global measures such as the rate of correct predictions or the sum of squared (or absolute) deviations of the various models as part of this evaluation process. Reliance on such measures hides the (often very high) level of agreement between the predictions of the various models and does not highlight properly the relative performance of the competing models in those critical cases where they make distinct predictions. To address this important problem we propose the use of pair-wise comparisons of models to produce more informative and targeted comparisons of their performance, and we illustrate this procedure with data from two recently published papers. We use Multidimensional Scaling of these comparisons to map the competing models. We also demonstrate how intransitive cycles of pair-wise model performance can signal that certain models perform better for a given subset of decision problems.
Volume (Year): 6 (2011)
Issue (Month): 8 (December)
|Contact details of provider:|| |
References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Craig R. Fox & Liat Hadar, 2006. ""Decisions from experience" = sampling error + prospect theory: Reconsidering Hertwig, Barron, Weber & Erev (2004)," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 1, pages 159161-1591, November.
- Benjamin E. Hilbig, 2008. "One-reason decision making in risky choice? A closer look at the priority heuristic," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 3(6), pages 457-462, August.
- Erev, Ido & Roth, Alvin E. & Slonim, Robert L. & Barron, Greg, 2002. "Predictive value and the usefulness of game theoretic models," International Journal of Forecasting, Elsevier, vol. 18(3), pages 359-368.
- Tversky, Amos & Kahneman, Daniel, 1992. "Advances in Prospect Theory: Cumulative Representation of Uncertainty," Journal of Risk and Uncertainty, Springer, vol. 5(4), pages 297-323, October.
When requesting a correction, please mention this item's handle: RePEc:jdm:journl:v:6:y:2011:i:8:p:821-831. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Jonathan Baron)
If references are entirely missing, you can add them using this form.