IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v53y2007i11p1745-1755.html
   My bibliography  Save this article

Probability Elicitation, Scoring Rules, and Competition Among Forecasters

Author

Listed:
  • Kenneth C. Lichtendahl, Jr.

    (Darden School of Business, University of Virginia, Charlottesville, Virginia 22906)

  • Robert L. Winkler

    (Fuqua School of Business, Duke University, Durham, North Carolina 27708)

Abstract

Probability forecasters who are rewarded via a proper scoring rule may care not only about the score, but also about their performance relative to other forecasters. We model this type of preference and show that a competitive forecaster who wants to do better than another forecaster typically should report more extreme probabilities, exaggerating toward zero or one. We consider a competitive forecaster's best response to truthful reporting and also investigate equilibrium reporting functions in the case where another forecaster also cares about relative performance. We show how a decision maker can revise probabilities of an event after receiving reported probabilities from competitive forecasters and note that the strategy of exaggerating probabilities can make well-calibrated forecasters (and a decision maker who takes their reported probabilities at face value) appear to be overconfident. However, a decision maker who adjusts appropriately for the misrepresentation of probabilities by one or more forecasters can still be well calibrated. Finally, to try to overcome the forecasters' competitive instincts and induce cooperative behavior, we develop the notion of joint scoring rules based on business sharing and show that these scoring rules are strictly proper.

Suggested Citation

  • Kenneth C. Lichtendahl, Jr. & Robert L. Winkler, 2007. "Probability Elicitation, Scoring Rules, and Competition Among Forecasters," Management Science, INFORMS, vol. 53(11), pages 1745-1755, November.
  • Handle: RePEc:inm:ormnsc:v:53:y:2007:i:11:p:1745-1755
    DOI: 10.1287/mnsc.1070.0729
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mnsc.1070.0729
    Download Restriction: no

    File URL: https://libkey.io/10.1287/mnsc.1070.0729?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Marshall Robert C. & Meurer Michael J. & Richard Jean-Francois & Stromquist Walter, 1994. "Numerical Analysis of Asymmetric First Price Auctions," Games and Economic Behavior, Elsevier, vol. 7(2), pages 193-220, September.
    2. A. Dawid & M. DeGroot & J. Mortera & R. Cooke & S. French & C. Genest & M. Schervish & D. Lindley & K. McConway & R. Winkler, 1995. "Coherent combination of experts' opinions," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 4(2), pages 263-313, December.
    3. R. Winkler & Javier Muñoz & José Cervera & José Bernardo & Gail Blattenberger & Joseph Kadane & Dennis Lindley & Allan Murphy & Robert Oliver & David Ríos-Insua, 1996. "Scoring rules and the evaluation of probabilities," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 5(1), pages 1-60, June.
    4. Lebrun, Bernard, 1999. "First Price Auctions in the Asymmetric N Bidder Case," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 40(1), pages 125-142, February.
    5. Peter A. Morris, 1974. "Decision Analysis Expert Use," Management Science, INFORMS, vol. 20(9), pages 1233-1241, May.
    6. Anil Gaba & Ilia Tsetlin & Robert L. Winkler, 2004. "Modifying Variability and Correlations in Winner-Take-All Contests," Operations Research, INFORMS, vol. 52(3), pages 384-395, June.
    7. D. Marc Kilgour & Yigal Gerchak, 2004. "Elicitation of Probabilities Using Competitive Scoring Rules," Decision Analysis, INFORMS, vol. 1(2), pages 108-113, June.
    8. Pfeifer, Phillip E., 1994. "Are We Overconfident in the Belief That Probability Forecasters Are Overconfident?," Organizational Behavior and Human Decision Processes, Elsevier, vol. 58(2), pages 203-213, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fibich, Gadi & Gavish, Nir, 2011. "Numerical simulations of asymmetric first-price auctions," Games and Economic Behavior, Elsevier, vol. 73(2), pages 479-495.
    2. Leandro Arozamena & Estelle Cantillon, 2004. "Investment Incentives in Procurement Auctions," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 71(1), pages 1-18.
    3. Rene Kirkegaard, 2005. "A Simple Approach to Analyzing Asymmetric First Price Auctions," Working Papers 0504, Brock University, Department of Economics, revised Nov 2005.
    4. Biran, Omer & Forges, Françoise, 2011. "Core-stable rings in auctions with independent private values," Games and Economic Behavior, Elsevier, vol. 73(1), pages 52-64, September.
    5. Kotowski, Maciej H., 2018. "On asymmetric reserve prices," Theoretical Economics, Econometric Society, vol. 13(1), January.
    6. Lebrun, Bernard, 2006. "Uniqueness of the equilibrium in first-price auctions," Games and Economic Behavior, Elsevier, vol. 55(1), pages 131-151, April.
    7. Lebrun, Bernard, 1998. "Comparative Statics in First Price Auctions," Games and Economic Behavior, Elsevier, vol. 25(1), pages 97-110, October.
    8. Brendan Daley & Ruoyu Wang, 2018. "When to Release Feedback in a Dynamic Tournament," Decision Analysis, INFORMS, vol. 15(1), pages 11-26, March.
    9. de Frutos, M. Angeles, 2000. "Asymmetric Price-Benefits Auctions," Games and Economic Behavior, Elsevier, vol. 33(1), pages 48-71, October.
    10. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    11. Cantillon, Estelle, 2008. "The effect of bidders' asymmetries on expected revenue in auctions," Games and Economic Behavior, Elsevier, vol. 62(1), pages 1-25, January.
    12. Wayne-Roy Gayle & Jean Richard, 2008. "Numerical Solutions of Asymmetric, First-Price, Independent Private Values Auctions," Computational Economics, Springer;Society for Computational Economics, vol. 32(3), pages 245-278, October.
    13. David J. Johnstone, 2007. "The Parimutuel Kelly Probability Scoring Rule," Decision Analysis, INFORMS, vol. 4(2), pages 66-75, June.
    14. Lee, Joon-Suk, 2008. "Favoritism in asymmetric procurement auctions," International Journal of Industrial Organization, Elsevier, vol. 26(6), pages 1407-1424, November.
    15. Jofre-Bonet, Mireia & Pesendorfer, Martin, 2014. "Optimal sequential auctions," International Journal of Industrial Organization, Elsevier, vol. 33(C), pages 61-71.
    16. Chernomaz, Kirill & Levin, Dan, 2012. "Efficiency and synergy in a multi-unit auction with and without package bidding: An experimental study," Games and Economic Behavior, Elsevier, vol. 76(2), pages 611-635.
    17. Timothy P. Hubbard & Rene Kirkegaard, 2015. "Asymmetric Auctions with More Than Two Bidders," Working Papers 1502, University of Guelph, Department of Economics and Finance.
    18. Li, Huagang & Riley, John G., 2007. "Auction choice," International Journal of Industrial Organization, Elsevier, vol. 25(6), pages 1269-1298, December.
    19. Ali E. Abbas, 2009. "A Kullback-Leibler View of Linear and Log-Linear Pools," Decision Analysis, INFORMS, vol. 6(1), pages 25-37, March.
    20. Roberto Burguet & Martin K. Perry, 2003. "Preferred Suppliers and Vertical Integration in Auction Market," Working Papers 74, Barcelona School of Economics.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:53:y:2007:i:11:p:1745-1755. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.