IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v29y1983i4p447-454.html
   My bibliography  Save this article

Effective Scoring Rules for Probabilistic Forecasts

Author

Listed:
  • Daniel Friedman

    (University of California at Los Angeles)

Abstract

This paper studies the use of a scoring rule for the elicitation of forecasts in the form of probability distributions and for the subsequent evaluation of such forecasts. Given a metric (distance function) on a space of probability distributions, a scoring rule is said to be effective if the forecaster's expected score is a strictly decreasing function of the distance between the elicited and "true" distributions. Two simple, well-known rules (the spherical and the quadratic) are shown to be effective with respect to suitable metrics. Examples and a practical application (in Foreign Exchange rate forecasting) are also provided.

Suggested Citation

  • Daniel Friedman, 1983. "Effective Scoring Rules for Probabilistic Forecasts," Management Science, INFORMS, vol. 29(4), pages 447-454, April.
    • Handle: RePEc:inm:ormnsc:v:29:y:1983:i:4:p:447-454
    DOI: 10.1287/mnsc.29.4.447
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mnsc.29.4.447
    Download Restriction: no
    File URL: https://libkey.io/10.1287/mnsc.29.4.447?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Onkal, Dilek & Muradoglu, Gulnur, 1995. "Effects of feedback on probabilistic forecasts of stock prices," International Journal of Forecasting, Elsevier, vol. 11(2), pages 307-319, June.
    2. Norde, Henk & Voorneveld, Mark, 2019. "Feasible best-response correspondences and quadratic scoring rules," SSE Working Paper Series in Economics 2019:2, Stockholm School of Economics.
    3. Arthur Carvalho & Stanko Dimitrov & Kate Larson, 2018. "On proper scoring rules and cumulative prospect theory," EURO Journal on Decision Processes, Springer;EURO - The Association of European Operational Research Societies, vol. 6(3), pages 343-376, November.
    4. Leonard Smith & Emma Suckling & Erica Thompson & Trevor Maynard & Hailiang Du, 2015. "Towards improving the framework for probabilistic forecast evaluation," Climatic Change, Springer, vol. 132(1), pages 31-45, September.
    5. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    6. Papakonstantinou, A. & Bogetoft, P., 2013. "Crowd-sourcing with uncertain quality - an auction approach," MPRA Paper 46055, University Library of Munich, Germany.
    7. Victor Richmond R. Jose & Robert F. Nau & Robert L. Winkler, 2008. "Scoring Rules, Generalized Entropy, and Utility Maximization," Operations Research, INFORMS, vol. 56(5), pages 1146-1157, October.
    8. Atanasios Mitropoulos, 2001. "On the Measurement of the Predictive Success of Learning Theories in Repeated Games," Experimental 0110001, University Library of Munich, Germany.
    9. Wheatcroft, Edward, 2021. "Evaluating probabilistic forecasts of football matches: the case against the ranked probability score," LSE Research Online Documents on Economics 111494, London School of Economics and Political Science, LSE Library.
    10. Radosveta Ivanova-Stenzel & Timothy C. Salmon, 2004. "Bidder Preferences among Auction Institutions," Economic Inquiry, Western Economic Association International, vol. 42(2), pages 223-236, April.
    11. Papakonstantinou, Athanasios & Bogetoft, Peter, 2017. "Multi-dimensional procurement auction under uncertain and asymmetric information," European Journal of Operational Research, Elsevier, vol. 258(3), pages 1171-1180.
    12. Lambert, Nicolas S. & Langford, John & Wortman Vaughan, Jennifer & Chen, Yiling & Reeves, Daniel M. & Shoham, Yoav & Pennock, David M., 2015. "An axiomatic characterization of wagering mechanisms," Journal of Economic Theory, Elsevier, vol. 156(C), pages 389-416.
    13. J. Eric Bickel, 2007. "Some Comparisons among Quadratic, Spherical, and Logarithmic Scoring Rules," Decision Analysis, INFORMS, vol. 4(2), pages 49-65, June.
    14. Plott, Charles R. & Salmon, Timothy C., 2004. "The simultaneous, ascending auction: dynamics of price adjustment in experiments and in the UK3G spectrum auction," Journal of Economic Behavior & Organization, Elsevier, vol. 53(3), pages 353-383, March.
    15. Nolan Miller & Paul Resnick & Richard Zeckhauser, 2005. "Eliciting Informative Feedback: The Peer-Prediction Method," Management Science, INFORMS, vol. 51(9), pages 1359-1373, September.
    16. Fang, Fang & Stinchcombe, Maxwell B. & Whinston, Andrew B., 2010. "Proper scoring rules with arbitrary value functions," Journal of Mathematical Economics, Elsevier, vol. 46(6), pages 1200-1210, November.
    17. D. Johnstone, 2007. "The Value of a Probability Forecast from Portfolio Theory," Theory and Decision, Springer, vol. 63(2), pages 153-203, September.
    18. Wheatcroft Edward, 2021. "Evaluating probabilistic forecasts of football matches: the case against the ranked probability score," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 17(4), pages 273-287, December.
    19. Tang, Fang-Fang, 2003. "A comparative study on learning in a normal form game experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 50(3), pages 385-390, March.
    20. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    21. Reinhard Selten, 1998. "Axiomatic Characterization of the Quadratic Scoring Rule," Experimental Economics, Springer;Economic Science Association, vol. 1(1), pages 43-61, June.
    22. Alexandre Vasconcelos Lima & Rogério Boueri Miranda & Mathias Schneid Tessmann, 2022. "Evaluation of the Future Price of Brazilian Commodities as a Predictor of the Price of the Spot Market," International Journal of Economics and Finance, Canadian Center of Science and Education, vol. 14(4), pages 1-51, April.

    More about this item

    Keywords

    forecasting; Delphi technique;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:29:y:1983:i:4:p:447-454. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.