IDEAS home Printed from https://ideas.repec.org/a/inm/ordeca/v13y2016i4p223-242.html
   My bibliography  Save this article

An Overview of Applications of Proper Scoring Rules

Author

Listed:
  • Arthur Carvalho

    (Farmer School of Business, Miami University, Oxford, Ohio 45056)

Abstract

We present a study on the evolution of publications about applications of proper scoring rules. Specifically, we consider articles reporting the use of proper scoring rules when either measuring the accuracy of forecasts or for inducing honest reporting of private information within a certain context. Our analysis of a data set containing 201 articles published between 1950 and 2015 suggests that there has been a tremendous increase in the number of published articles about proper scoring rules over the years. Moreover, the weather/climate, prediction markets, psychology, and energy domains are the four most popular application areas. After providing some insights on how proper scoring rules are applied in different domains, we analyze the publication outlets where the articles in our data set were published. In this regard, we find that an increasing number of articles are now being published in conference proceedings related to artificial intelligence, as opposed to traditional academic journals. We conclude this review by suggesting that the wisdom-of-crowds phenomenon might be a driving force behind the recent popularity of proper scoring rules.

Suggested Citation

  • Arthur Carvalho, 2016. "An Overview of Applications of Proper Scoring Rules," Decision Analysis, INFORMS, vol. 13(4), pages 223-242, December.
    • Handle: RePEc:inm:ordeca:v:13:y:2016:i:4:p:223-242
    DOI: 10.1287/deca.2016.0337
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/deca.2016.0337
    Download Restriction: no
    File URL: https://libkey.io/10.1287/deca.2016.0337?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Norde, Henk & Voorneveld, Mark, 2019. "Feasible best-response correspondences and quadratic scoring rules," SSE Working Paper Series in Economics 2019:2, Stockholm School of Economics.
    2. Carroll, Gabriel, 2019. "Robust incentives for information acquisition," Journal of Economic Theory, Elsevier, vol. 181(C), pages 382-420.
    3. Menapace, Luisa & Raffaelli, Roberta, 2020. "Unraveling hypothetical bias in discrete choice experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 176(C), pages 416-430.
    4. Jared A. Beekman & Ronald F. A. Woodaman & Dennis M. Buede, 2020. "A Review of Probabilistic Opinion Pooling Algorithms with Application to Insider Threat Detection," Decision Analysis, INFORMS, vol. 17(1), pages 39-55, March.
    5. Matthew B. Welsh & Steve H. Begg, 2018. "More-or-less elicitation (MOLE): reducing bias in range estimation and forecasting," EURO Journal on Decision Processes, Springer;EURO - The Association of European Operational Research Societies, vol. 6(1), pages 171-212, June.
    6. Ronald Peeters & Leonard Wolk, 2019. "Elicitation of expectations using Colonel Blotto," Experimental Economics, Springer;Economic Science Association, vol. 22(1), pages 268-288, March.
    7. Merkle, Edgar C. & Steyvers, Mark & Mellers, Barbara & Tetlock, Philip E., 2017. "A neglected dimension of good forecasting judgment: The questions we choose also matter," International Journal of Forecasting, Elsevier, vol. 33(4), pages 817-832.
    8. Ming Tang & Huchang Liao, 2023. "Group Structure and Information Distribution on the Emergence of Collective Intelligence," Decision Analysis, INFORMS, vol. 20(2), pages 133-150, June.
    9. Tino Werner, 2022. "Elicitability of Instance and Object Ranking," Decision Analysis, INFORMS, vol. 19(2), pages 123-140, June.
    10. repec:cup:judgdm:v:13:y:2018:i:2:p:185-201 is not listed on IDEAS
    11. Robert L. Winkler & Yael Grushka-Cockayne & Kenneth C. Lichtendahl Jr. & Victor Richmond R. Jose, 2019. "Probability Forecasts and Their Combination: A Research Perspective," Decision Analysis, INFORMS, vol. 16(4), pages 239-260, December.
    12. Andrew Grant & David Johnstone & Oh Kang Kwon, 2019. "A Probability Scoring Rule for Simultaneous Events," Decision Analysis, INFORMS, vol. 16(4), pages 301-313, December.
    13. Zachary J. Smith & J. Eric Bickel, 2020. "Additive Scoring Rules for Discrete Sample Spaces," Decision Analysis, INFORMS, vol. 17(2), pages 115-133, June.
    14. Edgar C. Merkle & Robert Hartman, 2018. "Weighted Brier score decompositions for topically heterogenous forecasting tournaments," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 13(2), pages 185-201, March.
    15. David Johnstone & Stewart Jones & Oliver Jones & Steve Tulig, 2021. "Scoring Probability Forecasts by a User’s Bets Against a Market Consensus," Decision Analysis, INFORMS, vol. 18(3), pages 169-184, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ordeca:v:13:y:2016:i:4:p:223-242. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.