IDEAS home Printed from https://ideas.repec.org/a/inm/ordeca/v4y2007i2p66-75.html
   My bibliography  Save this article

The Parimutuel Kelly Probability Scoring Rule

Author

Listed:
  • David J. Johnstone

    (School of Business, University of Sydney, NSW 2006, Australia)

Abstract

Kilgour and Gerchak (Kilgour, D. M., Y. Gerchak. 2004. Elicitation of probabilities using competitive scoring rules. Decision Anal. 2 108--113) introduce a competitive probability scoring rule designed to reward forecasters for their accuracy relative to rival forecasters. The rule proposed is both proper (in the usual sense of scoring rules) and self-financing in that forecasters share a preset reward pool that can be fixed at net zero. This paper elaborates on the Kilgour-Gerchak forecasting competition by demonstrating its possible analogy with forecasters making “Kelly bets” in an exogenous fixed-odds betting market. The construction of forecasters as Kelly (log utility) bettors leads to a modification of the Kilgour-Gerchak rule, called the parimutuel Kelly competitive scoring rule. Unlike the Kilgour-Gerchak score, the parimutuel Kelly score is not proper. Its appeal, however, is that it emulates a prediction market formed within a cohort of forecasters, as if they engage in a closed betting tournament where each attempts to accumulate maximum possible wealth over the long run and ruin all others. Within such a competition, near honesty is a Nash equilibrium forecasting strategy under realistic conditions.

Suggested Citation

  • David J. Johnstone, 2007. "The Parimutuel Kelly Probability Scoring Rule," Decision Analysis, INFORMS, vol. 4(2), pages 66-75, June.
  • Handle: RePEc:inm:ordeca:v:4:y:2007:i:2:p:66-75
    DOI: 10.1287/deca.1070.0091
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/deca.1070.0091
    Download Restriction: no

    File URL: https://libkey.io/10.1287/deca.1070.0091?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Kenneth C. Lichtendahl, Jr. & Robert L. Winkler, 2007. "Probability Elicitation, Scoring Rules, and Competition Among Forecasters," Management Science, INFORMS, vol. 53(11), pages 1745-1755, November.
    2. Blume, Lawrence E. & Easley, David, 2002. "Optimality and Natural Selection in Markets," Journal of Economic Theory, Elsevier, vol. 107(1), pages 95-135, November.
    3. L. C. MacLean & W. T. Ziemba & G. Blazenko, 1992. "Growth Versus Security in Dynamic Investment Analysis," Management Science, INFORMS, vol. 38(11), pages 1562-1585, November.
    4. Lawrence Blume & David Easley, 2006. "If You're so Smart, why Aren't You Rich? Belief Selection in Complete and Incomplete Markets," Econometrica, Econometric Society, vol. 74(4), pages 929-966, July.
    5. Rubinstein, Mark, 1976. "The Strong Case for the Generalized Logarithmic Utility Model as the Premier Model of Financial Markets," Journal of Finance, American Finance Association, vol. 31(2), pages 551-571, May.
    6. Hakansson, Nils H., 1971. "Capital Growth and the Mean-Variance Approach to Portfolio Selection," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 6(1), pages 517-557, January.
    7. Robert T. Clemen & Robert L. Winkler, 1999. "Combining Probability Distributions From Experts in Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 19(2), pages 187-203, April.
    8. R. Winkler & Javier Muñoz & José Cervera & José Bernardo & Gail Blattenberger & Joseph Kadane & Dennis Lindley & Allan Murphy & Robert Oliver & David Ríos-Insua, 1996. "Scoring rules and the evaluation of probabilities," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 5(1), pages 1-60, June.
    9. Donald C. Aucamp, 1993. "On the Extensive Number of Plays to Achieve Superior Performance with the Geometric Mean Strategy," Management Science, INFORMS, vol. 39(9), pages 1163-1172, September.
    10. Robert F. Nau, 1985. "Should Scoring Rules be "Effective"?," Management Science, INFORMS, vol. 31(5), pages 527-535, May.
    11. Harry M. Markowitz, 2011. "Investment for the Long Run: New Evidence for an Old Rule," World Scientific Book Chapters, in: Leonard C MacLean & Edward O Thorp & William T Ziemba (ed.), THE KELLY CAPITAL GROWTH INVESTMENT CRITERION THEORY and PRACTICE, chapter 35, pages 495-508, World Scientific Publishing Co. Pte. Ltd..
    12. Lopez, Jose A, 2001. "Evaluating the Predictive Accuracy of Volatility Models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 20(2), pages 87-109, March.
    13. Leonard MacLean & William Ziemba, 1999. "Growth versus security tradeoffs indynamic investment analysis," Annals of Operations Research, Springer, vol. 85(0), pages 193-225, January.
    14. Michael P. Clements, 2004. "Evaluating the Bank of England Density Forecasts of Inflation," Economic Journal, Royal Economic Society, vol. 114(498), pages 844-866, October.
    15. Robert T. Clemen, 1987. "Combining Overlapping Information," Management Science, INFORMS, vol. 33(3), pages 373-380, March.
    16. Yuming Li, 1993. "Growth-Security Investment Strategy for Long and Short Runs," Management Science, INFORMS, vol. 39(8), pages 915-924, August.
    17. MacLean, Leonard C. & Sanegre, Rafael & Zhao, Yonggan & Ziemba, William T., 2004. "Capital growth with security," Journal of Economic Dynamics and Control, Elsevier, vol. 28(5), pages 937-954, February.
    18. D. Marc Kilgour & Yigal Gerchak, 2004. "Elicitation of Probabilities Using Competitive Scoring Rules," Decision Analysis, INFORMS, vol. 1(2), pages 108-113, June.
    19. Robert L. Winkler & Robert T. Clemen, 2004. "Multiple Experts vs. Multiple Methods: Combining Correlation Assessments," Decision Analysis, INFORMS, vol. 1(3), pages 167-176, September.
    20. Blume, Lawrence & Easley, David, 1992. "Evolution and market behavior," Journal of Economic Theory, Elsevier, vol. 58(1), pages 9-40, October.
    21. Robert T. Clemen, 1986. "Calibration and the Aggregation of Probabilities," Management Science, INFORMS, vol. 32(3), pages 312-314, March.
    22. Murphy, Allan H. & Winkler, Robert L., 1992. "Diagnostic verification of probability forecasts," International Journal of Forecasting, Elsevier, vol. 7(4), pages 435-455, March.
    23. Steven D. Levitt, 2004. "Why are gambling markets organised so differently from financial markets?," Economic Journal, Royal Economic Society, vol. 114(495), pages 223-246, April.
    24. Roll, Richard, 1973. "Evidence on the "Growth-Optimum" Model," Journal of Finance, American Finance Association, vol. 28(3), pages 551-566, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. L. Robin Keller, 2009. "From the Editor..," Decision Analysis, INFORMS, vol. 6(1), pages 1-3, March.
    2. L. Robin Keller & Ali Abbas & Manel Baucells & Vicki M. Bier & David Budescu & John C. Butler & Philippe Delquié & Jason R. W. Merrick & Ahti Salo & George Wu, 2010. "From the Editors..," Decision Analysis, INFORMS, vol. 7(4), pages 327-330, December.
      • L. Robin Keller & Manel Baucells & Kevin F. McCardle & Gregory S. Parnell & Ahti Salo, 2007. "From the Editors..," Decision Analysis, INFORMS, vol. 4(4), pages 173-175, December.
      • L. Robin Keller & Manel Baucells & John C. Butler & Philippe Delquié & Jason R. W. Merrick & Gregory S. Parnell & Ahti Salo, 2008. "From the Editors..," Decision Analysis, INFORMS, vol. 5(4), pages 173-176, December.
      • L. Robin Keller & Manel Baucells & John C. Butler & Philippe Delquié & Jason R. W. Merrick & Gregory S. Parnell & Ahti Salo, 2009. "From the Editors ..," Decision Analysis, INFORMS, vol. 6(4), pages 199-201, December.
    3. L. Robin Keller, 2008. "From the Editor..," Decision Analysis, INFORMS, vol. 5(1), pages 1-4, March.
    4. Grant, Andrew & Johnstone, David, 2010. "Finding profitable forecast combinations using probability scoring rules," International Journal of Forecasting, Elsevier, vol. 26(3), pages 498-510, July.
    5. L. Robin Keller & Kelly M. Kophazi, 2012. "From the Editors ---Copulas, Group Preferences, Multilevel Defenders, Sharing Rewards, and Communicating Analytics," Decision Analysis, INFORMS, vol. 9(3), pages 213-218, September.
    6. L. Robin Keller & Kelly M. Kophazi, 2011. "From the Editors---Deterrence, Multiattribute Utility, and Probability and Bayes' Updating," Decision Analysis, INFORMS, vol. 8(2), pages 83-87, June.
    7. David J. Johnstone & Victor Richmond R. Jose & Robert L. Winkler, 2011. "Tailored Scoring Rules for Probabilities," Decision Analysis, INFORMS, vol. 8(4), pages 256-268, December.
    8. L. Robin Keller & Ali Abbas & J. Eric Bickel & Vicki M. Bier & David V. Budescu & John C. Butler & Philippe Delquié & Kenneth C. Lichtendahl & Jason R. W. Merrick & Ahti Salo & George Wu, 2011. "From the Editors ---Probability Scoring Rules, Ambiguity, Multiattribute Terrorist Utility, and Sensitivity Analysis," Decision Analysis, INFORMS, vol. 8(4), pages 251-255, December.
    9. D. J. Johnstone, 2011. "Economic Interpretation of Probabilities Estimated by Maximum Likelihood or Score," Management Science, INFORMS, vol. 57(2), pages 308-314, February.
    10. D. J. Johnstone & S. Jones & V. R. R. Jose & M. Peat, 2013. "Measures of the economic value of probabilities of bankruptcy," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(3), pages 635-653, June.
    11. Edgar C. Merkle & Mark Steyvers, 2013. "Choosing a Strictly Proper Scoring Rule," Decision Analysis, INFORMS, vol. 10(4), pages 292-304, December.
    12. L. Robin Keller & Ali Abbas & J. Eric Bickel & Vicki M. Bier & David V. Budescu & John C. Butler & Enrico Diecidue & Robin L. Dillon-Merrill & Raimo P. Hämäläinen & Kenneth C. Lichtendahl & Jason R. W, 2012. "From the Editors ---Brainstorming, Multiplicative Utilities, Partial Information on Probabilities or Outcomes, and Regulatory Focus," Decision Analysis, INFORMS, vol. 9(4), pages 297-302, December.
    13. Jianqing Chen & Lizhen Xu & Andrew Whinston, 2010. "Managing Project Failure Risk Through Contingent Contracts in Procurement Auctions," Decision Analysis, INFORMS, vol. 7(1), pages 23-39, March.
    14. Lessmann, Stefan & Sung, Ming-Chien & Johnson, Johnnie E.V. & Ma, Tiejun, 2012. "A new methodology for generating and combining statistical forecasting models to enhance competitive event prediction," European Journal of Operational Research, Elsevier, vol. 218(1), pages 163-174.
    15. L. Robin Keller, 2011. "From the Editor ---Multiattribute and Intertemporal Preferences, Probability, and Stochastic Processes: Models and Assessment," Decision Analysis, INFORMS, vol. 8(3), pages 165-169, September.
    16. Rakesh K. Sarin, 2013. "From the Editor —Optimal Betting, Reducing Unnecessary Mammography in Breast Cancer Diagnosis, Product Line Design, and Value of Information," Decision Analysis, INFORMS, vol. 10(3), pages 187-188, September.
    17. Rakesh K. Sarin & L. Robin Keller, 2013. "From the Editors: Probability Approximations, Anti-Terrorism Strategy, and Bull's-Eye Display for Performance Feedback," Decision Analysis, INFORMS, vol. 10(1), pages 1-5, March.
    18. Ali E. Abbas, 2009. "A Kullback-Leibler View of Linear and Log-Linear Pools," Decision Analysis, INFORMS, vol. 6(1), pages 25-37, March.
    19. Lambert, Nicolas S. & Langford, John & Wortman Vaughan, Jennifer & Chen, Yiling & Reeves, Daniel M. & Shoham, Yoav & Pennock, David M., 2015. "An axiomatic characterization of wagering mechanisms," Journal of Economic Theory, Elsevier, vol. 156(C), pages 389-416.
    20. L. Robin Keller, 2011. "Investment and Defense Strategies, Heuristics, and Games: From the Editor ..," Decision Analysis, INFORMS, vol. 8(1), pages 1-3, March.
    21. Brendan Daley & Ruoyu Wang, 2018. "When to Release Feedback in a Dynamic Tournament," Decision Analysis, INFORMS, vol. 15(1), pages 11-26, March.
    22. David Johnstone & Stewart Jones & Oliver Jones & Steve Tulig, 2021. "Scoring Probability Forecasts by a User’s Bets Against a Market Consensus," Decision Analysis, INFORMS, vol. 18(3), pages 169-184, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David Johnstone, 2007. "Economic Darwinism: Who has the Best Probabilities?," Theory and Decision, Springer, vol. 62(1), pages 47-96, February.
    2. Grant, Andrew & Johnstone, David, 2010. "Finding profitable forecast combinations using probability scoring rules," International Journal of Forecasting, Elsevier, vol. 26(3), pages 498-510, July.
    3. Ziemba, William, 2016. "A response to Professor Paul A. Samuelson's objections to Kelly capital growth investing," LSE Research Online Documents on Economics 119002, London School of Economics and Political Science, LSE Library.
    4. David J Johnstone, 2023. "Capital budgeting and Kelly betting," Australian Journal of Management, Australian School of Business, vol. 48(3), pages 625-651, August.
    5. David Johnstone & Stewart Jones & Oliver Jones & Steve Tulig, 2021. "Scoring Probability Forecasts by a User’s Bets Against a Market Consensus," Decision Analysis, INFORMS, vol. 18(3), pages 169-184, September.
    6. D. J. Johnstone, 2021. "Accounting information, disclosure, and expected utility: Do investors really abhor uncertainty?," Journal of Business Finance & Accounting, Wiley Blackwell, vol. 48(1-2), pages 3-35, January.
    7. D. J. Johnstone & S. Jones & V. R. R. Jose & M. Peat, 2013. "Measures of the economic value of probabilities of bankruptcy," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(3), pages 635-653, June.
    8. Rakesh K. Sarin, 2013. "From the Editor ---Median Aggregation, Scoring Rules, Expert Forecasts, Choices with Binary Attributes, Portfolio with Dependent Projects, and Information Security," Decision Analysis, INFORMS, vol. 10(4), pages 277-278, December.
    9. G. Bottazzi & D. Giachini, 2019. "Far from the madding crowd: collective wisdom in prediction markets," Quantitative Finance, Taylor & Francis Journals, vol. 19(9), pages 1461-1471, September.
    10. Robert L. Winkler & Yael Grushka-Cockayne & Kenneth C. Lichtendahl Jr. & Victor Richmond R. Jose, 2019. "Probability Forecasts and Their Combination: A Research Perspective," Decision Analysis, INFORMS, vol. 16(4), pages 239-260, December.
    11. Lambert, Nicolas S. & Langford, John & Wortman Vaughan, Jennifer & Chen, Yiling & Reeves, Daniel M. & Shoham, Yoav & Pennock, David M., 2015. "An axiomatic characterization of wagering mechanisms," Journal of Economic Theory, Elsevier, vol. 156(C), pages 389-416.
    12. Mikhail Zhitlukhin, 2022. "A continuous-time asset market game with short-lived assets," Finance and Stochastics, Springer, vol. 26(3), pages 587-630, July.
    13. Sonntag, Dominik, 2018. "Die Theorie der fairen geometrischen Rendite [The Theory of Fair Geometric Returns]," MPRA Paper 87082, University Library of Munich, Germany.
    14. Gneiting, Tilmann, 2011. "Quantiles as optimal point forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 197-207, April.
    15. Jarrod Wilcox & Frank Fabozzi, 2009. "A Discretionary Wealth Approach to Investment Policy," Yale School of Management Working Papers amz2434, Yale School of Management.
    16. Enrique Urbano Arellano & Xinyang Wang, 2023. "Social Learning of General Rules," Papers 2310.15861, arXiv.org.
    17. J. Eric Bickel, 2007. "Some Comparisons among Quadratic, Spherical, and Logarithmic Scoring Rules," Decision Analysis, INFORMS, vol. 4(2), pages 49-65, June.
    18. Levy, Moshe & Nir, Adi Rizansky, 2012. "The utility of health and wealth," Journal of Health Economics, Elsevier, vol. 31(2), pages 379-392.
    19. Jarrod Wilcox & Frank Fabozzi, 2009. "A Discretionary Wealth Approach to Investment Policy," Yale School of Management Working Papers amz2434, Yale School of Management.
    20. Gneiting, Tilmann, 2011. "Quantiles as optimal point forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 197-207.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ordeca:v:4:y:2007:i:2:p:66-75. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.