IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v52y2006i3p311-322.html
   My bibliography  Save this article

The Optimizer's Curse: Skepticism and Postdecision Surprise in Decision Analysis

Author

Listed:
  • James E. Smith

    (Fuqua School of Business, Duke University, Durham, North Carolina 27708)

  • Robert L. Winkler

    (Fuqua School of Business, Duke University, Durham, North Carolina 27708)

Abstract

Decision analysis produces measures of value such as expected net present values or expected utilities and ranks alternatives by these value estimates. Other optimization-based processes operate in a similar manner. With uncertainty and limited resources, an analysis is never perfect, so these value estimates are subject to error. We show that if we take these value estimates at face value and select accordingly, we should expect the value of the chosen alternative to be less than its estimate, even if the value estimates are unbiased. Thus, when comparing actual outcomes to value estimates, we should expect to be disappointed on average, not because of any inherent bias in the estimates themselves, but because of the optimization-based selection process. We call this phenomenon the optimizer's curse and argue that it is not well understood or appreciated in the decision analysis and management science communities. This curse may be a factor in creating skepticism in decision makers who review the results of an analysis. In this paper, we study the optimizer's curse and show that the resulting expected disappointment may be substantial. We then propose the use of Bayesian methods to adjust value estimates. These Bayesian methods can be viewed as disciplined skepticism and provide a method for avoiding this postdecision disappointment.

Suggested Citation

  • James E. Smith & Robert L. Winkler, 2006. "The Optimizer's Curse: Skepticism and Postdecision Surprise in Decision Analysis," Management Science, INFORMS, vol. 52(3), pages 311-322, March.
    • Handle: RePEc:inm:ormnsc:v:52:y:2006:i:3:p:311-322
    DOI: 10.1287/mnsc.1050.0451
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mnsc.1050.0451
    Download Restriction: no
    File URL: https://libkey.io/10.1287/mnsc.1050.0451?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Steven N. Tani, 1978. "A Perspective on Modeling in Decision Analysis," Management Science, INFORMS, vol. 24(14), pages 1500-1506, October.
    2. Porter, Robert H, 1995. "The Role of Information in U.S. Offshore Oil and Gas Lease Auctions," Econometrica, Econometric Society, vol. 63(1), pages 1-27, January.
    3. Brown, Stephen J, et al, 1992. "Survivorship Bias in Performance Studies," The Review of Financial Studies, Society for Financial Studies, vol. 5(4), pages 553-580.
    4. David E. Bell, 1985. "Disappointment in Decision Making Under Uncertainty," Operations Research, INFORMS, vol. 33(1), pages 1-27, February.
    5. Robert L. Winkler & Daniel G. Brooks, 1980. "Competitive Bidding with Dependent Value Estimates," Operations Research, INFORMS, vol. 28(3-part-i), pages 603-613, June.
    6. Peter A. Morris, 1974. "Decision Analysis Expert Use," Management Science, INFORMS, vol. 20(9), pages 1233-1241, May.
    7. Robert T. Clemen & Robert C. Kwit, 2001. "The Value of Decision Analysis at Eastman Kodak Company, 1990-1999," Interfaces, INFORMS, vol. 31(5), pages 74-92, October.
    8. Brown, Keith C, 1974. "A Note on the Apparent Bias of Net Revenue Estimates for Capital Investment Projects," Journal of Finance, American Finance Association, vol. 29(4), pages 1215-1216, September.
    9. Robert C. Nickerson & Dean W. Boyd, 1980. "The Use and Value of Models in Decision Analysis," Operations Research, INFORMS, vol. 28(1), pages 139-155, February.
    10. Dennis V. Lindley, 1986. "The Reconciliation of Decision Analyses," Operations Research, INFORMS, vol. 34(2), pages 289-295, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stephen Brown & William Goetzmann & Bing Liang & Christopher Schwarz, 2008. "Mandatory Disclosure and Operational Risk: Evidence from Hedge Fund Registration," Journal of Finance, American Finance Association, vol. 63(6), pages 2785-2815, December.
    2. Vilkkumaa, Eeva & Liesiö, Juuso & Salo, Ahti, 2014. "Optimal strategies for selecting project portfolios using uncertain value estimates," European Journal of Operational Research, Elsevier, vol. 233(3), pages 772-783.
    3. Butt, Prof. Khursheed A & Pandow, Bilal Ahmad, 2013. "An analysis into the Stock Selectivity skill of Indian Fund Managers," MPRA Paper 83500, University Library of Munich, Germany, revised 2013.
    4. A. Colin Cameron & Anthony D. Hall, 2003. "A Survival Analysis of Australian Equity Mutual Funds," Australian Journal of Management, Australian School of Business, vol. 28(2), pages 209-226, September.
    5. Colson, Gérard, 1993. "Prenons-nous assez de risque dans les théories du risque?," L'Actualité Economique, Société Canadienne de Science Economique, vol. 69(1), pages 111-141, mars.
    6. Chateauneuf, Alain & Eichberger, Jurgen & Grant, Simon, 2007. "Choice under uncertainty with the best and worst in mind: Neo-additive capacities," Journal of Economic Theory, Elsevier, vol. 137(1), pages 538-567, November.
    7. Jean-Michel Benkert, 2015. "Bilateral trade with loss-averse agents," ECON - Working Papers 188, Department of Economics - University of Zurich, revised Jul 2022.
    8. repec:cup:judgdm:v:1:y:2006:i::p:162-173 is not listed on IDEAS
    9. Karle, Heiko & Schumacher, Heiner & Vølund, Rune, 2023. "Consumer loss aversion and scale-dependent psychological switching costs," Games and Economic Behavior, Elsevier, vol. 138(C), pages 214-237.
    10. Terrence Hallahan & Robert Faff, 2001. "Induced persistence or reversals in fund performance?: the effect of survivorship bias," Applied Financial Economics, Taylor & Francis Journals, vol. 11(2), pages 119-126.
    11. van Dijk, Wilco W. & van der Pligt, Joop, 1997. "The Impact of Probability and Magnitude of Outcome on Disappointment and Elation," Organizational Behavior and Human Decision Processes, Elsevier, vol. 69(3), pages 277-284, March.
    12. Kalwij, Adriaan, 2018. "The effects of competition outcomes on health: Evidence from the lifespans of U.S. Olympic medalists," Economics & Human Biology, Elsevier, vol. 31(C), pages 276-286.
    13. Ian Tonks, 2005. "Performance Persistence of Pension-Fund Managers," The Journal of Business, University of Chicago Press, vol. 78(5), pages 1917-1942, September.
    14. Mohammed Abdellaoui & Olivier L’Haridon & Horst Zank, 2010. "Separating curvature and elevation: A parametric probability weighting function," Journal of Risk and Uncertainty, Springer, vol. 41(1), pages 39-65, August.
    15. Chen, Zhimin & Ibragimov, Rustam, 2019. "One country, two systems? The heavy-tailedness of Chinese A- and H- share markets," Emerging Markets Review, Elsevier, vol. 38(C), pages 115-141.
    16. Matoso, Rafael & Rezende, Marcelo, 2014. "Asymmetric information in oil and gas lease auctions with a national company," International Journal of Industrial Organization, Elsevier, vol. 33(C), pages 72-82.
    17. Vaida Kaduskeviciute & Sigitas Urbonavicius, 2019. "Webrooming: A Way of Dealing with Uncertainties in Purchasing," Tržište/Market, Faculty of Economics and Business, University of Zagreb, vol. 31(2), pages 139-152.
    18. Qin, Jie, 2015. "A model of regret, investor behavior, and market turbulence," Journal of Economic Theory, Elsevier, vol. 160(C), pages 150-174.
    19. Bali, Turan G. & Brown, Stephen J. & Caglayan, Mustafa O., 2019. "Upside potential of hedge funds as a predictor of future performance," Journal of Banking & Finance, Elsevier, vol. 98(C), pages 212-229.
    20. Thomas Buser, 2016. "The Impact of Losing in a Competition on the Willingness to Seek Further Challenges," Management Science, INFORMS, vol. 62(12), pages 3439-3449, December.
    21. Michael K. Berkowitz & Yehuda Kotowitz, 1997. "Management Compensation and the Performance of Mutual Funds," Working Papers berk-97-01, University of Toronto, Department of Economics.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:52:y:2006:i:3:p:311-322. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.