IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v52y2006i3p311-322.html
   My bibliography  Save this article

The Optimizer's Curse: Skepticism and Postdecision Surprise in Decision Analysis

Author

Listed:
  • James E. Smith

    (Fuqua School of Business, Duke University, Durham, North Carolina 27708)

  • Robert L. Winkler

    (Fuqua School of Business, Duke University, Durham, North Carolina 27708)

Abstract

Decision analysis produces measures of value such as expected net present values or expected utilities and ranks alternatives by these value estimates. Other optimization-based processes operate in a similar manner. With uncertainty and limited resources, an analysis is never perfect, so these value estimates are subject to error. We show that if we take these value estimates at face value and select accordingly, we should expect the value of the chosen alternative to be less than its estimate, even if the value estimates are unbiased. Thus, when comparing actual outcomes to value estimates, we should expect to be disappointed on average, not because of any inherent bias in the estimates themselves, but because of the optimization-based selection process. We call this phenomenon the optimizer's curse and argue that it is not well understood or appreciated in the decision analysis and management science communities. This curse may be a factor in creating skepticism in decision makers who review the results of an analysis. In this paper, we study the optimizer's curse and show that the resulting expected disappointment may be substantial. We then propose the use of Bayesian methods to adjust value estimates. These Bayesian methods can be viewed as disciplined skepticism and provide a method for avoiding this postdecision disappointment.

Suggested Citation

  • James E. Smith & Robert L. Winkler, 2006. "The Optimizer's Curse: Skepticism and Postdecision Surprise in Decision Analysis," Management Science, INFORMS, vol. 52(3), pages 311-322, March.
    • Handle: RePEc:inm:ormnsc:v:52:y:2006:i:3:p:311-322
    DOI: 10.1287/mnsc.1050.0451
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mnsc.1050.0451
    Download Restriction: no
    File URL: https://libkey.io/10.1287/mnsc.1050.0451?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Steven N. Tani, 1978. "A Perspective on Modeling in Decision Analysis," Management Science, INFORMS, vol. 24(14), pages 1500-1506, October.
    2. Porter, Robert H, 1995. "The Role of Information in U.S. Offshore Oil and Gas Lease Auctions," Econometrica, Econometric Society, vol. 63(1), pages 1-27, January.
    3. Brown, Stephen J, et al, 1992. "Survivorship Bias in Performance Studies," The Review of Financial Studies, Society for Financial Studies, vol. 5(4), pages 553-580.
    4. David E. Bell, 1985. "Disappointment in Decision Making Under Uncertainty," Operations Research, INFORMS, vol. 33(1), pages 1-27, February.
    5. Robert L. Winkler & Daniel G. Brooks, 1980. "Competitive Bidding with Dependent Value Estimates," Operations Research, INFORMS, vol. 28(3-part-i), pages 603-613, June.
    6. Peter A. Morris, 1974. "Decision Analysis Expert Use," Management Science, INFORMS, vol. 20(9), pages 1233-1241, May.
    7. Robert T. Clemen & Robert C. Kwit, 2001. "The Value of Decision Analysis at Eastman Kodak Company, 1990-1999," Interfaces, INFORMS, vol. 31(5), pages 74-92, October.
    8. Brown, Keith C, 1974. "A Note on the Apparent Bias of Net Revenue Estimates for Capital Investment Projects," Journal of Finance, American Finance Association, vol. 29(4), pages 1215-1216, September.
    9. Robert C. Nickerson & Dean W. Boyd, 1980. "The Use and Value of Models in Decision Analysis," Operations Research, INFORMS, vol. 28(1), pages 139-155, February.
    10. Dennis V. Lindley, 1986. "The Reconciliation of Decision Analyses," Operations Research, INFORMS, vol. 34(2), pages 289-295, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vilkkumaa, Eeva & Liesiö, Juuso & Salo, Ahti, 2014. "Optimal strategies for selecting project portfolios using uncertain value estimates," European Journal of Operational Research, Elsevier, vol. 233(3), pages 772-783.
    2. Chateauneuf, Alain & Eichberger, Jurgen & Grant, Simon, 2007. "Choice under uncertainty with the best and worst in mind: Neo-additive capacities," Journal of Economic Theory, Elsevier, vol. 137(1), pages 538-567, November.
    3. van Dijk, Wilco W. & van der Pligt, Joop, 1997. "The Impact of Probability and Magnitude of Outcome on Disappointment and Elation," Organizational Behavior and Human Decision Processes, Elsevier, vol. 69(3), pages 277-284, March.
    4. Ian Tonks, 2005. "Performance Persistence of Pension-Fund Managers," The Journal of Business, University of Chicago Press, vol. 78(5), pages 1917-1942, September.
    5. Mohammed Abdellaoui & Olivier L’Haridon & Horst Zank, 2010. "Separating curvature and elevation: A parametric probability weighting function," Journal of Risk and Uncertainty, Springer, vol. 41(1), pages 39-65, August.
    6. Chen, Zhimin & Ibragimov, Rustam, 2019. "One country, two systems? The heavy-tailedness of Chinese A- and H- share markets," Emerging Markets Review, Elsevier, vol. 38(C), pages 115-141.
    7. Vaida Kaduskeviciute & Sigitas Urbonavicius, 2019. "Webrooming: A Way of Dealing with Uncertainties in Purchasing," Tržište/Market, Faculty of Economics and Business, University of Zagreb, vol. 31(2), pages 139-152.
    8. Bali, Turan G. & Brown, Stephen J. & Caglayan, Mustafa O., 2019. "Upside potential of hedge funds as a predictor of future performance," Journal of Banking & Finance, Elsevier, vol. 98(C), pages 212-229.
    9. Thomas Buser, 2016. "The Impact of Losing in a Competition on the Willingness to Seek Further Challenges," Management Science, INFORMS, vol. 62(12), pages 3439-3449, December.
    10. Ana C. Díaz†Mendoza & Germán López†Espinosa & Miguel A. Martínez, 2014. "The Efficiency of Performance†Based Fee Funds," European Financial Management, European Financial Management Association, vol. 20(4), pages 825-855, September.
    11. Hall, Stephen G. & Mitchell, James, 2007. "Combining density forecasts," International Journal of Forecasting, Elsevier, vol. 23(1), pages 1-13.
    12. Enrico G. De Giorgi & Thierry Post, 2011. "Loss Aversion with a State-Dependent Reference Point," Management Science, INFORMS, vol. 57(6), pages 1094-1110, June.
    13. Gao, Dong Li & Xie, Wei & Ming Lee, Eric Wai, 2022. "Individual-level exit choice behaviour under uncertain risk," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 604(C).
    14. Du, Ding & Huang, Zhaodan & Blanchfield, Peter J., 2009. "Do fixed income mutual fund managers have managerial skills?," The Quarterly Review of Economics and Finance, Elsevier, vol. 49(2), pages 378-397, May.
    15. David A. Asch & James P. Patton & John C. Hershey, 1991. "Prognostic Information versus Accuracy," Medical Decision Making, , vol. 11(1), pages 45-47, February.
    16. Shen, Yinhai & Zhang, Qing & Zhang, Zhichao & Ma, Xinyu, 2022. "Omnichannel retailing return operations with consumer disappointment aversion," Operations Research Perspectives, Elsevier, vol. 9(C).
    17. Sebastian Lobe & Christian Walkshäusl, 2016. "Vice versus virtue investing around the world," Review of Managerial Science, Springer, vol. 10(2), pages 303-344, March.
    18. Vassilios Babalos & Michael Doumpos & Nikolaos Philippas & Constantin Zopounidis, 2015. "Towards a Holistic Approach for Mutual Fund Performance Appraisal," Computational Economics, Springer;Society for Computational Economics, vol. 46(1), pages 35-53, June.
    19. Ali Mosleh & George Apostolakis, 1986. "The Assessment of Probability Distributions from Expert Opinions with an Application to Seismic Fragility Curves," Risk Analysis, John Wiley & Sons, vol. 6(4), pages 447-461, December.
    20. Thibault Gajdos & Jean-Christophe Vergnaud, 2013. "Decisions with conflicting and imprecise information," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 41(2), pages 427-452, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:52:y:2006:i:3:p:311-322. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.