IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v64y2018i3p1155-1170.html
   My bibliography  Save this article

Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them

Author

Listed:
  • Berkeley J. Dietvorst

    (Booth School of Business, University of Chicago, Chicago, Illinois 60637)

  • Joseph P. Simmons

    (The Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania 19104)

  • Cade Massey

    (The Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania 19104)

Abstract

Although evidence-based algorithms consistently outperform human forecasters, people often fail to use them after learning that they are imperfect, a phenomenon known as algorithm aversion . In this paper, we present three studies investigating how to reduce algorithm aversion. In incentivized forecasting tasks, participants chose between using their own forecasts or those of an algorithm that was built by experts. Participants were considerably more likely to choose to use an imperfect algorithm when they could modify its forecasts, and they performed better as a result. Notably, the preference for modifiable algorithms held even when participants were severely restricted in the modifications they could make (Studies 1–3). In fact, our results suggest that participants’ preference for modifiable algorithms was indicative of a desire for some control over the forecasting outcome, and not for a desire for greater control over the forecasting outcome, as participants’ preference for modifiable algorithms was relatively insensitive to the magnitude of the modifications they were able to make (Study 2). Additionally, we found that giving participants the freedom to modify an imperfect algorithm made them feel more satisfied with the forecasting process, more likely to believe that the algorithm was superior, and more likely to choose to use an algorithm to make subsequent forecasts (Study 3). This research suggests that one can reduce algorithm aversion by giving people some control—even a slight amount—over an imperfect algorithm’s forecast.

Suggested Citation

  • Berkeley J. Dietvorst & Joseph P. Simmons & Cade Massey, 2018. "Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them," Management Science, INFORMS, vol. 64(3), pages 1155-1170, March.
  • Handle: RePEc:inm:ormnsc:v:64:y:2018:i:3:p:1155-1170
    DOI: 10.1287/mnsc.2016.2643
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/mnsc.2016.2643
    Download Restriction: no

    File URL: https://libkey.io/10.1287/mnsc.2016.2643?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Highhouse, Scott, 2008. "Stubborn Reliance on Intuition and Subjectivity in Employee Selection," Industrial and Organizational Psychology, Cambridge University Press, vol. 1(3), pages 333-342, September.
    2. Sanders, Nada R. & Manrodt, Karl B., 2003. "The efficacy of using judgmental versus quantitative forecasting methods in practice," Omega, Elsevier, vol. 31(6), pages 511-522, December.
    3. Lawrence, Michael & Goodwin, Paul & O'Connor, Marcus & Onkal, Dilek, 2006. "Judgmental forecasting: A review of progress over the last 25 years," International Journal of Forecasting, Elsevier, vol. 22(3), pages 493-518.
    4. Robin M. Hogarth & Spyros Makridakis, 1981. "Forecasting and Planning: An Evaluation," Management Science, INFORMS, vol. 27(2), pages 115-138, February.
    5. Fildes, Robert & Goodwin, Paul & Lawrence, Michael & Nikolopoulos, Konstantinos, 2009. "Effective forecasting and judgmental adjustments: an empirical evaluation and strategies for improvement in supply-chain planning," International Journal of Forecasting, Elsevier, vol. 25(1), pages 3-23.
    6. Robert Fildes & Paul Goodwin, 2007. "Against Your Better Judgment? How Organizations Can Improve Their Use of Management Judgment in Forecasting," Interfaces, INFORMS, vol. 37(6), pages 570-576, December.
    7. Willemain, Thomas R., 1991. "The effect of graphical adjustment on forecast accuracy," International Journal of Forecasting, Elsevier, vol. 7(2), pages 151-154, August.
    8. Maurice E. Schweitzer & Gérard P. Cachon, 2000. "Decision Bias in the Newsvendor Problem with a Known Demand Distribution: Experimental Evidence," Management Science, INFORMS, vol. 46(3), pages 404-420, March.
    9. Robert Carbone & Allan Andersen & Yvan Corriveau & Paul Piat Corson, 1983. "Comparing for Different Time Series Methods the Value of Technical Expertise Individualized Analysis, and Judgmental Adjustment," Management Science, INFORMS, vol. 29(5), pages 559-566, May.
    10. Bonaccio, Silvia & Dalal, Reeshad S., 2006. "Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences," Organizational Behavior and Human Decision Processes, Elsevier, vol. 101(2), pages 127-151, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Leitner, Johannes & Leopold-Wildburger, Ulrike, 2011. "Experiments on forecasting behavior with several sources of information - A review of the literature," European Journal of Operational Research, Elsevier, vol. 213(3), pages 459-469, September.
    2. Arvan, Meysam & Fahimnia, Behnam & Reisi, Mohsen & Siemsen, Enno, 2019. "Integrating human judgement into quantitative forecasting methods: A review," Omega, Elsevier, vol. 86(C), pages 237-252.
    3. Perera, H. Niles & Hurley, Jason & Fahimnia, Behnam & Reisi, Mohsen, 2019. "The human factor in supply chain forecasting: A systematic review," European Journal of Operational Research, Elsevier, vol. 274(2), pages 574-600.
    4. Sroginis, Anna & Fildes, Robert & Kourentzes, Nikolaos, 2023. "Use of contextual and model-based information in adjusting promotional forecasts," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1177-1191.
    5. Christiane B. Haubitz & Cedric A. Lehmann & Andreas Fügener & Ulrich W. Thonemann, 2021. "The Risk of Algorithm Transparency: How Algorithm Complexity Drives the Effects on Use of Advice," ECONtribute Discussion Papers Series 078, University of Bonn and University of Cologne, Germany.
    6. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    7. Petropoulos, Fotios & Fildes, Robert & Goodwin, Paul, 2016. "Do ‘big losses’ in judgmental adjustments to statistical forecasts affect experts’ behaviour?," European Journal of Operational Research, Elsevier, vol. 249(3), pages 842-852.
    8. Abolghasemi, Mahdi & Hurley, Jason & Eshragh, Ali & Fahimnia, Behnam, 2020. "Demand forecasting in the presence of systematic events: Cases in capturing sales promotions," International Journal of Production Economics, Elsevier, vol. 230(C).
    9. Lin, Vera Shanshan & Goodwin, Paul & Song, Haiyan, 2014. "Accuracy and bias of experts’ adjusted forecasts," Annals of Tourism Research, Elsevier, vol. 48(C), pages 156-174.
    10. Surti, Chirag & Celani, Anthony & Gajpal, Yuvraj, 2020. "The newsvendor problem: The role of prospect theory and feedback," European Journal of Operational Research, Elsevier, vol. 287(1), pages 251-261.
    11. Baecke, Philippe & De Baets, Shari & Vanderheyden, Karlien, 2017. "Investigating the added value of integrating human judgement into statistical demand forecasting systems," International Journal of Production Economics, Elsevier, vol. 191(C), pages 85-96.
    12. Fildes, Robert & Goodwin, Paul & Lawrence, Michael & Nikolopoulos, Konstantinos, 2009. "Effective forecasting and judgmental adjustments: an empirical evaluation and strategies for improvement in supply-chain planning," International Journal of Forecasting, Elsevier, vol. 25(1), pages 3-23.
    13. Cedric A. Lehmann & Christiane B. Haubitz & Andreas Fügener & Ulrich W. Thonemann, 2022. "The risk of algorithm transparency: How algorithm complexity drives the effects on the use of advice," Production and Operations Management, Production and Operations Management Society, vol. 31(9), pages 3419-3434, September.
    14. De Baets, Shari & Harvey, Nigel, 2018. "Forecasting from time series subject to sporadic perturbations: Effectiveness of different types of forecasting support," International Journal of Forecasting, Elsevier, vol. 34(2), pages 163-180.
    15. Pennings, Clint L.P. & van Dalen, Jan & Rook, Laurens, 2019. "Coordinating judgmental forecasting: Coping with intentional biases," Omega, Elsevier, vol. 87(C), pages 46-56.
    16. De Baets, Shari & Harvey, Nigel, 2020. "Using judgment to select and adjust forecasts from statistical models," European Journal of Operational Research, Elsevier, vol. 284(3), pages 882-895.
    17. Alvarado-Valencia, Jorge & Barrero, Lope H. & Önkal, Dilek & Dennerlein, Jack T., 2017. "Expertise, credibility of system forecasts and integration methods in judgmental demand forecasting," International Journal of Forecasting, Elsevier, vol. 33(1), pages 298-313.
    18. Eksoz, Can & Mansouri, S. Afshin & Bourlakis, Michael, 2014. "Collaborative forecasting in the food supply chain: A conceptual framework," International Journal of Production Economics, Elsevier, vol. 158(C), pages 120-135.
    19. Van den Broeke, Maud & De Baets, Shari & Vereecke, Ann & Baecke, Philippe & Vanderheyden, Karlien, 2019. "Judgmental forecast adjustments over different time horizons," Omega, Elsevier, vol. 87(C), pages 34-45.
    20. Khosrowabadi, Naghmeh & Hoberg, Kai & Imdahl, Christina, 2022. "Evaluating human behaviour in response to AI recommendations for judgemental forecasting," European Journal of Operational Research, Elsevier, vol. 303(3), pages 1151-1167.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:64:y:2018:i:3:p:1155-1170. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.