IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2205.04619.html
   My bibliography  Save this paper

Risk Preferences of Learning Algorithms

Author

Listed:
  • Andreas Haupt
  • Aroon Narayanan

Abstract

Agents' learning from feedback shapes economic outcomes, and many economic decision-makers today employ learning algorithms to make consequential choices. This note shows that a widely used learning algorithm, $\varepsilon$-Greedy, exhibits emergent risk aversion: it prefers actions with lower variance. When presented with actions of the same expectation, under a wide range of conditions, $\varepsilon$-Greedy chooses the lower-variance action with probability approaching one. This emergent preference can have wide-ranging consequences, ranging from concerns about fairness to homogenization, and holds transiently even when the riskier action has a strictly higher expected payoff. We discuss two methods to correct this bias. The first method requires the algorithm to reweight data as a function of how likely the actions were to be chosen. The second requires the algorithm to have optimistic estimates of actions for which it has not collected much data. We show that risk-neutrality is restored with these corrections.

Suggested Citation

  • Andreas Haupt & Aroon Narayanan, 2022. "Risk Preferences of Learning Algorithms," Papers 2205.04619, arXiv.org, revised Dec 2023.
    • Handle: RePEc:arx:papers:2205.04619
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2205.04619
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Vasserman, Shoshana & Watt, Mitchell, 2021. "Risk aversion and auction design: Theoretical and empirical evidence," International Journal of Industrial Organization, Elsevier, vol. 79(C).
    2. Emilio Calvano & Giacomo Calzolari & Vincenzo Denicolò & Sergio Pastorello, 2020. "Artificial Intelligence, Algorithmic Pricing, and Collusion," American Economic Review, American Economic Association, vol. 110(10), pages 3267-3297, October.
    3. Karsten T. Hansen & Kanishka Misra & Mallesh M. Pai, 2021. "Frontiers: Algorithmic Collusion: Supra-competitive Prices via," Marketing Science, INFORMS, vol. 40(1), pages 1-12, January.
    4. John Asker & Chaim Fershtman & Ariel Pakes, 2021. "Artificial Intelligence and Pricing: The Impact of Algorithm Design," NBER Working Papers 28535, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Martin, Simon & Rasch, Alexander, 2022. "Collusion by algorithm: The role of unobserved actions," DICE Discussion Papers 382, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    2. Aniko …ry & Ali Horta su & Kevin Williams, 2022. "Dynamic Price Competition: Theory and Evidence from Airline Markets," Cowles Foundation Discussion Papers 2341R1, Cowles Foundation for Research in Economics, Yale University, revised Apr 2023.
    3. Simon Martin & Alexander Rasch, 2022. "Collusion by Algorithm: The Role of Unobserved Actions," CESifo Working Paper Series 9629, CESifo.
    4. Andreas A. Haupt & Phillip J. K. Christoffersen & Mehul Damani & Dylan Hadfield-Menell, 2022. "Formal Contracts Mitigate Social Dilemmas in Multi-Agent RL," Papers 2208.10469, arXiv.org, revised Jan 2024.
    5. Wuming Fu & Qian Qi, 2023. "Artificial Intelligence and Dual Contract," Papers 2303.12350, arXiv.org.
    6. Erik Hermann, 2022. "Leveraging Artificial Intelligence in Marketing for Social Good—An Ethical Perspective," Journal of Business Ethics, Springer, vol. 179(1), pages 43-61, August.
    7. Bingyan Han, 2022. "Can maker-taker fees prevent algorithmic cooperation in market making?," Papers 2211.00496, arXiv.org.
    8. Ludovico Crippa & Yonatan Gur & Bar Light, 2022. "Equilibria in Repeated Games under No-Regret with Dynamic Benchmarks," Papers 2212.03152, arXiv.org, revised Jul 2023.
    9. Inkoo Cho & Noah Williams, 2024. "Collusive Outcomes Without Collusion," Papers 2403.07177, arXiv.org.
    10. Ding, Shasha & Sun, Hao & Sun, Panfei & Han, Weibin, 2022. "Dynamic outcome of coopetition duopoly with implicit collusion," Chaos, Solitons & Fractals, Elsevier, vol. 160(C).
    11. Daníelsson, Jón & Macrae, Robert & Uthemann, Andreas, 2022. "Artificial intelligence and systemic risk," Journal of Banking & Finance, Elsevier, vol. 140(C).
    12. Thomas Loots & Arnoud V. den Boer, 2023. "Data‐driven collusion and competition in a pricing duopoly with multinomial logit demand," Production and Operations Management, Production and Operations Management Society, vol. 32(4), pages 1169-1186, April.
    13. Soria, Jorge & Moya, Jorge & Mohazab, Amin, 2023. "Optimal mining in proof-of-work blockchain protocols," Finance Research Letters, Elsevier, vol. 53(C).
    14. Sinziana-Maria Rindasu & Ioan Dan Topor & Liliana Ionescu-Feleaga, 2023. "The Evolution of Management Accountants' Digital Skills in Industry 4.0: A Qualitative Approach," Oblik i finansi, Institute of Accounting and Finance, issue 1, pages 38-48, March.
    15. Zhijun Chen & pch346 & Chongwoo Choe & Jiajia Cong & Noriaki Matsushima, 2020. "Data-Driven Mergers and Personalization," Monash Economics Working Papers 16-20, Monash University, Department of Economics.
    16. Lucila Porto, 2022. "Q-Learning algorithms in a Hotelling model," Asociación Argentina de Economía Política: Working Papers 4587, Asociación Argentina de Economía Política.
    17. Böheim, René & Hackl, Franz & Hölzl-Leitner, Michael, 2021. "The impact of price adjustment costs on price dispersion in e-commerce," International Journal of Industrial Organization, Elsevier, vol. 77(C).
    18. Emilio Calvano & Giacomo Calzolari & Vincenzo Denicolò & Sergio Pastorello, 2019. "Algorithmic Pricing What Implications for Competition Policy?," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 55(1), pages 155-171, August.
    19. Bingyan Han, 2021. "Understanding algorithmic collusion with experience replay," Papers 2102.09139, arXiv.org, revised Mar 2021.
    20. Marcel Wieting & Geza Sapi, 2021. "Algorithms in the Marketplace: An Empirical Analysis of Automated Pricing in E-Commerce," Working Papers 21-06, NET Institute.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2205.04619. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.