IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_11862.html
   My bibliography  Save this paper

Making GenAI Smarter: Evidence from a Portfolio Allocation Experiment

Author

Listed:
  • Lars Hornuf
  • David J. Streich
  • Niklas Töllich

Abstract

Retrieval-augmented generation (RAG) has emerged as a promising way to improve task-specific performance in generative artificial intelligence (GenAI) applications such as large language models (LLMs). In this study, we evaluate the performance implications of providing various types of domain-specific information to LLMs in a simple portfolio allocation task. We compare the recommendations of seven state-of-the-art LLMs in various experimental conditions against a benchmark of professional financial advisors. Our main result is that the provision of domain-specific information does not unambiguously improve the quality of recommendations. In particular, we find that LLM recommendations underperform recommendations by human financial advisors in the baseline condition. However, providing firm-specific information improves historical performance in LLM portfolios and closes the gap with human advisors. Performance improvements are achieved through higher exposure to market risk and not through an increase in mean-variance efficiency within the risky portfolio share. Notably, portfolio risk increases primarily for risk-averse investors. We also document that quantitative firm-specific information affects recommendations more than qualitative firm-specific information, and that equipping models with generic finance theory does not affect recommendations.

Suggested Citation

  • Lars Hornuf & David J. Streich & Niklas Töllich, 2025. "Making GenAI Smarter: Evidence from a Portfolio Allocation Experiment," CESifo Working Paper Series 11862, CESifo.
  • Handle: RePEc:ces:ceswps:_11862
    as

    Download full text from publisher

    File URL: https://www.ifo.de/DocDL/cesifo1_wp11862.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Sean Cao Robert H. Smith & Wei Jiang & Baozhong Yang J. Mack Robinson & Alan L Zhang & Tarun Ramadorai, 2023. "How to Talk When a Machine Is Listening: Corporate Disclosure in the Age of AI," The Review of Financial Studies, Society for Financial Studies, vol. 36(9), pages 3603-3642.
    2. Fama, Eugene F & French, Kenneth R, 1992. "The Cross-Section of Expected Stock Returns," Journal of Finance, American Finance Association, vol. 47(2), pages 427-465, June.
    3. Koedijk, Kees & ter Horst, Jenke & Borgers, Arian & Derwall, Jeroen, 2015. "Do Social Factors Influence Investment Behaviour and Performance? Evidence from Mutual Fund Holdings," CEPR Discussion Papers 10740, C.E.P.R. Discussion Papers.
    4. Thomas Dohmen & Armin Falk & David Huffman & Uwe Sunde & Jürgen Schupp & Gert G. Wagner, 2011. "Individual Risk Attitudes: Measurement, Determinants, And Behavioral Consequences," Journal of the European Economic Association, European Economic Association, vol. 9(3), pages 522-550, June.
    5. Cardak, Buly A. & Wilkins, Roger, 2009. "The determinants of household risky asset holdings: Australian evidence on background risk and other factors," Journal of Banking & Finance, Elsevier, vol. 33(5), pages 850-860, May.
    6. Benedikt Berger & Martin Adam & Alexander Rühr & Alexander Benlian, 2021. "Watch Me Improve—Algorithm Aversion and Demonstrating the Ability to Learn," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 63(1), pages 55-68, February.
    7. Oehler, Andreas & Horn, Matthias, 2024. "Does ChatGPT provide better advice than robo-advisors?," Finance Research Letters, Elsevier, vol. 60(C).
    8. Huber, Christoph & Huber, Jürgen, 2020. "Bad bankers no more? Truth-telling and (dis)honesty in the finance industry," Journal of Economic Behavior & Organization, Elsevier, vol. 180(C), pages 472-493.
    9. Annamaria Lusardi & Olivia S. Mitchell, 2023. "The Importance of Financial Literacy: Opening a New Field," Journal of Economic Perspectives, American Economic Association, vol. 37(4), pages 137-154, Fall.
    10. Andrei, Daniel & Friedman, Henry & Ozel, N. Bugra, 2023. "Economic uncertainty and investor attention," Journal of Financial Economics, Elsevier, vol. 149(2), pages 179-217.
    11. David J. Streich, 2023. "Risk Preference Elicitation and Financial Advice Taking," Journal of Behavioral Finance, Taylor & Francis Journals, vol. 24(3), pages 259-275, July.
    12. Niszczota, Paweł & Abbas, Sami, 2023. "GPT has become financially literate: Insights from financial literacy tests of GPT and a preliminary test of how people use it as a source of advice," Finance Research Letters, Elsevier, vol. 58(PA).
    13. Siemroth, Christoph & Hornuf, Lars, 2023. "Why Do Retail Investors Pick Green Investments? A Lab-in-the-Field Experiment with Crowdfunders," Journal of Economic Behavior & Organization, Elsevier, vol. 209(C), pages 74-90.
    14. repec:osf:osfxxx:b5682_v1 is not listed on IDEAS
    15. Döttling, Robin & Kim, Sehoon, 2024. "Sustainability Preferences Under Stress: Evidence from COVID-19," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 59(2), pages 435-473, March.
    16. Joao F. Cocco, 2005. "Consumption and Portfolio Choice over the Life Cycle," The Review of Financial Studies, Society for Financial Studies, vol. 18(2), pages 491-533.
    17. Ko, Hyungjin & Lee, Jaewook, 2024. "Can ChatGPT improve investment decisions? From a portfolio management perspective," Finance Research Letters, Elsevier, vol. 64(C).
    18. J. Tobin, 1958. "Liquidity Preference as Behavior Towards Risk," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 25(2), pages 65-86.
    19. Shijie Wu & Ozan Irsoy & Steven Lu & Vadim Dabravolski & Mark Dredze & Sebastian Gehrmann & Prabhanjan Kambadur & David Rosenberg & Gideon Mann, 2023. "BloombergGPT: A Large Language Model for Finance," Papers 2303.17564, arXiv.org, revised Dec 2023.
    20. Borgers, Arian & Derwall, Jeroen & Koedijk, Kees & ter Horst, Jenke, 2015. "Do social factors influence investment behavior and performance? Evidence from mutual fund holdings," Journal of Banking & Finance, Elsevier, vol. 60(C), pages 112-126.
    21. Lusardi, Annamaria & Mitchell, Olivia S., 2011. "Financial literacy around the world: an overview," Journal of Pension Economics and Finance, Cambridge University Press, vol. 10(4), pages 497-508, October.
    22. Zhi Da & Joseph Engelberg & Pengjie Gao, 2011. "In Search of Attention," Journal of Finance, American Finance Association, vol. 66(5), pages 1461-1499, October.
    23. Pedersen, Lasse Heje & Fitzgibbons, Shaun & Pomorski, Lukasz, 2021. "Responsible investing: The ESG-efficient frontier," Journal of Financial Economics, Elsevier, vol. 142(2), pages 572-597.
    24. Carhart, Mark M, 1997. "On Persistence in Mutual Fund Performance," Journal of Finance, American Finance Association, vol. 52(1), pages 57-82, March.
    25. de Haan, Ed & Lawrence, Alastair & Litjens, Robin, 2024. "Measuring investor attention using Google Search," Other publications TiSEM a186d16d-dce3-4d48-b0fc-1, Tilburg University, School of Economics and Management.
    26. Chen, Huaizhi & Cohen, Lauren & Gurun, Umit & Lou, Dong & Malloy, Christopher, 2020. "IQ from IP: simplifying search in portfolio choice," LSE Research Online Documents on Economics 101133, London School of Economics and Political Science, LSE Library.
    27. Chen, Huaizhi & Cohen, Lauren & Gurun, Umit & Lou, Dong & Malloy, Christopher, 2020. "IQ from IP: Simplifying search in portfolio choice," Journal of Financial Economics, Elsevier, vol. 138(1), pages 118-137.
    28. Hong, Harrison & Kacperczyk, Marcin, 2009. "The price of sin: The effects of social norms on markets," Journal of Financial Economics, Elsevier, vol. 93(1), pages 15-36, July.
    29. Berger, Benedikt & Adam, Martin & Rühr, Alexander & Benlian, Alexander, 2021. "Watch Me Improve—Algorithm Aversion and Demonstrating the Ability to Learn," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 124219, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    30. Yi Yang & Yixuan Tang & Kar Yan Tam, 2023. "InvestLM: A Large Language Model for Investment using Financial Domain Instruction Tuning," Papers 2309.13064, arXiv.org.
    31. Tim Loughran & Bill Mcdonald, 2011. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks," Journal of Finance, American Finance Association, vol. 66(1), pages 35-65, February.
    32. Brad M. Barber & Terrance Odean, 2008. "All That Glitters: The Effect of Attention and News on the Buying Behavior of Individual and Institutional Investors," The Review of Financial Studies, Society for Financial Studies, vol. 21(2), pages 785-818, April.
    33. Björn Hansson & Mattias Persson, 2000. "Time Diversification and Estimation Risk," Financial Analysts Journal, Taylor & Francis Journals, vol. 56(5), pages 55-62, September.
    34. Cordes, Henning & Nolte, Sven & Schneider, Judith C., 2023. "Dynamics of stock market developments, financial behavior, and emotions," Journal of Banking & Finance, Elsevier, vol. 154(C).
    35. Pelster, Matthias & Val, Joel, 2024. "Can ChatGPT assist in picking stocks?," Finance Research Letters, Elsevier, vol. 59(C).
    36. Linh Nguyen & Gerry Gallery & Cameron Newton, 2019. "The joint influence of financial risk perception and risk tolerance on individual investment decision‐making," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 59(S1), pages 747-771, April.
    37. Jegadeesh, Narasimhan & Titman, Sheridan, 1993. "Returns to Buying Winners and Selling Losers: Implications for Stock Market Efficiency," Journal of Finance, American Finance Association, vol. 48(1), pages 65-91, March.
    38. Jiafu An & Difang Huang & Chen Lin & Mingzhu Tai, 2024. "Measuring Gender and Racial Biases in Large Language Models," Papers 2403.15281, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christian Fieberg & Lars Hornuf & Maximilian Meiler & David J. Streich, 2025. "Using Large Language Models for Financial Advice," CESifo Working Paper Series 11666, CESifo.
    2. Ahmad, Fawad & Oriani, Raffaele, 2022. "Investor attention, information acquisition, and value premium: A mispricing perspective," International Review of Financial Analysis, Elsevier, vol. 79(C).
    3. Dong, Mengming Michael & Stratopoulos, Theophanis C. & Wang, Victor Xiaoqi, 2024. "A scoping review of ChatGPT research in accounting and finance," International Journal of Accounting Information Systems, Elsevier, vol. 55(C).
    4. Kleffel, Philipp & Muck, Matthias, 2024. "The confusion of taste and consumption: Evidence from a stated-choice experiment," Journal of Behavioral and Experimental Finance, Elsevier, vol. 43(C).
    5. Ciciretti, Rocco & Dalò, Ambrogio & Dam, Lammertjan, 2023. "The contributions of betas versus characteristics to the ESG premium," Journal of Empirical Finance, Elsevier, vol. 71(C), pages 104-124.
    6. Li, Frank Weikai & Sun, Chengzhu, 2022. "Information acquisition and expected returns: Evidence from EDGAR search traffic," Journal of Economic Dynamics and Control, Elsevier, vol. 141(C).
    7. David Hirshleifer & Po-Hsuan Hsu & Dongmei Li, 2018. "Innovative Originality, Profitability, and Stock Returns," The Review of Financial Studies, Society for Financial Studies, vol. 31(7), pages 2553-2605.
    8. Ma, Yao & Yang, Baochen & Su, Yunpeng, 2021. "Stock return predictability: Evidence from moving averages of trading volume," Pacific-Basin Finance Journal, Elsevier, vol. 65(C).
    9. Ryan Flugum & Choonsik Lee & Matthew E. Souther, 2023. "What happens in Vegas stays in Vegas? Firsthand experience and EDGAR search activity in Las Vegas casino hotels," Financial Management, Financial Management Association International, vol. 52(3), pages 409-432, September.
    10. Pungaliya, Raunaq S. & Wang, Yanbo, 2023. "Machine invasion: Automation in information acquisition and the cross-section of stock returns," Journal of Financial Markets, Elsevier, vol. 64(C).
    11. Stotz, Olaf & Georgi, Dominik, 2012. "A logit model of retail investors' individual trading decisions and their relations to insider trades," Review of Financial Economics, Elsevier, vol. 21(4), pages 159-167.
    12. Ahmad, Khurshid & Han, JingGuang & Hutson, Elaine & Kearney, Colm & Liu, Sha, 2016. "Media-expressed negative tone and firm-level stock returns," Journal of Corporate Finance, Elsevier, vol. 37(C), pages 152-172.
    13. Fan, Kwok Yuen & Shen, Jianfu & Hui, Eddie C.M. & Cheng, Louis T.W., 2024. "ESG components and equity returns: Evidence from real estate investment trusts," International Review of Financial Analysis, Elsevier, vol. 96(PB).
    14. Heusel, Nicola & Mager, Ferdinand, 2023. "Pension funding and the cross section of stock returns - The case of Germany," Journal of Banking & Finance, Elsevier, vol. 150(C).
    15. Rudkin, Wanling & Cai, Charlie X. & Zhou, You, 2025. "Can we enhance investment with ESG?," International Review of Financial Analysis, Elsevier, vol. 97(C).
    16. Rui Fan & Alex Nikolsko-Rzhevskyy & Oleksandr Talavera, 2025. "Foreign Eyes on Wall Street: Investor Attention and U.S. Stock Reactions," Discussion Papers 25-02, Department of Economics, University of Birmingham.
    17. Atilgan, Yigit & Bali, Turan G. & Demirtas, K. Ozgur & Gunaydin, A. Doruk, 2020. "Left-tail momentum: Underreaction to bad news, costly arbitrage and equity returns," Journal of Financial Economics, Elsevier, vol. 135(3), pages 725-753.
    18. Teng, Chia-Chen & Yang, J. Jimmy, 2021. "Media exposure on corporate social irresponsibility and firm performance," Pacific-Basin Finance Journal, Elsevier, vol. 68(C).
    19. Liu, Sha & Han, Jingguang, 2020. "Media tone and expected stock returns," International Review of Financial Analysis, Elsevier, vol. 70(C).
    20. Ho, Thang, 2022. "Climate change news sensitivity and mutual fund performance," International Review of Financial Analysis, Elsevier, vol. 83(C).

    More about this item

    Keywords

    generative artificial intelligence; large language models; domain-specific information; retrieval-augmented generation; portfolio management; portfolio allocation.;
    All these keywords.

    JEL classification:

    • G00 - Financial Economics - - General - - - General
    • G11 - Financial Economics - - General Financial Markets - - - Portfolio Choice; Investment Decisions
    • G40 - Financial Economics - - Behavioral Finance - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_11862. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Klaus Wohlrabe (email available below). General contact details of provider: https://edirc.repec.org/data/cesifde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.