IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2309.00649.html
   My bibliography  Save this paper

GPT has become financially literate: Insights from financial literacy tests of GPT and a preliminary test of how people use it as a source of advice

Author

Listed:
  • Pawe{l} Niszczota
  • Sami Abbas

Abstract

We assess the ability of GPT -- a large language model -- to serve as a financial robo-advisor for the masses, by using a financial literacy test. Davinci and ChatGPT based on GPT-3.5 score 66% and 65% on the financial literacy test, respectively, compared to a baseline of 33%. However, ChatGPT based on GPT-4 achieves a near-perfect 99% score, pointing to financial literacy becoming an emergent ability of state-of-the-art models. We use the Judge-Advisor System and a savings dilemma to illustrate how researchers might assess advice-utilization from large language models. We also present a number of directions for future research.

Suggested Citation

  • Pawe{l} Niszczota & Sami Abbas, 2023. "GPT has become financially literate: Insights from financial literacy tests of GPT and a preliminary test of how people use it as a source of advice," Papers 2309.00649, arXiv.org, revised Sep 2024.
    • Handle: RePEc:arx:papers:2309.00649
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2309.00649
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. van Rooij, Maarten & Lusardi, Annamaria & Alessie, Rob, 2011. "Financial literacy and stock market participation," Journal of Financial Economics, Elsevier, vol. 101(2), pages 449-472, August.
    2. Sniezek, Janet A. & Buckley, Timothy, 1995. "Cueing and Cognitive Conflict in Judge-Advisor Decision Making," Organizational Behavior and Human Decision Processes, Elsevier, vol. 62(2), pages 159-174, May.
    3. Dowling, Michael & Lucey, Brian, 2023. "ChatGPT for (Finance) research: The Bananarama Conjecture," Finance Research Letters, Elsevier, vol. 53(C).
    4. David R. Lewis, 2018. "The perils of overconfidence: Why many consumers fail to seek advice when they really should," Journal of Financial Services Marketing, Palgrave Macmillan, vol. 23(2), pages 104-111, June.
    5. Palan, Stefan & Schitter, Christian, 2018. "Prolific.ac—A subject pool for online experiments," Journal of Behavioral and Experimental Finance, Elsevier, vol. 17(C), pages 22-27.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pelster, Matthias & Val, Joel, 2024. "Can ChatGPT assist in picking stocks?," Finance Research Letters, Elsevier, vol. 59(C).
    2. Yuqi Nie & Yaxuan Kong & Xiaowen Dong & John M. Mulvey & H. Vincent Poor & Qingsong Wen & Stefan Zohren, 2024. "A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges," Papers 2406.11903, arXiv.org.
    3. Oehler, Andreas & Horn, Matthias, 2024. "Does ChatGPT provide better advice than robo-advisors?," Finance Research Letters, Elsevier, vol. 60(C).
    4. Jeongbin Kim & Matthew Kovach & Kyu-Min Lee & Euncheol Shin & Hector Tzavellas, 2024. "Learning to be Homo Economicus: Can an LLM Learn Preferences from Choice," Papers 2401.07345, arXiv.org.
    5. Ko, Hyungjin & Lee, Jaewook, 2024. "Can ChatGPT improve investment decisions? From a portfolio management perspective," Finance Research Letters, Elsevier, vol. 64(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Niszczota, Paweł & Abbas, Sami, 2023. "GPT has become financially literate: Insights from financial literacy tests of GPT and a preliminary test of how people use it as a source of advice," Finance Research Letters, Elsevier, vol. 58(PA).
    2. Thérèse Lind & Ali Ahmed & Kenny Skagerlund & Camilla Strömbäck & Daniel Västfjäll & Gustav Tinghög, 2020. "Competence, Confidence, and Gender: The Role of Objective and Subjective Financial Knowledge in Household Finance," Journal of Family and Economic Issues, Springer, vol. 41(4), pages 626-638, December.
    3. Niszczota, Paweł & Błaszczyński, Jakub, 2024. "Hard to digest investments: People oppose investment in both conventional and cultured meat producers," Ecological Economics, Elsevier, vol. 218(C).
    4. Stone, Gregory & Walton, Stephanie & Zhang, Yibo (James), 2023. "The impact of online tax community advice on individual taxpayer decision making," Advances in accounting, Elsevier, vol. 63(C).
    5. Christelis, Dimitris & Dobrescu, Loretti I. & Motta, Alberto, 2020. "Early life conditions and financial risk-taking in older age," The Journal of the Economics of Ageing, Elsevier, vol. 17(C).
    6. Tullio Jappelli, 2010. "Economic Literacy: An International Comparison," Economic Journal, Royal Economic Society, vol. 120(548), pages 429-451, November.
    7. Ni, Dan & Jiwen Song, Lynda & Zheng, Xiaoming & Zhu, Jinlong & Zhang, Mengyi & Xu, Lingxiao, 2022. "Extending a helping hand: How receiving gratitude makes a difference in employee performance during a crisis," Journal of Business Research, Elsevier, vol. 149(C), pages 967-982.
    8. Anna Ispierto Maté, Irma Martínez García, Gloria Ruiz Suárez., 2021. "Educación financiera y decisiones de ahorro e inversión: un análisis de la Encuesta de Competencias Financieras (ECF)," CNMV Documentos de Trabajo CNMV Documentos de Trabaj, CNMV- Comisión Nacional del Mercado de Valores - Departamento de Estudios y Estadísticas.
    9. Julie Bayle-Cordier & Loïc Berger & Rayan Elatmani & Massimo Tavoni, 2023. "Breath, Love, Walk? The Impact of Mindfulness Interventions on Climate Policy Support and Environmental Attitudes," Sustainability, MDPI, vol. 15(13), pages 1-29, July.
    10. Alisdair McKay, 2011. "Household Saving Behavior and Social Security Privatization," Boston University - Department of Economics - Working Papers Series WP2011-027, Boston University - Department of Economics.
    11. Ranganathan, Kavitha & Lejarraga, Tomás, 2021. "Elicitation of risk preferences through satisficing," Journal of Behavioral and Experimental Finance, Elsevier, vol. 32(C).
    12. Dr Samamba, Lennox Trivedi, PhD—Law, 2023. "Legal Aspects of Promoting the Participation of Collective Investment Schemes in Eastern and Southern African Frontier Securities Markets," International Journal of Research and Innovation in Social Science, International Journal of Research and Innovation in Social Science (IJRISS), vol. 7(4), pages 121-141, April.
    13. Bannier, Christina E. & Neubert, Milena, 2016. "Gender differences in financial risk taking: The role of financial literacy and risk tolerance," Economics Letters, Elsevier, vol. 145(C), pages 130-135.
    14. Dräger, Lena & Lamla, Michael J. & Pfajfar, Damjan, 2020. "The Hidden Heterogeneity of Inflation and Interest Rate Expectations: The Role of Preferences," Hannover Economic Papers (HEP) dp-666, Leibniz Universität Hannover, Wirtschaftswissenschaftliche Fakultät, revised Feb 2023.
    15. Kessel, Dany & Mollerstrom, Johanna & van Veldhuizen, Roel, 2021. "Can simple advice eliminate the gender gap in willingness to compete?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 138, pages 1-1.
    16. Sheridan Titman & Chishen Wei. Wei & Bin Zhao, 2021. "Corporate Actions and the Manipulation of Retail Investors in China: An Analysis of Stock Splits," NBER Working Papers 29212, National Bureau of Economic Research, Inc.
    17. Morten Balling, 2011. "Asymmetries in Financial Information, Risk and Know-how: The Roles of Disclosure Rules, Financial Safety Nets and Market Discipline," Chapters, in: Christopher J. Green & Eric J. Pentecost & Tom Weyman-Jones (ed.), The Financial Crisis and the Regulation of Finance, chapter 13, Edward Elgar Publishing.
    18. Michael Ziegelmeyer & Julius Nick, 2013. "Backing out of private pension provision: lessons from Germany," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 40(3), pages 505-539, August.
    19. Cupák, Andrej & Fessler, Pirmin & Hsu, Joanne W. & Paradowski, Piotr R., 2022. "Investor confidence and high financial literacy jointly shape investments in risky assets," Economic Modelling, Elsevier, vol. 116(C).
    20. Yusuf, Fatima & Yousaf, Amna & Saeed, Abubakr, 2018. "Rethinking agency theory in developing countries: A case study of Pakistan," Accounting forum, Elsevier, vol. 42(4), pages 281-292.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2309.00649. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.