IDEAS home Printed from https://ideas.repec.org/p/bhd/dpaper/202501.html

Let U​​s Ask a Gen AI Chatbot: Machine Versus Bank Survey Results

Author

Listed:
  • Veronica B. Bayangos

    (Bangko Sentral ng Pilipinas)

  • Neil Fidelle G. Lomibao

    (Bangko Sentral ng Pilipinas)

  • Ralph Joshua P. Macarasig

    (Bangko Sentral ng Pilipinas)

Abstract

This study evaluates the performance of two generative pre-trained transformer (GPT) models, ChatGPT 4o and Gemini Pro 1.5, in responding to a banking sector outlook survey designed to support supervisory policy assessment. The study is structured around three main exercises. The first classifies questions related to the analysis of financial risks, challenges, and outlooks. The second employs prompt engineering to mitigate biases and address potential hallucinations in the models. The third constructs Cosine Similarity Indices (CSIs) to compare the responses of the GPT models with those of banks. The findings suggest that both models can serve as valuable tools for assessing a bank’s financial performance, risks, and macroeconomic outlook. These results underscore their potential to complement a supervisor’s analytical tasks. However, the observed differences in responses may point to the complex and potentially non-linear relationships that underlie bank performance, thereby reinforcing the continued importance of human judgment in bank supervision.

Suggested Citation

  • Veronica B. Bayangos & Neil Fidelle G. Lomibao & Ralph Joshua P. Macarasig, 2025. "Let U​​s Ask a Gen AI Chatbot: Machine Versus Bank Survey Results," BSP Discussion Paper Series 202501, Bangko Sentral ng Pilipinas.
    • Handle: RePEc:bhd:dpaper:202501
    as

    Download full text from publisher

    File URL: https://www.bsp.gov.ph/Sites/researchsite/Publications/BSP-Discussion-Papers/DP202501.pdf
    Download Restriction: no
    ---><---

    More about this item

    JEL classification:

    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • E42 - Macroeconomics and Monetary Economics - - Money and Interest Rates - - - Monetary Systems; Standards; Regimes; Government and the Monetary System
    • E5 - Macroeconomics and Monetary Economics - - Monetary Policy, Central Banking, and the Supply of Money and Credit
    • E61 - Macroeconomics and Monetary Economics - - Macroeconomic Policy, Macroeconomic Aspects of Public Finance, and General Outlook - - - Policy Objectives; Policy Designs and Consistency; Policy Coordination
    • G28 - Financial Economics - - Financial Institutions and Services - - - Government Policy and Regulation

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bhd:dpaper:202501. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MB Pagalunan (email available below). General contact details of provider: https://edirc.repec.org/data/bspgvph.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.