IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2502.01992.html

FinRLlama: A Solution to LLM-Engineered Signals Challenge at FinRL Contest 2024

Author

Listed:
  • Arnav Grover

Abstract

In response to Task II of the FinRL Challenge at ACM ICAIF 2024, this study proposes a novel prompt framework for fine-tuning large language models (LLM) with Reinforcement Learning from Market Feedback (RLMF). Our framework incorporates market-specific features and short-term price dynamics to generate more precise trading signals. Traditional LLMs, while competent in sentiment analysis, lack contextual alignment for financial market applications. To bridge this gap, we fine-tune the LLaMA-3.2-3B-Instruct model using a custom RLMF prompt design that integrates historical market data and reward-based feedback. Our evaluation shows that this RLMF-tuned framework outperforms baseline methods in signal consistency and achieving tighter trading outcomes; awarded as winner of Task II. You can find the code for this project on GitHub.

Suggested Citation

  • Arnav Grover, 2025. "FinRLlama: A Solution to LLM-Engineered Signals Challenge at FinRL Contest 2024," Papers 2502.01992, arXiv.org.
    • Handle: RePEc:arx:papers:2502.01992
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2502.01992
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Yuqi Nie & Yaxuan Kong & Xiaowen Dong & John M. Mulvey & H. Vincent Poor & Qingsong Wen & Stefan Zohren, 2024. "A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges," Papers 2406.11903, arXiv.org.
    2. Tim Loughran & Bill Mcdonald, 2011. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks," Journal of Finance, American Finance Association, vol. 66(1), pages 35-65, February.
    3. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Allen Yikuan Huang & Zheqi Fan, 2026. "Beyond Prompting: An Autonomous Framework for Systematic Factor Investing via Agentic AI," Papers 2603.14288, arXiv.org, revised Apr 2026.
    2. Zongwu Cai & Ying Fang & Ming Lin & Zeqin Liu, 2026. "From Text to Evidence: LLMs in Economics and Finance," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 202618, University of Kansas, Department of Economics, revised Sep 2026.
    3. Felix Drinkall & Janet B. Pierrehumbert & Stefan Zohren, 2024. "Forecasting Credit Ratings: A Case Study where Traditional Methods Outperform Generative LLMs," Papers 2407.17624, arXiv.org, revised Jan 2025.
    4. Arega Denekew & Tesfahun Berehane & Molalign Adam, 2025. "Integrating Large Language Models and CNN-LSTM for Enhanced Portfolio Optimization," SN Operations Research Forum, Springer, vol. 6(4), pages 1-31, December.
    5. Gu, Chen & Kurov, Alexander & Wolfe, Marketa Halova, 2018. "Relief Rallies after FOMC Announcements as a Resolution of Uncertainty," Journal of Empirical Finance, Elsevier, vol. 49(C), pages 1-18.
    6. Hongxin Yu & Lihui Zhang & Meng Zhang & Fengyue Jin & Yibing Wang, 2024. "Coordinated Ramp Metering Considering the Dynamics of Mixed-Autonomy Traffic," Sustainability, MDPI, vol. 16(22), pages 1-26, November.
    7. Oehler, Andreas & Neuss, Charlotte, 2025. "ESG disclosure vs. ESG ratings: Consistent information value?," International Review of Financial Analysis, Elsevier, vol. 107(C).
    8. Daniel Russo, 2023. "Approximation Benefits of Policy Gradient Methods with Aggregated States," Management Science, INFORMS, vol. 69(11), pages 6898-6911, November.
    9. D. Sujatha & A. Krishna Sudheer & Elamurugan Balasundaram, 2025. "Black swan dynamics: a network-based framework for systemic risk detection and mitigation," Risk Management, Palgrave Macmillan, vol. 27(4), pages 1-31, December.
    10. Müller, Karsten, 2020. "German forecasters' narratives: How informative are German business cycle forecast reports?," Working Papers 23, German Research Foundation's Priority Programme 1859 "Experience and Expectation. Historical Foundations of Economic Behaviour", Humboldt University Berlin.
    11. Aaron L. Garavito-Acosta & Wilmer Martinez-Rivera & Juan J. Ospina-Tejeiro & Edgar Caicedo-Garcia, 2026. "Experimenting with Large Language Models for Inflation Forecasting in Colombia," IHEID Working Papers 23-2026, Economics Section, The Graduate Institute of International Studies, revised 01 Sep 2026.
    12. Tulika Saha & Sriparna Saha & Pushpak Bhattacharyya, 2020. "Towards sentiment aided dialogue policy learning for multi-intent conversations using hierarchical reinforcement learning," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-28, July.
    13. Aaryan Gupta & Vinya Dengre & Hamza Abubakar Kheruwala & Manan Shah, 2020. "Comprehensive review of text-mining applications in finance," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 6(1), pages 1-25, December.
    14. Yan Luo & Linying Zhou, 2020. "Textual tone in corporate financial disclosures: a survey of the literature," International Journal of Disclosure and Governance, Palgrave Macmillan, vol. 17(2), pages 101-110, September.
    15. David M. Arseneau & Alejandro Drexler & Mitsuhiro Osada, 2022. "Central Bank Communication about Climate Change," Finance and Economics Discussion Series 2022-031, Board of Governors of the Federal Reserve System (U.S.).
    16. Jiao Ji & Oleksandr Talavera & Shuxing Yin, 2018. "The Hidden Information Content: Evidence from the Tone of Independent Director Reports," Working Papers 2018-28, Swansea University, School of Management.
    17. Moreno-Pérez, Carlos & Minozzo, Marco, 2026. "Monetary policy uncertainty in Mexico: An unsupervised approach," International Economics, Elsevier, vol. 186(C).
    18. Jang, Junkyu, 2025. "Selective news selection model for explainable stock prediction via cross-attention integration," Finance Research Letters, Elsevier, vol. 85(PD).
    19. Hamid Ebrahimi, 2026. "A Novel Evidential Uncertainty Framework for Hybrid Models in Rainfall Simulation," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 40(2), pages 1-24, January.
    20. Bennani, Hamza & Romelli, Davide, 2024. "Exploring the informativeness and drivers of tone during committee meetings: The case of the Federal Reserve," Journal of International Money and Finance, Elsevier, vol. 148(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2502.01992. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.