IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2211.03107.html
   My bibliography  Save this paper

FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning

Author

Listed:
  • Xiao-Yang Liu
  • Ziyi Xia
  • Jingyang Rui
  • Jiechao Gao
  • Hongyang Yang
  • Ming Zhu
  • Christina Dan Wang
  • Zhaoran Wang
  • Jian Guo

Abstract

Finance is a particularly difficult playground for deep reinforcement learning. However, establishing high-quality market environments and benchmarks for financial reinforcement learning is challenging due to three major factors, namely, low signal-to-noise ratio of financial data, survivorship bias of historical data, and model overfitting in the backtesting stage. In this paper, we present an openly accessible FinRL-Meta library that has been actively maintained by the AI4Finance community. First, following a DataOps paradigm, we will provide hundreds of market environments through an automatic pipeline that collects dynamic datasets from real-world markets and processes them into gym-style market environments. Second, we reproduce popular papers as stepping stones for users to design new trading strategies. We also deploy the library on cloud platforms so that users can visualize their own results and assess the relative performance via community-wise competitions. Third, FinRL-Meta provides tens of Jupyter/Python demos organized into a curriculum and a documentation website to serve the rapidly growing community. FinRL-Meta is available at: https://github.com/AI4Finance-Foundation/FinRL-Meta

Suggested Citation

  • Xiao-Yang Liu & Ziyi Xia & Jingyang Rui & Jiechao Gao & Hongyang Yang & Ming Zhu & Christina Dan Wang & Zhaoran Wang & Jian Guo, 2022. "FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning," Papers 2211.03107, arXiv.org.
    • Handle: RePEc:arx:papers:2211.03107
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2211.03107
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Raberto, Marco & Cincotti, Silvano & Focardi, Sergio M. & Marchesi, Michele, 2001. "Agent-based simulation of a financial market," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 299(1), pages 319-327.
    2. Agrim Gupta & Silvio Savarese & Surya Ganguli & Li Fei-Fei, 2021. "Embodied intelligence via learning and evolution," Nature Communications, Nature, vol. 12(1), pages 1-12, December.
    3. Leo Ardon & Nelson Vadori & Thomas Spooner & Mengda Xu & Jared Vann & Sumitra Ganesh, 2021. "Towards a fully RL-based Market Simulator," Papers 2110.06829, arXiv.org, revised Nov 2021.
    4. Mao Guan & Xiao-Yang Liu, 2021. "Explainable Deep Reinforcement Learning for Portfolio Management: An Empirical Approach," Papers 2111.03995, arXiv.org, revised Dec 2021.
    5. Andrea Coletta & Matteo Prata & Michele Conti & Emanuele Mercanti & Novella Bartolini & Aymeric Moulin & Svitlana Vyetrenko & Tucker Balch, 2021. "Towards Realistic Market Simulations: a Generative Adversarial Networks Approach," Papers 2110.13287, arXiv.org.
    6. Wenhang Bao & Xiao-yang Liu, 2019. "Multi-Agent Deep Reinforcement Learning for Liquidation Strategy Analysis," Papers 1906.11046, arXiv.org.
    7. Brown, Stephen J, et al, 1992. "Survivorship Bias in Performance Studies," The Review of Financial Studies, Society for Financial Studies, vol. 5(4), pages 553-580.
    8. Tidor-Vlad Pricope, 2021. "Deep Reinforcement Learning in Quantitative Algorithmic Trading: A Review," Papers 2106.00123, arXiv.org.
    9. Zechu Li & Xiao-Yang Liu & Jiahao Zheng & Zhaoran Wang & Anwar Walid & Jian Guo, 2021. "FinRL-Podracer: High Performance and Scalable Deep Reinforcement Learning for Quantitative Finance," Papers 2111.05188, arXiv.org.
    10. David Silver & Aja Huang & Chris J. Maddison & Arthur Guez & Laurent Sifre & George van den Driessche & Julian Schrittwieser & Ioannis Antonoglou & Veda Panneershelvam & Marc Lanctot & Sander Dieleman, 2016. "Mastering the game of Go with deep neural networks and tree search," Nature, Nature, vol. 529(7587), pages 484-489, January.
    11. David Byrd & Antigoni Polychroniadou, 2020. "Differentially Private Secure Multi-Party Computation for Federated Learning in Financial Applications," Papers 2010.05867, arXiv.org.
    12. Burton G. Malkiel, 2003. "Passive Investment Strategies and Efficient Markets," European Financial Management, European Financial Management Association, vol. 9(1), pages 1-10, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shuyang Wang & Diego Klabjan, 2023. "An Ensemble Method of Deep Reinforcement Learning for Automated Cryptocurrency Trading," Papers 2309.00626, arXiv.org.
    2. Xiao-Yang Liu & Guoxuan Wang & Hongyang Yang & Daochen Zha, 2023. "FinGPT: Democratizing Internet-scale Data for Financial Large Language Models," Papers 2307.10485, arXiv.org, revised Nov 2023.
    3. Junyi Ye & Bhaskar Goswami & Jingyi Gu & Ajim Uddin & Guiling Wang, 2024. "From Factor Models to Deep Learning: Machine Learning in Reshaping Empirical Asset Pricing," Papers 2403.06779, arXiv.org.
    4. Hongyang Yang & Xiao-Yang Liu & Christina Dan Wang, 2023. "FinGPT: Open-Source Financial Large Language Models," Papers 2306.06031, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xiao-Yang Liu & Jingyang Rui & Jiechao Gao & Liuqing Yang & Hongyang Yang & Zhaoran Wang & Christina Dan Wang & Jian Guo, 2021. "FinRL-Meta: A Universe of Near-Real Market Environments for Data-Driven Deep Reinforcement Learning in Quantitative Finance," Papers 2112.06753, arXiv.org, revised Mar 2022.
    2. Xiao-Yang Liu & Hongyang Yang & Jiechao Gao & Christina Dan Wang, 2021. "FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance," Papers 2111.09395, arXiv.org.
    3. Xiao-Yang Liu & Hongyang Yang & Qian Chen & Runjia Zhang & Liuqing Yang & Bowen Xiao & Christina Dan Wang, 2020. "FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance," Papers 2011.09607, arXiv.org, revised Mar 2022.
    4. Trianti, Nikoletta, 2015. "Portfolio Management in Public Pension Reserve Funds/Gestión de carteras en los Fondos de Reserva de las Pensiones Públicas," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 33, pages 985-1008, Septiembr.
    5. Iwao Maeda & David deGraw & Michiharu Kitano & Hiroyasu Matsushima & Hiroki Sakaji & Kiyoshi Izumi & Atsuo Kato, 2020. "Deep Reinforcement Learning in Agent Based Financial Market Simulation," JRFM, MDPI, vol. 13(4), pages 1-17, April.
    6. Sánchez-González, Carlos & Sarto, José Luis & Vicente, Luis, 2017. "The efficiency of mutual fund companies: Evidence from an innovative network SBM approach," Omega, Elsevier, vol. 71(C), pages 114-128.
    7. Bessler, Wolfgang & Drobetz, Wolfgang & Zimmermann, Heinz, 2007. "Conditional Performance Evaluation for German Mutual Equity Funds," Working papers 2007/22, Faculty of Business and Economics - University of Basel.
    8. Anthony Zador & Sean Escola & Blake Richards & Bence Ölveczky & Yoshua Bengio & Kwabena Boahen & Matthew Botvinick & Dmitri Chklovskii & Anne Churchland & Claudia Clopath & James DiCarlo & Surya Gangu, 2023. "Catalyzing next-generation Artificial Intelligence through NeuroAI," Nature Communications, Nature, vol. 14(1), pages 1-7, December.
    9. Jinan Zou & Qingying Zhao & Yang Jiao & Haiyao Cao & Yanxi Liu & Qingsen Yan & Ehsan Abbasnejad & Lingqiao Liu & Javen Qinfeng Shi, 2022. "Stock Market Prediction via Deep Learning Techniques: A Survey," Papers 2212.12717, arXiv.org, revised Feb 2023.
    10. Stephen Brown & William Goetzmann & Bing Liang & Christopher Schwarz, 2008. "Mandatory Disclosure and Operational Risk: Evidence from Hedge Fund Registration," Journal of Finance, American Finance Association, vol. 63(6), pages 2785-2815, December.
    11. Butt, Prof. Khursheed A & Pandow, Bilal Ahmad, 2013. "An analysis into the Stock Selectivity skill of Indian Fund Managers," MPRA Paper 83500, University Library of Munich, Germany, revised 2013.
    12. A. Colin Cameron & Anthony D. Hall, 2003. "A Survival Analysis of Australian Equity Mutual Funds," Australian Journal of Management, Australian School of Business, vol. 28(2), pages 209-226, September.
    13. Tian Zhu & Merry H. Ma, 2022. "Deriving the Optimal Strategy for the Two Dice Pig Game via Reinforcement Learning," Stats, MDPI, vol. 5(3), pages 1-14, August.
    14. Xiaoyue Li & John M. Mulvey, 2023. "Optimal Portfolio Execution in a Regime-switching Market with Non-linear Impact Costs: Combining Dynamic Program and Neural Network," Papers 2306.08809, arXiv.org.
    15. Pedro Afonso Fernandes, 2024. "Forecasting with Neuro-Dynamic Programming," Papers 2404.03737, arXiv.org.
    16. Terrence Hallahan & Robert Faff, 2001. "Induced persistence or reversals in fund performance?: the effect of survivorship bias," Applied Financial Economics, Taylor & Francis Journals, vol. 11(2), pages 119-126.
    17. Ian Tonks, 2005. "Performance Persistence of Pension-Fund Managers," The Journal of Business, University of Chicago Press, vol. 78(5), pages 1917-1942, September.
    18. Chen, Zhimin & Ibragimov, Rustam, 2019. "One country, two systems? The heavy-tailedness of Chinese A- and H- share markets," Emerging Markets Review, Elsevier, vol. 38(C), pages 115-141.
    19. Nathan Companez & Aldeida Aleti, 2016. "Can Monte-Carlo Tree Search learn to sacrifice?," Journal of Heuristics, Springer, vol. 22(6), pages 783-813, December.
    20. Yuchen Zhang & Wei Yang, 2022. "Breakthrough invention and problem complexity: Evidence from a quasi‐experiment," Strategic Management Journal, Wiley Blackwell, vol. 43(12), pages 2510-2544, December.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2211.03107. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.