IDEAS home Printed from https://ideas.repec.org/p/ven/wpaper/201415.html

Q-Learning-based financial trading systems with applications

Author

Listed:
  • Marco Corazza

    (Department of Economics, Ca� Foscari University of Venice; Advanced School of Economics in Venice.)

  • Francesco Bertoluzzo

    (Department of Economics, Ca� Foscari University of Venice.)

Abstract

The design of financial trading systems (FTSs) is a subject of high interest both for the academic environment and for the professional one due to the promises by machine learning methodologies. In this paper we consider the Reinforcement Learning-based policy evaluation approach known as Q-Learning algorithm (QLa). QLa is an algorithm which real-time optimizes its behavior in relation to the responses it gets from the environment in which it operates. In particular: first we introduce the essential aspects of QLa which are of interest for our purposes; second we present some original FTSs based on differently configured QLas; then we apply such FTSs to an artificial time series of daily stock prices and to six real ones from the Italian stock market belonging to the FTSE MIB basket. The results we achieve are generally satisfactory.

Suggested Citation

  • Marco Corazza & Francesco Bertoluzzo, 2014. "Q-Learning-based financial trading systems with applications," Working Papers 2014:15, Department of Economics, University of Venice "Ca' Foscari".
    • Handle: RePEc:ven:wpaper:2014:15
    as

    Download full text from publisher

    File URL: https://www.unive.it/web/fileadmin/user_upload/dipartimenti/DEC/doc/Pubblicazioni_scientifiche/working_papers/2014/WP_DSE_corazza_bertoluzzo_15_14.pdf
    File Function: First version, anno
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Bekiros, Stelios D., 2010. "Heterogeneous trading strategies with adaptive fuzzy Actor-Critic reinforcement learning: A behavioral approach," Journal of Economic Dynamics and Control, Elsevier, vol. 34(6), pages 1153-1170, June.
    2. Francesco Bertoluzzo & Marco Corazza, 2006. "Financial trading systems: Is recurrent reinforcement the via?," Working Papers 141, Department of Applied Mathematics, Università Ca' Foscari Venezia.
    3. Xufre Casqueiro, Patricia & Rodrigues, Antonio J.L., 2006. "Neuro-dynamic trading methods," European Journal of Operational Research, Elsevier, vol. 175(3), pages 1400-1412, December.
    4. J. Doyne Farmer, 2002. "Market force, ecology and evolution," Industrial and Corporate Change, Oxford University Press and the Associazione ICC, vol. 11(5), pages 895-953, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Taylan Kabbani & Ekrem Duman, 2022. "Deep Reinforcement Learning Approach for Trading Automation in The Stock Market," Papers 2208.07165, arXiv.org.
    2. Johann Lussange & Stefano Vrizzi & Stefano Palminteri & Boris Gutkin, 2024. "Mesoscale effects of trader learning behaviors in financial markets: A multi-agent reinforcement learning study," PLOS ONE, Public Library of Science, vol. 19(4), pages 1-40, April.
    3. Johann Lussange & Stefano Vrizzi & Stefano Palminteri & Boris Gutkin, 2024. "Mesoscale effects of trader learning behaviors in financial markets: A multi-agent reinforcement learning study," Post-Print hal-04790290, HAL.
    4. Fischer, Thomas G., 2018. "Reinforcement learning in financial markets - a survey," FAU Discussion Papers in Economics 12/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    5. Xiao-Yang Liu & Hongyang Yang & Qian Chen & Runjia Zhang & Liuqing Yang & Bowen Xiao & Christina Dan Wang, 2020. "FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance," Papers 2011.09607, arXiv.org, revised Mar 2022.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fischer, Thomas G., 2018. "Reinforcement learning in financial markets - a survey," FAU Discussion Papers in Economics 12/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    2. Bekiros, Stelios D., 2015. "Heuristic learning in intraday trading under uncertainty," Journal of Empirical Finance, Elsevier, vol. 30(C), pages 34-49.
    3. Citera, Emanuele & Sau, Lino, 2019. "Complexity, Conventions and Instability: the role of monetary policy," Department of Economics and Statistics Cognetti de Martiis. Working Papers 201924, University of Turin.
    4. Anufriev, Mikhail & Bottazzi, Giulio & Pancotto, Francesca, 2006. "Equilibria, stability and asymptotic dominance in a speculative market with heterogeneous traders," Journal of Economic Dynamics and Control, Elsevier, vol. 30(9-10), pages 1787-1835.
    5. Challet, Damien, 2008. "Inter-pattern speculation: Beyond minority, majority and $-games," Journal of Economic Dynamics and Control, Elsevier, vol. 32(1), pages 85-100, January.
    6. Pankaj Kumar, 2021. "Deep Hawkes Process for High-Frequency Market Making," Papers 2109.15110, arXiv.org.
    7. Caccioli, Fabio & Shrestha, Munik & Moore, Cristopher & Farmer, J. Doyne, 2014. "Stability analysis of financial contagion due to overlapping portfolios," Journal of Banking & Finance, Elsevier, vol. 46(C), pages 233-245.
    8. Hommes, C.H. & Wagener, F.O.O., 2008. "Complex evolutionary systems in behavioral finance," CeNDEF Working Papers 08-05, Universiteit van Amsterdam, Center for Nonlinear Dynamics in Economics and Finance.
    9. Hyungjun Park & Min Kyu Sim & Dong Gu Choi, 2019. "An intelligent financial portfolio trading strategy using deep Q-learning," Papers 1907.03665, arXiv.org, revised Nov 2019.
    10. Brock, William A. & Hommes, Cars H. & Wagener, Florian O. O., 2005. "Evolutionary dynamics in markets with many trader types," Journal of Mathematical Economics, Elsevier, vol. 41(1-2), pages 7-42, February.
    11. A. Corcos & J-P Eckmann & A. Malaspinas & Y. Malevergne & D. Sornette, 2002. "Imitation and contrarian behaviour: hyperbolic bubbles, crashes and chaos," Quantitative Finance, Taylor & Francis Journals, vol. 2(4), pages 264-281.
    12. Salzman, Diego & Trifan, Emanuela, 2006. "Emotions, Bayesian Inference, and Financial Decision Making," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 28163, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    13. Gaël Giraud & Céline Rochon, 2010. "Transition to Equilibrium in International Trades," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-00657038, HAL.
    14. Po-Keng Cheng, 2022. "Transitions among states behind interactive agent model," Computational and Mathematical Organization Theory, Springer, vol. 28(1), pages 27-51, March.
    15. Kun Xing & Honggang Li, 2024. "The profitability of interacting trading strategies from an ecological perspective," Annals of Finance, Springer, vol. 20(3), pages 377-394, September.
    16. Winkler, Julian, 2023. "Managing fundamentals versus preferences: Re-balancing portfolios and stock returns," MPRA Paper 119149, University Library of Munich, Germany.
    17. Syed Qasim Shah & Izlin Ismail & Aidial Rizal bin Shahrin, 2020. "Heterogeneous investors and deterioration of market integrity: an analysis of market manipulation cases," Journal of Financial Crime, Emerald Group Publishing Limited, vol. 30(2), pages 389-403, May.
    18. Leal, Sandrine Jacob & Napoletano, Mauro, 2019. "Market stability vs. market resilience: Regulatory policies experiments in an agent-based model with low- and high-frequency trading," Journal of Economic Behavior & Organization, Elsevier, vol. 157(C), pages 15-41.
    19. Amir Mosavi & Pedram Ghamisi & Yaser Faghan & Puhong Duan, 2020. "Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics," Papers 2004.01509, arXiv.org.
    20. Roberto Savona & Maxence Soumare & Jørgen Vitting Andersen, 2015. "Financial Symmetry and Moods in the Market," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-21, April.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C61 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Optimization Techniques; Programming Models; Dynamic Analysis
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • G11 - Financial Economics - - General Financial Markets - - - Portfolio Choice; Investment Decisions

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ven:wpaper:2014:15. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sassano Sonia (email available below). General contact details of provider: https://edirc.repec.org/data/dsvenit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.