IDEAS home Printed from https://ideas.repec.org/a/gam/jjrfmx/v14y2021i2p54-d487912.html
   My bibliography  Save this article

Observation Time Effects in Reinforcement Learning on Contracts for Difference

Author

Listed:
  • Maximilian Wehrmann

    (Hochschule Ruhr West, University of Applied Sciences, Duisburger Str. 100, 45479 Mülheim an der Ruhr, Germany)

  • Nico Zengeler

    (Hochschule Ruhr West, University of Applied Sciences, Duisburger Str. 100, 45479 Mülheim an der Ruhr, Germany)

  • Uwe Handmann

    (Hochschule Ruhr West, University of Applied Sciences, Duisburger Str. 100, 45479 Mülheim an der Ruhr, Germany)

Abstract

In this paper, we present a study on Reinforcement Learning optimization models for automatic trading, in which we focus on the effects of varying the observation time. Our Reinforcement Learning agents feature a Convolutional Neural Network (CNN) together with Long Short-Term Memory (LSTM) and act on the basis of different observation time spans. Each agent tries to maximize trading profit by buying or selling one of a number of contracts in a simulated market environment for Contracts for Difference (CfD), considering correlations between individual assets by architecture. To decide which action to take on a specific contract, an agent develops a policy which relies on an observation of the whole market for a certain period of time. We investigate whether or not there exists an optimal observation sequence length, and conclude that such a value depends on market dynamics.

Suggested Citation

  • Maximilian Wehrmann & Nico Zengeler & Uwe Handmann, 2021. "Observation Time Effects in Reinforcement Learning on Contracts for Difference," JRFM, MDPI, vol. 14(2), pages 1-15, January.
    • Handle: RePEc:gam:jjrfmx:v:14:y:2021:i:2:p:54-:d:487912
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1911-8074/14/2/54/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1911-8074/14/2/54/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Nico Zengeler & Uwe Handmann, 2020. "Contracts for Difference: A Reinforcement Learning Approach," JRFM, MDPI, vol. 13(4), pages 1-12, April.
    2. Kumar Venkataraman, 2001. "Automated Versus Floor Trading: An Analysis of Execution Costs on the Paris and New York Exchanges," Journal of Finance, American Finance Association, vol. 56(4), pages 1445-1485, August.
    3. Fischer, Thomas G., 2018. "Reinforcement learning in financial markets - a survey," FAU Discussion Papers in Economics 12/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Olaf Korn & Paolo Krischak & Erik Theissen, 2019. "Illiquidity transmission from spot to futures markets," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 39(10), pages 1228-1249, October.
    2. Gousgounis, Eleni & Onur, Esen, 2018. "The effect of pit closure on futures trading," Journal of Commodity Markets, Elsevier, vol. 10(C), pages 69-90.
    3. Battalio, Robert & Ellul, Andrew & Jennings, Robert, 2005. "Reputation effects in trading on the New York Stock Exchange," LSE Research Online Documents on Economics 24659, London School of Economics and Political Science, LSE Library.
    4. Charl Maree & Christian W. Omlin, 2022. "Balancing Profit, Risk, and Sustainability for Portfolio Management," Papers 2207.02134, arXiv.org.
    5. Longbing Cao, 2021. "AI in Finance: Challenges, Techniques and Opportunities," Papers 2107.09051, arXiv.org.
    6. Bourghelle, David & Declerck, Fany, 2004. "Why markets should not necessarily reduce the tick size," Journal of Banking & Finance, Elsevier, vol. 28(2), pages 373-398, February.
    7. Suyeol Yun, 2024. "Pretrained LLM Adapted with LoRA as a Decision Transformer for Offline RL in Quantitative Trading," Papers 2411.17900, arXiv.org.
    8. Huang, Yu Chuan, 2004. "The market microstructure and relative performance of Taiwan stock index futures: a comparison of the Singapore exchange and the Taiwan futures exchange," Journal of Financial Markets, Elsevier, vol. 7(3), pages 335-350, June.
    9. Uwe Helmes & Julia Henker & Thomas Henker & Tom Smith, 2017. "Effect of the ban on short selling on market prices and volatility," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 57(3), pages 727-757, September.
    10. Ben Hambly & Renyuan Xu & Huining Yang, 2021. "Recent Advances in Reinforcement Learning in Finance," Papers 2112.04553, arXiv.org, revised Feb 2023.
    11. G. Wuyts, 2007. "Stock Market Liquidity.Determinants and Implications," Review of Business and Economic Literature, KU Leuven, Faculty of Economics and Business (FEB), Review of Business and Economic Literature, vol. 0(2), pages 279-316.
    12. Blau, Benjamin M., 2018. "Does religiosity affect liquidity in financial markets?," Journal of Behavioral and Experimental Finance, Elsevier, vol. 19(C), pages 72-83.
    13. Qin Lei & Murli Rajan & Xuewu Wang, 2012. "An empirical analysis of corporate insiders' trading performance," China Finance Review International, Emerald Group Publishing Limited, vol. 2(3), pages 246-264, June.
    14. Berkman, Henk & Koch, Paul D., 2008. "Noise trading and the price formation process," Journal of Empirical Finance, Elsevier, vol. 15(2), pages 232-250, March.
    15. Jondeau, Eric & Lahaye, Jérôme & Rockinger, Michael, 2015. "Estimating the price impact of trades in a high-frequency microstructure model with jumps," Journal of Banking & Finance, Elsevier, vol. 61(S2), pages 205-224.
    16. Maria Kasch‐Haroutounian & Erik Theissen, 2009. "Competition between Exchanges: Euronext versus Xetra," European Financial Management, European Financial Management Association, vol. 15(1), pages 181-207, January.
    17. Blonski, Matthias & von Lilienfeld-Toal, Ulf, 2008. "Excess returns and the distinguished player paradox," University of Göttingen Working Papers in Economics 78, University of Goettingen, Department of Economics.
    18. Shuo Sun & Rundong Wang & Bo An, 2021. "Reinforcement Learning for Quantitative Trading," Papers 2109.13851, arXiv.org.
    19. Gajewski, Jean-Francois & Gresse, Carole, 2007. "Centralised order books versus hybrid order books: A paired comparison of trading costs on NSC (Euronext Paris) and SETS (London Stock Exchange)," Journal of Banking & Finance, Elsevier, vol. 31(9), pages 2906-2924, September.
    20. Terrence Hendershott & Ryan Riordan, 2009. "Algorithmic Trading and Information," Working Papers 09-08, NET Institute, revised Aug 2009.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jjrfmx:v:14:y:2021:i:2:p:54-:d:487912. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.