IDEAS home Printed from https://ideas.repec.org/a/gam/jjrfmx/v14y2021i2p54-d487912.html
   My bibliography  Save this article

Observation Time Effects in Reinforcement Learning on Contracts for Difference

Author

Listed:
  • Maximilian Wehrmann

    (Hochschule Ruhr West, University of Applied Sciences, Duisburger Str. 100, 45479 Mülheim an der Ruhr, Germany)

  • Nico Zengeler

    (Hochschule Ruhr West, University of Applied Sciences, Duisburger Str. 100, 45479 Mülheim an der Ruhr, Germany)

  • Uwe Handmann

    (Hochschule Ruhr West, University of Applied Sciences, Duisburger Str. 100, 45479 Mülheim an der Ruhr, Germany)

Abstract

In this paper, we present a study on Reinforcement Learning optimization models for automatic trading, in which we focus on the effects of varying the observation time. Our Reinforcement Learning agents feature a Convolutional Neural Network (CNN) together with Long Short-Term Memory (LSTM) and act on the basis of different observation time spans. Each agent tries to maximize trading profit by buying or selling one of a number of contracts in a simulated market environment for Contracts for Difference (CfD), considering correlations between individual assets by architecture. To decide which action to take on a specific contract, an agent develops a policy which relies on an observation of the whole market for a certain period of time. We investigate whether or not there exists an optimal observation sequence length, and conclude that such a value depends on market dynamics.

Suggested Citation

  • Maximilian Wehrmann & Nico Zengeler & Uwe Handmann, 2021. "Observation Time Effects in Reinforcement Learning on Contracts for Difference," JRFM, MDPI, vol. 14(2), pages 1-15, January.
    • Handle: RePEc:gam:jjrfmx:v:14:y:2021:i:2:p:54-:d:487912
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1911-8074/14/2/54/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1911-8074/14/2/54/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Nico Zengeler & Uwe Handmann, 2020. "Contracts for Difference: A Reinforcement Learning Approach," JRFM, MDPI, vol. 13(4), pages 1-12, April.
    2. Kumar Venkataraman, 2001. "Automated Versus Floor Trading: An Analysis of Execution Costs on the Paris and New York Exchanges," Journal of Finance, American Finance Association, vol. 56(4), pages 1445-1485, August.
    3. Fischer, Thomas G., 2018. "Reinforcement learning in financial markets - a survey," FAU Discussion Papers in Economics 12/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Weiguang Han & Boyi Zhang & Qianqian Xie & Min Peng & Yanzhao Lai & Jimin Huang, 2023. "Select and Trade: Towards Unified Pair Trading with Hierarchical Reinforcement Learning," Papers 2301.10724, arXiv.org, revised Feb 2023.
    2. He, Yan & Wu, Chunchi, 2006. "Is stock price rounded for economic reasons in the Chinese markets?," Global Finance Journal, Elsevier, vol. 17(1), pages 119-135, September.
    3. Olaf Korn & Paolo Krischak & Erik Theissen, 2019. "Illiquidity transmission from spot to futures markets," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 39(10), pages 1228-1249, October.
    4. Ravi Kashyap, 2016. "A Tale of Two Consequences: Intended and Unintended Outcomes of the Japan TOPIX Tick Size Changes," Papers 1602.00839, arXiv.org, revised Jul 2019.
    5. Gousgounis, Eleni & Onur, Esen, 2018. "The effect of pit closure on futures trading," Journal of Commodity Markets, Elsevier, vol. 10(C), pages 69-90.
    6. Angelo Ranaldo, 2002. "Market Dynamics Around Public Information Arrivals," FAME Research Paper Series rp45, International Center for Financial Asset Management and Engineering.
    7. repec:dau:papers:123456789/295 is not listed on IDEAS
    8. Battalio, Robert & Ellul, Andrew & Jennings, Robert, 2005. "Reputation effects in trading on the New York Stock Exchange," LSE Research Online Documents on Economics 24659, London School of Economics and Political Science, LSE Library.
    9. Charl Maree & Christian W. Omlin, 2022. "Balancing Profit, Risk, and Sustainability for Portfolio Management," Papers 2207.02134, arXiv.org.
    10. Bryant, Henry L. & Haigh, Michael S., 2002. "Bid-Ask Spreads In Commodity Futures Markets," Working Papers 28587, University of Maryland, Department of Agricultural and Resource Economics.
    11. Berkman, Henk & Brailsford, Tim & Frino, Alex, 2005. "A note on execution costs for stock index futures: Information versus liquidity effects," Journal of Banking & Finance, Elsevier, vol. 29(3), pages 565-577, March.
    12. Longbing Cao, 2021. "AI in Finance: Challenges, Techniques and Opportunities," Papers 2107.09051, arXiv.org.
    13. Bourghelle, David & Declerck, Fany, 2004. "Why markets should not necessarily reduce the tick size," Journal of Banking & Finance, Elsevier, vol. 28(2), pages 373-398, February.
    14. Huang, Yu Chuan, 2004. "The market microstructure and relative performance of Taiwan stock index futures: a comparison of the Singapore exchange and the Taiwan futures exchange," Journal of Financial Markets, Elsevier, vol. 7(3), pages 335-350, June.
    15. Clapham, Benjamin & Gomber, Peter & Lausen, Jens & Panz, Sven, 2021. "Liquidity provider incentives in fragmented securities markets," Journal of Empirical Finance, Elsevier, vol. 60(C), pages 16-38.
    16. Uwe Helmes & Julia Henker & Thomas Henker & Tom Smith, 2017. "Effect of the ban on short selling on market prices and volatility," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 57(3), pages 727-757, September.
    17. Ben Hambly & Renyuan Xu & Huining Yang, 2021. "Recent Advances in Reinforcement Learning in Finance," Papers 2112.04553, arXiv.org, revised Feb 2023.
    18. Schnaubelt, Matthias, 2022. "Deep reinforcement learning for the optimal placement of cryptocurrency limit orders," European Journal of Operational Research, Elsevier, vol. 296(3), pages 993-1006.
    19. G. Wuyts, 2007. "Stock Market Liquidity.Determinants and Implications," Review of Business and Economic Literature, KU Leuven, Faculty of Economics and Business (FEB), Review of Business and Economic Literature, vol. 0(2), pages 279-316.
    20. Yu Chuan Huang & Jian‐Hsin Chou, 2007. "Order Imbalance and Its Impact on Market Performance: Order‐driven vs. Quote‐driven Markets," Journal of Business Finance & Accounting, Wiley Blackwell, vol. 34(9‐10), pages 1596-1614, November.
    21. Patricia Chelley Steeley & Brian Lucey, 2008. "The Microstructure of the Irish Stock Market," Multinational Finance Journal, Multinational Finance Journal, vol. 12(3-4), pages 279-311, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jjrfmx:v:14:y:2021:i:2:p:54-:d:487912. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.