IDEAS home Printed from https://ideas.repec.org/p/rtv/ceisrp/552.html

A Reinforcement Learning Algorithm for Trading Commodities

Author

Listed:

Abstract

We propose a Reinforcement Learning (RL) algorithm for generating a trading strategy in a realistic setting, that includes transaction costs and factors driving the asset dynamics. We benchmark our algorithm against the analytical optimal solution, available when factors are linear and transaction costs are quadratic, showing that RL is able to mimic the optimal strategy. Then we consider a more realistic setting, including non-linear dynamics, that better describes the WTI spot prices time series. For these more general dynamics, an optimal strategy is not known and RL becomes a viable alternative. We show that on synthetic data generated from WTI spot prices, the RL agent outperforms a trader that linearizes the model to apply the theoretical optimal strategy.

Suggested Citation

  • Federico Giorgi & Stefano Herzel & Paolo Pigato, 2023. "A Reinforcement Learning Algorithm for Trading Commodities," CEIS Research Paper 552, Tor Vergata University, CEIS, revised 18 Feb 2023.
    • Handle: RePEc:rtv:ceisrp:552
    as

    Download full text from publisher

    File URL: https://ceistorvergata.it/RePEc/rpaper/RP552.pdf
    File Function: Main text
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Merton, Robert C, 1973. "An Intertemporal Capital Asset Pricing Model," Econometrica, Econometric Society, vol. 41(5), pages 867-887, September.
    2. Nicolae Gârleanu & Lasse Heje Pedersen, 2013. "Dynamic Trading with Predictable Returns and Transaction Costs," Journal of Finance, American Finance Association, vol. 68(6), pages 2309-2340, December.
    3. Stefan C. Endres & Carl Sandrock & Walter W. Focke, 2018. "A simplicial homology algorithm for Lipschitz optimisation," Journal of Global Optimization, Springer, vol. 72(2), pages 181-217, October.
    4. Christoffersen, Peter & Lunde, Asger & Olesen, Kasper V., 2019. "Factor Structure in Commodity Futures Return and Volatility," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 54(3), pages 1083-1115, June.
    5. Jay Cao & Jacky Chen & John Hull & Zissis Poulos, 2021. "Deep Hedging of Derivatives Using Reinforcement Learning," Papers 2103.16409, arXiv.org.
    6. Edoardo Vittori & Michele Trapletti & Marcello Restelli, 2020. "Option Hedging with Risk Averse Reinforcement Learning," Papers 2010.12245, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alejandro Rodriguez Dominguez, 2026. "Dynamic Causal Portfolio Choice: Hedging the Rotation of the Common-Driver Manifold," Papers 2607.06702, arXiv.org.
    2. Penaranda, Francisco & Sentana, Enrique, 2024. "Portfolio management with big data," CEPR Discussion Papers 19314, Centre for Economic Policy Research.
    3. Lei Zhao & Lin Cai, 2025. "Robust and Efficient Deep Hedging via Linearized Objective Neural Network," Papers 2502.17757, arXiv.org.
    4. Grønborg, Niels S. & Lunde, Asger & Olesen, Kasper V. & Vander Elst, Harry, 2022. "Realizing correlations across asset classes," Journal of Financial Markets, Elsevier, vol. 59(PA).
    5. Horváth, Ferenc, 2017. "Essays on robust asset pricing," Other publications TiSEM e54d7b33-1f27-4b0e-9f84-f, Tilburg University, School of Economics and Management.
    6. Guo, Ming & Ou-Yang, Hui, 2021. "Alpha decay and Sharpe ratio: Two measures of investor performance," Economic Modelling, Elsevier, vol. 104(C).
    7. Lu, Cheng & Ndiaye, Papa Momar & Simaan, Majeed, 2024. "Improved estimation of the correlation matrix using reinforcement learning and text-based networks," International Review of Financial Analysis, Elsevier, vol. 96(PA).
    8. Yiheng Ding & Gangnan Yuan & Dewei Zuo & Ting Gao, 2025. "Hedging with Sparse Reward Reinforcement Learning," Papers 2503.04218, arXiv.org.
    9. Zheng Gong & Carmine Ventre & John O'Hara, 2021. "The Efficient Hedging Frontier with Deep Neural Networks," Papers 2104.05280, arXiv.org.
    10. Pradosh Simlai, 2009. "Stock returns, size, and book‐to‐market equity," Studies in Economics and Finance, Emerald Group Publishing Limited, vol. 26(3), pages 198-212, July.
    11. Florent Gallien & Serge Kassibrakis & Semyon Malamud, 2018. "Hedge or Rebalance: Optimal Risk Management with Transaction Costs," Risks, MDPI, vol. 6(4), pages 1-14, October.
    12. Baoqiang Zhan & Shu Zhang & Helen S. Du & Xiaoguang Yang, 2022. "Exploring Statistical Arbitrage Opportunities Using Machine Learning Strategy," Computational Economics, Springer;Society for Computational Economics, vol. 60(3), pages 861-882, October.
    13. Luis F. Céspedes & Javier García-Cicco & Diego Saravia, 2014. "Monetary Policy at the Zero Lower Bound: The Chilean Experience," Central Banking, Analysis, and Economic Policies Book Series, in: Sofía Bauducco & Lawrence Christiano & Claudio Raddatz (ed.),Macroeconomic and Financial Stability: challenges for Monetary Policy, edition 1, volume 19, chapter 13, pages 427-460, Central Bank of Chile.
    14. Mr. Shaun K. Roache, 2008. "Commodities and the Market Price of Risk," IMF Working Papers 2008/221, International Monetary Fund.
    15. Alessia Naccarato & Andrea Pierini & Giovanna Ferraro, 2021. "Markowitz portfolio optimization through pairs trading cointegrated strategy in long-term investment," Annals of Operations Research, Springer, vol. 299(1), pages 81-99, April.
    16. Johannes A. Skjeltorp & Bernt Arne Ødegaard, 2009. "The information content of market liquidity: An empirical analysis of liquidity at the Oslo Stock Exchange?," Working Paper 2009/26, Norges Bank.
    17. Auffret, Philippe, 2001. "An alternative unifying measure of welfare gains from risk-sharing," Policy Research Working Paper Series 2676, The World Bank.
    18. Li, Yuming, 1998. "Expected stock returns, risk premiums and volatilities of economic factors1," Journal of Empirical Finance, Elsevier, vol. 5(2), pages 69-97, June.
    19. Andros Gregoriou & Christos Ioannidis, 2007. "Generalized method of moments and present value tests of the consumption-capital asset pricing model under transactions costs: evidence from the UK stock market," Empirical Economics, Springer, vol. 32(1), pages 19-39, April.
    20. Seok Young Hong & Oliver Linton, 2016. "Asymptotic properties of a Nadaraya-Watson type estimator for regression functions of in finite order," CeMMAP working papers 53/16, Institute for Fiscal Studies.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:rtv:ceisrp:552. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Barbara Piazzi (email available below). General contact details of provider: https://edirc.repec.org/data/csrotit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.