IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1403.2229.html
   My bibliography  Save this paper

A reinforcement learning extension to the Almgren-Chriss model for optimal trade execution

Author

Listed:
  • Dieter Hendricks
  • Diane Wilcox

Abstract

Reinforcement learning is explored as a candidate machine learning technique to enhance existing analytical solutions for optimal trade execution with elements from the market microstructure. Given a volume-to-trade, fixed time horizon and discrete trading periods, the aim is to adapt a given volume trajectory such that it is dynamic with respect to favourable/unfavourable conditions during realtime execution, thereby improving overall cost of trading. We consider the standard Almgren-Chriss model with linear price impact as a candidate base model. This model is popular amongst sell-side institutions as a basis for arrival price benchmark execution algorithms. By training a learning agent to modify a volume trajectory based on the market's prevailing spread and volume dynamics, we are able to improve post-trade implementation shortfall by up to 10.3% on average compared to the base model, based on a sample of stocks and trade sizes in the South African equity market.

Suggested Citation

  • Dieter Hendricks & Diane Wilcox, 2014. "A reinforcement learning extension to the Almgren-Chriss model for optimal trade execution," Papers 1403.2229, arXiv.org.
    • Handle: RePEc:arx:papers:1403.2229
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1403.2229
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Holthausen, Robert W. & Leftwich, Richard W. & Mayers, David, 1990. "Large-block transactions, the speed of response, and temporary and permanent stock-price effects," Journal of Financial Economics, Elsevier, vol. 26(1), pages 71-95, July.
    2. Abhijit Gosavi, 2009. "Reinforcement Learning: A Tutorial Survey and Recent Advances," INFORMS Journal on Computing, INFORMS, vol. 21(2), pages 178-192, May.
    3. Gur Huberman & Werner Stanzl, 2005. "Optimal Liquidity Trading," Review of Finance, European Finance Association, vol. 9(2), pages 165-200.
    4. James McCulloch, 2007. "Relative volume as a doubly stochastic binomial point process," Quantitative Finance, Taylor & Francis Journals, vol. 7(1), pages 55-62.
    5. Brock, William A. & Kleidon, Allan W., 1992. "Periodic market closure and trading volume : A model of intraday bids and asks," Journal of Economic Dynamics and Control, Elsevier, vol. 16(3-4), pages 451-489.
    6. Dimitri Vayanos, 2001. "Strategic Trading in a Dynamic Noisy Market," Journal of Finance, American Finance Association, vol. 56(1), pages 131-171, February.
    7. Chan, Louis K C & Lakonishok, Josef, 1995. "The Behavior of Stock Prices around Institutional Trades," Journal of Finance, American Finance Association, vol. 50(4), pages 1147-1174, September.
    8. Anat R. Admati, Paul Pfleiderer, 1988. "A Theory of Intraday Patterns: Volume and Price Variability," The Review of Financial Studies, Society for Financial Studies, vol. 1(1), pages 3-40.
    9. Bertsimas, Dimitris & Lo, Andrew W., 1998. "Optimal control of execution costs," Journal of Financial Markets, Elsevier, vol. 1(1), pages 1-50, April.
    10. Robert Almgren, 2003. "Optimal execution with nonlinear impact functions and trading-enhanced risk," Applied Mathematical Finance, Taylor & Francis Journals, vol. 10(1), pages 1-18.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Brian Ning & Franco Ho Ting Lin & Sebastian Jaimungal, 2018. "Double Deep Q-Learning for Optimal Execution," Papers 1812.06600, arXiv.org, revised Jun 2020.
    2. Alexandre Carbonneau & Frédéric Godin, 2023. "Deep Equal Risk Pricing of Financial Derivatives with Non-Translation Invariant Risk Measures," Risks, MDPI, vol. 11(8), pages 1-27, August.
    3. Schnaubelt, Matthias, 2020. "Deep reinforcement learning for the optimal placement of cryptocurrency limit orders," FAU Discussion Papers in Economics 05/2020, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    4. Ben Hambly & Renyuan Xu & Huining Yang, 2021. "Recent Advances in Reinforcement Learning in Finance," Papers 2112.04553, arXiv.org, revised Feb 2023.
    5. Soohan Kim & Jimyeong Kim & Hong Kee Sul & Youngjoon Hong, 2023. "An Adaptive Dual-level Reinforcement Learning Approach for Optimal Trade Execution," Papers 2307.10649, arXiv.org.
    6. Söhnke M. Bartram & Jürgen Branke & Mehrshad Motahari, 2020. "Artificial intelligence in asset management," Working Papers 20202001, Cambridge Judge Business School, University of Cambridge.
    7. Ivan Jericevich & Patrick Chang & Tim Gebbie, 2020. "Comparing the market microstructure between two South African exchanges," Papers 2011.04367, arXiv.org.
    8. Shuo Sun & Rundong Wang & Bo An, 2021. "Reinforcement Learning for Quantitative Trading," Papers 2109.13851, arXiv.org.
    9. Matthew Dicks & Andrew Paskaramoorthy & Tim Gebbie, 2023. "Many learning agents interacting with an agent-based market model," Papers 2303.07393, arXiv.org, revised Nov 2023.
    10. Patrick Chang & Etienne Pienaar & Tim Gebbie, 2020. "Malliavin-Mancino estimators implemented with non-uniform fast Fourier transforms," Papers 2003.02842, arXiv.org, revised Nov 2020.
    11. Donovan Platt & Tim Gebbie, 2016. "The Problem of Calibrating an Agent-Based Model of High-Frequency Trading," Papers 1606.01495, arXiv.org, revised Mar 2017.
    12. Xiaodong Li & Pangjing Wu & Chenxin Zou & Qing Li, 2022. "Hierarchical Deep Reinforcement Learning for VWAP Strategy Optimization," Papers 2212.14670, arXiv.org.
    13. Woo Jae Byun & Bumkyu Choi & Seongmin Kim & Joohyun Jo, 2023. "Practical Application of Deep Reinforcement Learning to Optimal Trade Execution," FinTech, MDPI, vol. 2(3), pages 1-16, June.
    14. Yuchao Dong, 2022. "Randomized Optimal Stopping Problem in Continuous time and Reinforcement Learning Algorithm," Papers 2208.02409, arXiv.org, revised Sep 2023.
    15. Haoran Wang & Xun Yu Zhou, 2019. "Continuous-Time Mean-Variance Portfolio Selection: A Reinforcement Learning Framework," Papers 1904.11392, arXiv.org, revised May 2019.
    16. Andrew Paskaramoorthy & Terence van Zyl & Tim Gebbie, 2020. "A Framework for Online Investment Algorithms," Papers 2003.13360, arXiv.org.
    17. Bruno Gašperov & Stjepan Begušić & Petra Posedel Šimović & Zvonko Kostanjčar, 2021. "Reinforcement Learning Approaches to Optimal Market Making," Mathematics, MDPI, vol. 9(21), pages 1-22, October.
    18. Alexandre Carbonneau & Fr'ed'eric Godin, 2021. "Deep equal risk pricing of financial derivatives with non-translation invariant risk measures," Papers 2107.11340, arXiv.org.
    19. Dieter Hendricks, 2016. "Using real-time cluster configurations of streaming asynchronous features as online state descriptors in financial markets," Papers 1603.06805, arXiv.org, revised May 2017.
    20. Wenhang Bao & Xiao-yang Liu, 2019. "Multi-Agent Deep Reinforcement Learning for Liquidation Strategy Analysis," Papers 1906.11046, arXiv.org.
    21. Ben Hambly & Renyuan Xu & Huining Yang, 2020. "Policy Gradient Methods for the Noisy Linear Quadratic Regulator over a Finite Horizon," Papers 2011.10300, arXiv.org, revised Jun 2021.
    22. Yuchen Fang & Kan Ren & Weiqing Liu & Dong Zhou & Weinan Zhang & Jiang Bian & Yong Yu & Tie-Yan Liu, 2021. "Universal Trading for Order Execution with Oracle Policy Distillation," Papers 2103.10860, arXiv.org.
    23. Harvey, M. & Hendricks, D. & Gebbie, T. & Wilcox, D., 2017. "Deviations in expected price impact for small transaction volumes under fee restructuring," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 471(C), pages 416-426.
    24. Schnaubelt, Matthias, 2022. "Deep reinforcement learning for the optimal placement of cryptocurrency limit orders," European Journal of Operational Research, Elsevier, vol. 296(3), pages 993-1006.
    25. Feiyang Pan & Tongzhe Zhang & Ling Luo & Jia He & Shuoling Liu, 2022. "Learn Continuously, Act Discretely: Hybrid Action-Space Reinforcement Learning For Optimal Execution," Papers 2207.11152, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Obizhaeva, Anna A. & Wang, Jiang, 2013. "Optimal trading strategy and supply/demand dynamics," Journal of Financial Markets, Elsevier, vol. 16(1), pages 1-32.
    2. Vayanos, Dimitri & Wang, Jiang, 2013. "Market Liquidity—Theory and Empirical Evidence ," Handbook of the Economics of Finance, in: G.M. Constantinides & M. Harris & R. M. Stulz (ed.), Handbook of the Economics of Finance, volume 2, chapter 0, pages 1289-1361, Elsevier.
    3. Hong, Harrison & Rady, Sven, 2002. "Strategic trading and learning about liquidity," Journal of Financial Markets, Elsevier, vol. 5(4), pages 419-450, October.
    4. Oehmke, Martin, 2014. "Liquidating illiquid collateral," Journal of Economic Theory, Elsevier, vol. 149(C), pages 183-210.
    5. Wei Cui & Anthony Brabazon & Michael O'Neill, 2011. "Dynamic trade execution: a grammatical evolution approach," International Journal of Financial Markets and Derivatives, Inderscience Enterprises Ltd, vol. 2(1/2), pages 4-31.
    6. Dimitri Vayanos, 1999. "Strategic Trading and Welfare in a Dynamic Market," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 66(2), pages 219-254.
    7. Griese, Knut & Kempf, Alexander, 2005. "Liquiditätsdynamik am deutschen Aktienmarkt," CFR Working Papers 05-12, University of Cologne, Centre for Financial Research (CFR).
    8. Dimitri Vayanos & Jiang Wang, 2012. "Market Liquidity -- Theory and Empirical Evidence," NBER Working Papers 18251, National Bureau of Economic Research, Inc.
    9. Ryosuke Ishii, 2009. "Optimal Execution in an Evolutionary Setting," KIER Working Papers 670, Kyoto University, Institute of Economic Research.
    10. Michael J. O'Neill & Geoffrey J. Warren, 2019. "Evaluating fund capacity: issues and methods," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 59(S1), pages 773-800, April.
    11. Olivier Guéant, 2016. "The Financial Mathematics of Market Liquidity: From Optimal Execution to Market Making," Post-Print hal-01393136, HAL.
    12. Joseph Golec, 2007. "Are the Insider Trades of a Large Institutional Investor Informed?," The Financial Review, Eastern Finance Association, vol. 42(2), pages 161-190, May.
    13. He, Hua & Mamaysky, Harry, 2005. "Dynamic trading policies with price impact," Journal of Economic Dynamics and Control, Elsevier, vol. 29(5), pages 891-930, May.
    14. Oehmke, Martin, 2014. "Liquidating illiquid collateral," LSE Research Online Documents on Economics 84518, London School of Economics and Political Science, LSE Library.
    15. Dimitri Vayanos, 2001. "Strategic Trading in a Dynamic Noisy Market," Journal of Finance, American Finance Association, vol. 56(1), pages 131-171, February.
    16. Hatheway, Frank & Kwan, Amy & Zheng, Hui, 2017. "An Empirical Analysis of Market Segmentation on U.S. Equity Markets," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 52(6), pages 2399-2427, December.
    17. Olivier Guéant & Charles-Albert Lehalle, 2015. "General Intensity Shapes In Optimal Liquidation," Mathematical Finance, Wiley Blackwell, vol. 25(3), pages 457-495, July.
    18. Schoeneborn, Torsten & Schied, Alexander, 2007. "Liquidation in the Face of Adversity: Stealth Vs. Sunshine Trading, Predatory Trading Vs. Liquidity Provision," MPRA Paper 5548, University Library of Munich, Germany.
    19. Samuel N. Cohen & Lukasz Szpruch, 2011. "A limit order book model for latency arbitrage," Papers 1110.4811, arXiv.org.
    20. Steven L. Heston & Robert A. Korajczyk & Ronnie Sadka, 2010. "Intraday Patterns in the Cross‐section of Stock Returns," Journal of Finance, American Finance Association, vol. 65(4), pages 1369-1407, August.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1403.2229. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.