IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2511.07434.html

RL-Exec: Impact-Aware Reinforcement Learning for Opportunistic Optimal Liquidation, Outperforms TWAP and a Book-Liquidity VWAP on BTC-USD Replays

Author

Listed:
  • Enzo Duflot
  • Stanislas Robineau

Abstract

We study opportunistic optimal liquidation over fixed deadlines on BTC-USD limit-order books (LOB). We present RL-Exec, a PPO agent trained on historical replays augmented with endogenous transient impact (resilience), partial fills, maker/taker fees, and latency. The policy observes depth-20 LOB features plus microstructure indicators and acts under a sell-only inventory constraint to reach a residual target. Evaluation follows a strict time split (train: Jan-2020; test: Feb-2020) and a per-day protocol: for each test day we run ten independent start times and aggregate to a single daily score, avoiding pseudo-replication. We compare the agent to (i) TWAP and (ii) a VWAP-like baseline allocating using opposite-side order-book liquidity (top-20 levels), both executed on identical timestamps and costs. Statistical inference uses one-sided Wilcoxon signed-rank tests on daily RL-baseline differences with Benjamini-Hochberg FDR correction and bootstrap confidence intervals. On the Feb-2020 test set, RL-Exec significantly outperforms both baselines and the gap increases with the execution horizon (+2-3 bps at 30 min, +7-8 bps at 60 min, +23 bps at 120 min). Code: github.com/Giafferri/RL-Exec

Suggested Citation

  • Enzo Duflot & Stanislas Robineau, 2025. "RL-Exec: Impact-Aware Reinforcement Learning for Opportunistic Optimal Liquidation, Outperforms TWAP and a Book-Liquidity VWAP on BTC-USD Replays," Papers 2511.07434, arXiv.org.
    • Handle: RePEc:arx:papers:2511.07434
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2511.07434
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zihao Zhang & Stefan Zohren & Stephen Roberts, 2018. "BDLOB: Bayesian Deep Convolutional Neural Networks for Limit Order Books," Papers 1811.10041, arXiv.org.
    2. Justin A. Sirignano, 2019. "Deep learning for limit order books," Quantitative Finance, Taylor & Francis Journals, vol. 19(4), pages 549-570, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Petter N. Kolm & Jeremy Turiel & Nicholas Westray, 2023. "Deep order flow imbalance: Extracting alpha at multiple horizons from the limit order book," Mathematical Finance, Wiley Blackwell, vol. 33(4), pages 1044-1081, October.
    2. Yufei Wu & Mahmoud Mahfouz & Daniele Magazzeni & Manuela Veloso, 2021. "Towards Robust Representation of Limit Orders Books for Deep Learning Models," Papers 2110.05479, arXiv.org, revised Dec 2022.
    3. Antonio Briola & Jeremy Turiel & Tomaso Aste, 2020. "Deep Learning modeling of Limit Order Book: a comparative perspective," Papers 2007.07319, arXiv.org, revised Oct 2020.
    4. Antonio Briola & Jeremy Turiel & Riccardo Marcaccioli & Alvaro Cauderan & Tomaso Aste, 2021. "Deep Reinforcement Learning for Active High Frequency Trading," Papers 2101.07107, arXiv.org, revised Aug 2023.
    5. Eghbal Rahimikia & Stefan Zohren & Ser-Huang Poon, 2021. "Realised Volatility Forecasting: Machine Learning via Financial Word Embedding," Papers 2108.00480, arXiv.org, revised Jan 2026.
    6. Jiwon Jung & Kiseop Lee, 2024. "Attention-Based Reading, Highlighting, and Forecasting of the Limit Order Book," Papers 2409.02277, arXiv.org, revised Nov 2024.
    7. Antonio Briola & Silvia Bartolucci & Tomaso Aste, 2024. "Deep Limit Order Book Forecasting," Papers 2403.09267, arXiv.org, revised Jun 2024.
    8. Xianfeng Jiao & Zizhong Li & Chang Xu & Yang Liu & Weiqing Liu & Jiang Bian, 2023. "Microstructure-Empowered Stock Factor Extraction and Utilization," Papers 2308.08135, arXiv.org.
    9. Antonio Briola & Silvia Bartolucci & Tomaso Aste, 2024. "HLOB -- Information Persistence and Structure in Limit Order Books," Papers 2405.18938, arXiv.org, revised Jun 2024.
    10. Andrea Coletta & Joseph Jerome & Rahul Savani & Svitlana Vyetrenko, 2023. "Conditional Generators for Limit Order Book Environments: Explainability, Challenges, and Robustness," Papers 2306.12806, arXiv.org.
    11. Alfred Backhouse & Kang Li & Jakob Foerster & Anisoara Calinescu & Stefan Zohren, 2025. "Painting the market: generative diffusion models for financial limit order book simulation and forecasting," Papers 2509.05107, arXiv.org.
    12. Leonardo Berti & Gjergji Kasneci, 2025. "TLOB: A Novel Transformer Model with Dual Attention for Price Trend Prediction with Limit Order Book Data," Papers 2502.15757, arXiv.org, revised May 2025.
    13. Yufei Wu & Mahmoud Mahfouz & Daniele Magazzeni & Manuela Veloso, 2021. "How Robust are Limit Order Book Representations under Data Perturbation?," Papers 2110.04752, arXiv.org.
    14. Schnaubelt, Matthias, 2022. "Deep reinforcement learning for the optimal placement of cryptocurrency limit orders," European Journal of Operational Research, Elsevier, vol. 296(3), pages 993-1006.
    15. Wenyong Zhang & Lingfei Li & Gongqiu Zhang, 2021. "A Two-Step Framework for Arbitrage-Free Prediction of the Implied Volatility Surface," Papers 2106.07177, arXiv.org, revised Jan 2022.
    16. Gao, Xuefeng & Xu, Tianrun, 2022. "Order scoring, bandit learning and order cancellations," Journal of Economic Dynamics and Control, Elsevier, vol. 134(C).
    17. Luca Grilli & Domenico Santoro, 2022. "Forecasting financial time series with Boltzmann entropy through neural networks," Computational Management Science, Springer, vol. 19(4), pages 665-681, October.
    18. Goodell, John W. & Kumar, Satish & Lim, Weng Marc & Pattnaik, Debidutta, 2021. "Artificial intelligence and machine learning in finance: Identifying foundations, themes, and research clusters from bibliometric analysis," Journal of Behavioral and Experimental Finance, Elsevier, vol. 32(C).
    19. Zihao Zhang & Stefan Zohren & Stephen Roberts, 2019. "Deep Reinforcement Learning for Trading," Papers 1911.10107, arXiv.org.
    20. Adamantios Ntakaris & Moncef Gabbouj & Juho Kanniainen, 2023. "Optimum Output Long Short-Term Memory Cell for High-Frequency Trading Forecasting," Papers 2304.09840, arXiv.org, revised May 2023.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2511.07434. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.