IDEAS home Printed from https://ideas.repec.org/a/eee/spapps/v194y2026ics0304414925002923.html

Continuous time reinforcement learning: A random measure approach

Author

Listed:
  • Bender, Christian
  • Thuan, Nguyen Tran

Abstract

We present a random measure approach for modeling exploration, i.e., the execution of measure-valued controls, in continuous-time reinforcement learning with controlled diffusion and jumps. We begin with the case when sampling the randomized control in continuous time takes place on a discrete-time grid and reformulate the resulting SDE as an equation driven by suitable random measures. Our main result is a limit theorem for these random measures as the mesh-size of the sampling grid goes to zero. The resulting limit SDE can be applied for the theoretical analysis of exploratory control problems and for the derivation of learning algorithms.

Suggested Citation

  • Bender, Christian & Thuan, Nguyen Tran, 2026. "Continuous time reinforcement learning: A random measure approach," Stochastic Processes and their Applications, Elsevier, vol. 194(C).
    • Handle: RePEc:eee:spapps:v:194:y:2026:i:c:s0304414925002923
    DOI: 10.1016/j.spa.2025.104848
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304414925002923
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.spa.2025.104848?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Haoran Wang & Xun Yu Zhou, 2020. "Continuous‐time mean–variance portfolio selection: A reinforcement learning framework," Mathematical Finance, Wiley Blackwell, vol. 30(4), pages 1273-1308, October.
    2. Ryan Donnelly & Sebastian Jaimungal, 2022. "Exploratory Control with Tsallis Entropy for Latent Factor Models," Papers 2211.07622, arXiv.org, revised Jan 2024.
    3. Xie, Yingchao, 1994. "Vague convergence of locally integrable martingale measures," Stochastic Processes and their Applications, Elsevier, vol. 52(2), pages 211-227, August.
    4. Xin Guo & Renyuan Xu & Thaleia Zariphopoulou, 2022. "Entropy Regularization for Mean Field Games with Learning," Mathematics of Operations Research, INFORMS, vol. 47(4), pages 3239-3260, November.
    5. Min Dai & Yuchao Dong & Yanwei Jia, 2023. "Learning equilibrium mean‐variance strategy," Mathematical Finance, Wiley Blackwell, vol. 33(4), pages 1166-1212, October.
    6. Wu, Bo & Li, Lingfei, 2024. "Reinforcement learning for continuous-time mean-variance portfolio selection in a regime-switching market," Journal of Economic Dynamics and Control, Elsevier, vol. 158(C).
    7. Yanwei Jia & Xun Yu Zhou, 2021. "Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithms," Papers 2111.11232, arXiv.org, revised Jul 2022.
    8. Yanwei Jia & Xun Yu Zhou, 2021. "Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approach," Papers 2108.06655, arXiv.org, revised Feb 2022.
    9. Xie, Yingchao, 1995. "Limit theorems of Hilbert valued semimartingales and Hilbert valued martingale measures," Stochastic Processes and their Applications, Elsevier, vol. 59(2), pages 277-293, October.
    10. Sun, Yeneng, 2006. "The exact law of large numbers via Fubini extension and characterization of insurable risks," Journal of Economic Theory, Elsevier, vol. 126(1), pages 31-69, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Min Dai & Yuchao Dong & Yanwei Jia & Xun Yu Zhou, 2026. "Merton's Problem with Recursive Perturbed Utility," Papers 2602.13544, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chen Ziyi & Gu Jia-wen, 2025. "Exploratory Utility Maximization Problem with Tsallis Entropy," Papers 2502.01269, arXiv.org.
    2. Boyu Wang & Xuefeng Gao & Lingfei Li, 2026. "Reinforcement learning for continuous-time optimal execution: actor–critic algorithm and error analysis," Finance and Stochastics, Springer, vol. 30(2), pages 597-655, April.
    3. Junyan Ye & Hoi Ying Wong & Kyunghyun Park, 2025. "Robust Exploratory Stopping under Ambiguity in Reinforcement Learning," Papers 2510.10260, arXiv.org, revised Apr 2026.
    4. Yuling Max Chen & Bin Li & David Saunders, 2025. "Exploratory Mean-Variance Portfolio Optimization with Regime-Switching Market Dynamics," Papers 2501.16659, arXiv.org.
    5. Yuling Max Chen & Bin Li & David Saunders, 2025. "Exploratory Mean-Variance with Jumps: An Equilibrium Approach," Papers 2512.09224, arXiv.org.
    6. Thai Nguyen & Pertiny Nkuize, 2026. "Optimal Investment and Entropy-Regularized Learning Under Stochastic Volatility Models with Portfolio Constraints," Papers 2604.22188, arXiv.org.
    7. Wu, Bo & Li, Lingfei, 2024. "Reinforcement learning for continuous-time mean-variance portfolio selection in a regime-switching market," Journal of Economic Dynamics and Control, Elsevier, vol. 158(C).
    8. Yanwei Jia & Xun Yu Zhou, 2022. "q-Learning in Continuous Time," Papers 2207.00713, arXiv.org, revised May 2025.
    9. Xiangyu Cui & Xun Li & Yun Shi & Si Zhao, 2023. "Discrete-Time Mean-Variance Strategy Based on Reinforcement Learning," Papers 2312.15385, arXiv.org.
    10. Yanwei Jia, 2024. "Continuous-time Risk-sensitive Reinforcement Learning via Quadratic Variation Penalty," Papers 2404.12598, arXiv.org, revised Mar 2026.
    11. Min Dai & Yu Sun & Zuo Quan Xu & Xun Yu Zhou, 2024. "Learning to Optimally Stop Diffusion Processes, with Financial Applications," Papers 2408.09242, arXiv.org, revised Aug 2025.
    12. Chau, Huy & Nguyen, Duy & Nguyen, Thai, 2026. "Continuous-time optimal investment with portfolio constraints: A reinforcement learning approach," European Journal of Operational Research, Elsevier, vol. 328(3), pages 1068-1092.
    13. Yu Li & Yuhan Wu & Shuhua Zhang, 2025. "The Exploratory Multi-Asset Mean-Variance Portfolio Selection using Reinforcement Learning," Papers 2505.07537, arXiv.org.
    14. Zhou Fang, 2023. "Continuous-Time Path-Dependent Exploratory Mean-Variance Portfolio Construction," Papers 2303.02298, arXiv.org.
    15. Wanting He & Wenyuan Li & Yunran Wei, 2025. "Periodic evaluation of defined-contribution pension fund: A dynamic risk measure approach," Papers 2508.05241, arXiv.org.
    16. Min Dai & Yuchao Dong & Yanwei Jia & Xun Yu Zhou, 2026. "Merton's Problem with Recursive Perturbed Utility," Papers 2602.13544, arXiv.org.
    17. Yilie Huang & Yanwei Jia & Xun Yu Zhou, 2024. "Mean--Variance Portfolio Selection by Continuous-Time Reinforcement Learning: Algorithms, Regret Analysis, and Empirical Study," Papers 2412.16175, arXiv.org, revised Mar 2026.
    18. Xuefeng Gao & Xunyu Zhou, 2026. "Square-Root Regret Bounds for Continuous-Time Episodic Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 51(1), pages 333-357, January.
    19. Lokman A Abbas-Turki & Jean-Franc{c}ois Chassagneux & Jean-Philippe Lemor & Gr'egoire Loeper & Simon Sananes, 2026. "Stochastic Policy Gradient Methods in the Uncertain Volatility Model," Papers 2605.06670, arXiv.org.
    20. Huy Chau & Duy Nguyen & Thai Nguyen, 2024. "Continuous-time optimal investment with portfolio constraints: a reinforcement learning approach," Papers 2412.10692, arXiv.org.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:spapps:v:194:y:2026:i:c:s0304414925002923. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/505572/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.