IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2502.14141.html
   My bibliography  Save this paper

Gaining efficiency in deep policy gradient method for continuous-time optimal control problems

Author

Listed:
  • Arash Fahim
  • Md. Arafatur Rahman

Abstract

In this paper, we propose an efficient implementation of deep policy gradient method (PGM) for optimal control problems in continuous time. The proposed method has the ability to manage the allocation of computational resources, number of trajectories, and complexity of architecture of the neural network. This is, in particular, important for continuous-time problems that require a fine time discretization. Each step of this method focuses on a different time scale and learns a policy, modeled by a neural network, for a discretized optimal control problem. The first step has the coarsest time discretization. As we proceed to other steps, the time discretization becomes finer. The optimal trained policy in each step is also used to provide data for the next step. We accompany the multi-scale deep PGM with a theoretical result on allocation of computational resources to obtain a targeted efficiency and test our methods on the linear-quadratic stochastic optimal control problem.

Suggested Citation

  • Arash Fahim & Md. Arafatur Rahman, 2025. "Gaining efficiency in deep policy gradient method for continuous-time optimal control problems," Papers 2502.14141, arXiv.org, revised Feb 2025.
    • Handle: RePEc:arx:papers:2502.14141
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2502.14141
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. repec:dau:papers:123456789/5524 is not listed on IDEAS
    2. Jianfeng Zhang & Jia Zhuo, 2014. "Monotone schemes for fully nonlinear parabolic path dependent PDEs," Journal of Financial Engineering (JFE), World Scientific Publishing Co. Pte. Ltd., vol. 1(01), pages 1-23.
    3. Huyên Pham & Xavier Warin & Maximilien Germain, 2021. "Neural networks-based backward scheme for fully nonlinear PDEs," Partial Differential Equations and Applications, Springer, vol. 2(1), pages 1-24, February.
    4. A. Max Reppen & H. Mete Soner & Valentin Tissot-Daguette, 2023. "Deep stochastic optimization in finance," Digital Finance, Springer, vol. 5(1), pages 91-111, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jiang Yu Nguwi & Nicolas Privault, 2023. "A deep learning approach to the probabilistic numerical solution of path-dependent partial differential equations," Partial Differential Equations and Applications, Springer, vol. 4(4), pages 1-20, August.
    2. Thibaut Mastrolia & Dylan Possamaï, 2018. "Moral Hazard Under Ambiguity," Journal of Optimization Theory and Applications, Springer, vol. 179(2), pages 452-500, November.
    3. Bruno Bouchard & Gr'egoire Loeper & Xiaolu Tan, 2021. "A $C^{0,1}$-functional It\^o's formula and its applications in mathematical finance," Papers 2101.03759, arXiv.org.
    4. Jean-Franc{c}ois Chassagneux & Junchao Chen & Noufel Frikha, 2022. "Deep Runge-Kutta schemes for BSDEs," Papers 2212.14372, arXiv.org.
    5. Bruno Bouchard & Grégoire Loeper & Xiaolu Tan, 2021. "A C^{0,1}-functional Itô's formula and its applications in mathematical finance," Working Papers hal-03105342, HAL.
    6. Teng, Long, 2022. "Gradient boosting-based numerical methods for high-dimensional backward stochastic differential equations," Applied Mathematics and Computation, Elsevier, vol. 426(C).
    7. Jeonggyu Huh & Jaegi Jeon, 2024. "Pontryagin-Guided Policy Optimization for Merton's Portfolio Problem," Papers 2412.13101, arXiv.org, revised Jan 2025.
    8. William Lefebvre & Gr'egoire Loeper & Huy^en Pham, 2022. "Differential learning methods for solving fully nonlinear PDEs," Papers 2205.09815, arXiv.org.
    9. Jiang, Weixin & Wang, Junfang & Varbanov, Petar Sabev & Yuan, Qing & Chen, Yujie & Wang, Bohong & Yu, Bo, 2024. "Hybrid data-mechanism-driven model of the unsteady soil temperature field for long-buried crude oil pipelines with non-isothermal batch transportation," Energy, Elsevier, vol. 292(C).
    10. Lorenc Kapllani & Long Teng, 2024. "A backward differential deep learning-based algorithm for solving high-dimensional nonlinear backward stochastic differential equations," Papers 2404.08456, arXiv.org.
    11. E. Subha & V. Jothi Prakash & S. Arul Antran Vijay, 2025. "A novel arctic fox survival strategy inspired optimization algorithm," Journal of Combinatorial Optimization, Springer, vol. 49(1), pages 1-73, January.
    12. Aghapour, Ahmad & Arian, Hamid & Seco, Luis, 2025. "Deep-time neural networks: An efficient approach for solving high-dimensional PDEs," Applied Mathematics and Computation, Elsevier, vol. 488(C).
    13. Bouchard, Bruno & Loeper, Grégoire & Tan, Xiaolu, 2022. "A ℂ0,1-functional Itô’s formula and its applications in mathematical finance," Stochastic Processes and their Applications, Elsevier, vol. 148(C), pages 299-323.
    14. Pieter van Staden & Peter Forsyth & Yuying Li, 2024. "Smart leverage? Rethinking the role of Leveraged Exchange Traded Funds in constructing portfolios to beat a benchmark," Papers 2412.05431, arXiv.org, revised Mar 2025.
    15. Bruno Bouchard & Grégoire Loeper & Xiaolu Tan, 2022. "A C^{0,1}-functional Itô's formula and its applications in mathematical finance," Post-Print hal-03105342, HAL.
    16. Ren, Zhenjie & Tan, Xiaolu, 2017. "On the convergence of monotone schemes for path-dependent PDEs," Stochastic Processes and their Applications, Elsevier, vol. 127(6), pages 1738-1762.
    17. Maximilien Germain & Joseph Mikael & Xavier Warin, 2022. "Numerical Resolution of McKean-Vlasov FBSDEs Using Neural Networks," Methodology and Computing in Applied Probability, Springer, vol. 24(4), pages 2557-2586, December.
    18. William Lefebvre & Grégoire Loeper & Huyên Pham, 2023. "Differential learning methods for solving fully nonlinear PDEs," Digital Finance, Springer, vol. 5(1), pages 183-229, March.
    19. Maximilien Germain & Mathieu Laurière & Huyên Pham & Xavier Warin, 2022. "DeepSets and their derivative networks for solving symmetric PDEs ," Post-Print hal-03154116, HAL.
    20. Jiefei Yang & Guanglian Li, 2024. "Gradient-enhanced sparse Hermite polynomial expansions for pricing and hedging high-dimensional American options," Papers 2405.02570, arXiv.org.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2502.14141. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.