IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2203.06539.html
   My bibliography  Save this paper

Regression Monte Carlo for Impulse Control

Author

Listed:
  • Mike Ludkovski

Abstract

I develop a numerical algorithm for stochastic impulse control in the spirit of Regression Monte Carlo for optimal stopping. The approach consists in generating statistical surrogates (aka functional approximators) for the continuation function. The surrogates are recursively trained by empirical regression over simulated state trajectories. In parallel, the same surrogates are used to learn the intervention function characterizing the optimal impulse amounts. I discuss appropriate surrogate types for this task, as well as the choice of training sets. Case studies from forest rotation and irreversible investment illustrate the numerical scheme and highlight its flexibility and extensibility. Implementation in \texttt{R} is provided as a publicly available package posted on GitHub.

Suggested Citation

  • Mike Ludkovski, 2022. "Regression Monte Carlo for Impulse Control," Papers 2203.06539, arXiv.org.
    • Handle: RePEc:arx:papers:2203.06539
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2203.06539
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Longstaff, Francis A & Schwartz, Eduardo S, 2001. "Valuing American Options by Simulation: A Simple Least-Squares Approach," The Review of Financial Studies, Society for Financial Studies, vol. 14(1), pages 113-147.
    2. Christensen, Sören, 2014. "On the solution of general impulse control problems using superharmonic functions," Stochastic Processes and their Applications, Elsevier, vol. 124(1), pages 709-729.
    3. Michael Kohler, 2008. "A regression-based smoothing spline Monte Carlo algorithm for pricing American options in discrete time," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 92(2), pages 153-178, May.
    4. Aïd, René & Federico, Salvatore & Pham, Huyên & Villeneuve, Bertrand, 2015. "Explicit investment rules with time-to-build and uncertainty," Journal of Economic Dynamics and Control, Elsevier, vol. 51(C), pages 240-256.
    5. Alvarez, Luis H.R., 2011. "Optimal capital accumulation under price uncertainty and costly reversibility," Journal of Economic Dynamics and Control, Elsevier, vol. 35(10), pages 1769-1788, October.
    6. Abel Cadenillas & Fernando Zapatero, 2000. "Classical and Impulse Stochastic Control of the Exchange Rate Using Interest Rates and Reserves," Mathematical Finance, Wiley Blackwell, vol. 10(2), pages 141-156, April.
    7. Alvarez, Luis H.R. & Koskela, Erkki, 2007. "Taxation and rotation age under stochastic forest stand value," Journal of Environmental Economics and Management, Elsevier, vol. 54(1), pages 113-127, July.
    8. Matteo Basei, 2018. "Optimal price management in retail energy markets: an impulse control problem with asymptotic estimates," Papers 1803.08166, arXiv.org, revised Mar 2019.
    9. Erhan Bayraktar & Michael Ludkovski, 2010. "Inventory management with partially observed nonstationary demand," Annals of Operations Research, Springer, vol. 176(1), pages 7-39, April.
    10. Alain Bensoussan & Benoît Chevalier-Roignant, 2019. "Sequential Capacity Expansion Options," Operations Research, INFORMS, vol. 67(1), pages 33-57, January.
    11. Salvatore Federico & Mauro Rosestolato & Elisa Tacconi, 2018. "Irreversible investment with fixed adjustment costs: a stochastic impulse control approach," Papers 1801.04491, arXiv.org, revised Feb 2019.
    12. Alvarez, Luis H.R. & Koskela, Erkki, 2007. "Optimal harvesting under resource stock and price uncertainty," Journal of Economic Dynamics and Control, Elsevier, vol. 31(7), pages 2461-2485, July.
    13. Mike Ludkovski, 2020. "mlOSP: Towards a Unified Implementation of Regression Monte Carlo Algorithms," Papers 2012.00729, arXiv.org, revised Oct 2022.
    14. Jianqiang Hu & Cheng Zhang & Chenbo Zhu, 2016. "( s , S ) Inventory Systems with Correlated Demands," INFORMS Journal on Computing, INFORMS, vol. 28(4), pages 603-611, November.
    15. Bayraktar, Erhan & Kyprianou, Andreas E. & Yamazaki, Kazutoshi, 2014. "Optimal dividends in the dual model under transaction costs," Insurance: Mathematics and Economics, Elsevier, vol. 54(C), pages 133-143.
    16. Guthrie, Graeme, 2012. "Uncertainty and the trade-off between scale and flexibility in investment," Journal of Economic Dynamics and Control, Elsevier, vol. 36(11), pages 1718-1728.
    17. Matteo Basei, 2019. "Optimal price management in retail energy markets: an impulse control problem with asymptotic estimates," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 89(3), pages 355-383, June.
    18. repec:dau:papers:123456789/14292 is not listed on IDEAS
    19. repec:cdl:anderf:qt43n1k4jb is not listed on IDEAS
    20. Irmina Czarna & Zbigniew Palmowski, 2009. "De Finetti's dividend problem and impulse control for a two-dimensional insurance risk process," Papers 0906.2100, arXiv.org, revised Feb 2011.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Salvatore Federico & Mauro Rosestolato & Elisa Tacconi, 2018. "Irreversible investment with fixed adjustment costs: a stochastic impulse control approach," Papers 1801.04491, arXiv.org, revised Feb 2019.
    2. A. Max Reppen & H. Mete Soner & Valentin Tissot-Daguette, 2022. "Deep Stochastic Optimization in Finance," Papers 2205.04604, arXiv.org.
    3. Diego Zabaljauregui, 2019. "A fixed-point policy-iteration-type algorithm for symmetric nonzero-sum stochastic impulse control games," Papers 1909.03574, arXiv.org, revised Jun 2020.
    4. Spiros H. Martzoukos & Nayia Pospori & Lenos Trigeorgis, 2024. "Corporate investment decisions with switch flexibility, constraints, and path-dependency," Review of Quantitative Finance and Accounting, Springer, vol. 62(3), pages 1223-1250, April.
    5. Perera, Sandun & Gupta, Varun & Buckley, Winston, 2020. "Management of online server congestion using optimal demand throttling," European Journal of Operational Research, Elsevier, vol. 285(1), pages 324-342.
    6. Koch, Torben & Vargiolu, Tiziano, 2019. "Optimal Installation of Solar Panels with Price Impact: a Solvable Singular Stochastic Control Problem," Center for Mathematical Economics Working Papers 627, Center for Mathematical Economics, Bielefeld University.
    7. Magnus Perninge, 2018. "A limited-feedback approximation scheme for optimal switching problems with execution delays," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 87(3), pages 347-382, June.
    8. Michael Ludkovski, 2015. "Kriging Metamodels and Experimental Design for Bermudan Option Pricing," Papers 1509.02179, arXiv.org, revised Oct 2016.
    9. Lotte Hezewijk & Nico P. Dellaert & Willem L. Jaarsveld, 2025. "On non-negative auto-correlated integer demand processes," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 101(2), pages 135-161, April.
    10. Manafzadeh Dizbin, Nima & Tan, Barış, 2020. "Optimal control of production-inventory systems with correlated demand inter-arrival and processing times," International Journal of Production Economics, Elsevier, vol. 228(C).
    11. Ren'e Aid & Lamia Ben Ajmia & M'hamed Gaigi & Mohamed Mnif, 2021. "Nonzero-sum stochastic impulse games with an application in competitive retail energy markets," Papers 2112.10213, arXiv.org.
    12. A. Max Reppen & H. Mete Soner & Valentin Tissot-Daguette, 2022. "Neural Optimal Stopping Boundary," Papers 2205.04595, arXiv.org, revised May 2023.
    13. van Hezewijk, Lotte & Dellaert, Nico P. & van Jaarsveld, Willem L., 2025. "Scalable deep reinforcement learning in the non-stationary capacitated lot sizing problem," International Journal of Production Economics, Elsevier, vol. 284(C).
    14. Mike Ludkovski, 2020. "mlOSP: Towards a Unified Implementation of Regression Monte Carlo Algorithms," Papers 2012.00729, arXiv.org, revised Oct 2022.
    15. Nacira Agram & Giulia Pucci & Bernt Øksendal, 2024. "Impulse Control of Conditional McKean–Vlasov Jump Diffusions," Journal of Optimization Theory and Applications, Springer, vol. 200(3), pages 1100-1130, March.
    16. Adriana Piazza & Bernardo Pagnoncelli, 2015. "The stochastic Mitra–Wan forestry model: risk neutral and risk averse cases," Journal of Economics, Springer, vol. 115(2), pages 175-194, June.
    17. Sebastian Becker & Patrick Cheridito & Arnulf Jentzen & Timo Welti, 2019. "Solving high-dimensional optimal stopping problems using deep learning," Papers 1908.01602, arXiv.org, revised Aug 2021.
    18. Harun Avci & Kagan Gokbayrak & Emre Nadar, 2020. "Structural Results for Average‐Cost Inventory Models with Markov‐Modulated Demand and Partial Information," Production and Operations Management, Production and Operations Management Society, vol. 29(1), pages 156-173, January.
    19. Raquel M. Gaspar & Sara D. Lopes & Bernardo Sequeira, 2020. "Neural Network Pricing of American Put Options," Risks, MDPI, vol. 8(3), pages 1-24, July.
    20. Pekka Matomäki, 2012. "On solvability of a two-sided singular control problem," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 76(3), pages 239-271, December.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2203.06539. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.