IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1905.00107.html
   My bibliography  Save this paper

Statistical Learning for Probability-Constrained Stochastic Optimal Control

Author

Listed:
  • Alessandro Balata
  • Michael Ludkovski
  • Aditya Maheshwari
  • Jan Palczewski

Abstract

We investigate Monte Carlo based algorithms for solving stochastic control problems with probabilistic constraints. Our motivation comes from microgrid management, where the controller tries to optimally dispatch a diesel generator while maintaining low probability of blackouts. The key question we investigate are empirical simulation procedures for learning the admissible control set that is specified implicitly through a probability constraint on the system state. We propose a variety of relevant statistical tools including logistic regression, Gaussian process regression, quantile regression and support vector machines, which we then incorporate into an overall Regression Monte Carlo (RMC) framework for approximate dynamic programming. Our results indicate that using logistic or Gaussian process regression to estimate the admissibility probability outperforms the other options. Our algorithms offer an efficient and reliable extension of RMC to probability-constrained control. We illustrate our findings with two case studies for the microgrid problem.

Suggested Citation

  • Alessandro Balata & Michael Ludkovski & Aditya Maheshwari & Jan Palczewski, 2019. "Statistical Learning for Probability-Constrained Stochastic Optimal Control," Papers 1905.00107, arXiv.org, revised Aug 2020.
    • Handle: RePEc:arx:papers:1905.00107
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1905.00107
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Longstaff, Francis A & Schwartz, Eduardo S, 2001. "Valuing American Options by Simulation: A Simple Least-Squares Approach," The Review of Financial Studies, Society for Financial Studies, vol. 14(1), pages 113-147.
    2. Andrieu, L. & Henrion, R. & Römisch, W., 2010. "A model for dynamic chance constraints in hydro power reservoir management," European Journal of Operational Research, Elsevier, vol. 207(2), pages 579-589, December.
    3. Abebe Geletu & Michael Klöppel & Hui Zhang & Pu Li, 2013. "Advances and applications of chance-constrained approaches to systems optimisation under uncertainty," International Journal of Systems Science, Taylor & Francis Journals, vol. 44(7), pages 1209-1232.
    4. repec:dau:papers:123456789/4273 is not listed on IDEAS
    5. Jun Xu & J. Scott Long, 2005. "Confidence intervals for predicted outcomes in regression models for categorical outcomes," Stata Journal, StataCorp LP, vol. 5(4), pages 537-559, December.
    6. Roustant, Olivier & Ginsbourger, David & Deville, Yves, 2012. "DiceKriging, DiceOptim: Two R Packages for the Analysis of Computer Experiments by Kriging-Based Metamodeling and Optimization," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 51(i01).
    7. repec:dau:papers:123456789/11531 is not listed on IDEAS
    8. Rene Carmona & Michael Ludkovski, 2010. "Valuation of energy storage: an optimal switching approach," Quantitative Finance, Taylor & Francis Journals, vol. 10(4), pages 359-374.
    9. Ying Jiao & Olivier Klopfenstein & Peter Tankov, 2017. "Hedging under multiple risk constraints," Finance and Stochastics, Springer, vol. 21(2), pages 361-396, April.
    10. Longstaff, Francis A & Schwartz, Eduardo S, 2001. "Valuing American Options by Simulation: A Simple Least-Squares Approach," University of California at Los Angeles, Anderson Graduate School of Management qt43n1k4jb, Anderson Graduate School of Management, UCLA.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Tao Chen & Michael Ludkovski, 2019. "A Machine Learning Approach to Adaptive Robust Utility Maximization and Hedging," Papers 1912.00244, arXiv.org, revised May 2020.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Balata, Alessandro & Ludkovski, Michael & Maheshwari, Aditya & Palczewski, Jan, 2021. "Statistical learning for probability-constrained stochastic optimal control," European Journal of Operational Research, Elsevier, vol. 290(2), pages 640-656.
    2. Hanfeld, Marc & Schlüter, Stephan, 2016. "Operating a swing option on today's gas markets: How least squares Monte Carlo works and why it is beneficial," FAU Discussion Papers in Economics 10/2016, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    3. Nemat Safarov & Colin Atkinson, 2017. "Natural Gas-Fired Power Plants Valuation And Optimization Under Lévy Copulas And Regime Switching," International Journal of Theoretical and Applied Finance (IJTAF), World Scientific Publishing Co. Pte. Ltd., vol. 20(01), pages 1-38, February.
    4. Bastian Felix, 2012. "Gas Storage Valuation: A Comparative Simulation Study," EWL Working Papers 1201, University of Duisburg-Essen, Chair for Management Science and Energy Economics, revised Apr 2014.
    5. Nemat Safarov & Colin Atkinson, 2016. "Natural gas-fired power plants valuation and optimisation under Levy copulas and regime-switching," Papers 1607.01207, arXiv.org, revised Jul 2016.
    6. Fabozzi, Frank J. & Paletta, Tommaso & Tunaru, Radu, 2017. "An improved least squares Monte Carlo valuation method based on heteroscedasticity," European Journal of Operational Research, Elsevier, vol. 263(2), pages 698-706.
    7. W. Ackooij & X. Warin, 2020. "On conditional cuts for stochastic dual dynamic programming," EURO Journal on Computational Optimization, Springer;EURO - The Association of European Operational Research Societies, vol. 8(2), pages 173-199, June.
    8. Trotter, Ian M. & Gomes, Marília Fernandes Maciel & Braga, Marcelo José & Brochmann, Bjørn & Lie, Ole Nikolai, 2016. "Optimal LNG (liquefied natural gas) regasification scheduling for import terminals with storage," Energy, Elsevier, vol. 105(C), pages 80-88.
    9. Michael Ludkovski, 2015. "Kriging Metamodels and Experimental Design for Bermudan Option Pricing," Papers 1509.02179, arXiv.org, revised Oct 2016.
    10. Nadarajah, Selvaprabu & Margot, François & Secomandi, Nicola, 2017. "Comparison of least squares Monte Carlo methods with applications to energy real options," European Journal of Operational Research, Elsevier, vol. 256(1), pages 196-204.
    11. Ruimeng Hu, 2019. "Deep Learning for Ranking Response Surfaces with Applications to Optimal Stopping Problems," Papers 1901.03478, arXiv.org, revised Mar 2020.
    12. Juri Hinz & Tanya Tarnopolskaya & Jeremy Yee, 2020. "Efficient algorithms of pathwise dynamic programming for decision optimization in mining operations," Annals of Operations Research, Springer, vol. 286(1), pages 583-615, March.
    13. Michael Ludkovski & Aditya Maheshwari, 2018. "Simulation Methods for Stochastic Storage Problems: A Statistical Learning Perspective," Papers 1803.11309, arXiv.org.
    14. Mike Ludkovski, 2020. "mlOSP: Towards a Unified Implementation of Regression Monte Carlo Algorithms," Papers 2012.00729, arXiv.org, revised Oct 2022.
    15. Kitapbayev, Yerkin & Moriarty, John & Mancarella, Pierluigi, 2015. "Stochastic control and real options valuation of thermal storage-enabled demand response from flexible district energy systems," Applied Energy, Elsevier, vol. 137(C), pages 823-831.
    16. Thompson, Matt, 2016. "Natural gas storage valuation, optimization, market and credit risk management," Journal of Commodity Markets, Elsevier, vol. 2(1), pages 26-44.
    17. Giovanni Mottola, 2014. "A stochastic switching control model arising in general OTC contracts with contingent CSA in presence of CVA, collateral and funding," Papers 1412.1469, arXiv.org.
    18. Alessio Trivella & Selvaprabu Nadarajah & Stein-Erik Fleten & Denis Mazieres & David Pisinger, 2021. "Managing Shutdown Decisions in Merchant Commodity and Energy Production: A Social Commerce Perspective," Manufacturing & Service Operations Management, INFORMS, vol. 23(2), pages 311-330, March.
    19. Cortazar, Gonzalo & Naranjo, Lorenzo & Sainz, Felipe, 2021. "Optimal decision policy for real options under general Markovian dynamics," European Journal of Operational Research, Elsevier, vol. 288(2), pages 634-647.
    20. R. Mark Reesor & T. James Marshall, 2020. "Forest of Stochastic Trees: A Method for Valuing Multiple Exercise Options," JRFM, MDPI, vol. 13(5), pages 1-31, May.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1905.00107. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.