IDEAS home Printed from https://ideas.repec.org/a/inm/orijoc/v35y2023i5p1161-1178.html
   My bibliography  Save this article

A Nonconvex Regularization Scheme for the Stochastic Dual Dynamic Programming Algorithm

Author

Listed:
  • Arnab Bhattacharya

    (Pacific Northwest National Laboratory, Richland, Washington 99352)

  • Jeffrey P. Kharoufeh

    (Department of Industrial Engineering, Clemson University, Clemson, South Carolina 29634)

  • Bo Zeng

    (Department of Industrial Engineering, University of Pittsburgh, Pittsburgh, Pennsylvania 15261)

Abstract

We propose a new nonconvex regularization scheme to improve the performance of the stochastic dual dynamic programming (SDDP) algorithm for solving large-scale multistage stochastic programs. Specifically, we use a class of nonconvex regularization functions, namely folded concave penalty functions, to improve solution quality and the convergence rate of the SDDP procedure. We develop a strategy based on mixed-integer programming to guarantee global optimality of the nonconvex regularization problem. Moreover, we establish provable convergence guarantees for our customized SDDP algorithm. The benefits of our regularization scheme are demonstrated by solving large-scale instances of two multistage stochastic optimization problems.

Suggested Citation

  • Arnab Bhattacharya & Jeffrey P. Kharoufeh & Bo Zeng, 2023. "A Nonconvex Regularization Scheme for the Stochastic Dual Dynamic Programming Algorithm," INFORMS Journal on Computing, INFORMS, vol. 35(5), pages 1161-1178, September.
    • Handle: RePEc:inm:orijoc:v:35:y:2023:i:5:p:1161-1178
    DOI: 10.1287/ijoc.2021.0255
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/ijoc.2021.0255
    Download Restriction: no
    File URL: https://libkey.io/10.1287/ijoc.2021.0255?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Shapiro, Alexander & Tekaya, Wajdi & da Costa, Joari Paulo & Soares, Murilo Pereira, 2013. "Risk neutral and risk averse Stochastic Dual Dynamic Programming method," European Journal of Operational Research, Elsevier, vol. 224(2), pages 375-391.
    2. Alexander Shapiro, 2003. "Inference of statistical bounds for multistage stochastic programming problems," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 58(1), pages 57-68, September.
    3. Kouwenberg, Roy, 2001. "Scenario generation and stochastic programming models for asset liability management," European Journal of Operational Research, Elsevier, vol. 134(2), pages 279-292, October.
    4. Shapiro, Alexander, 2011. "Analysis of stochastic dual dynamic programming method," European Journal of Operational Research, Elsevier, vol. 209(1), pages 63-72, February.
    5. Julia L. Higle & Suvrajeet Sen, 1991. "Stochastic Decomposition: An Algorithm for Two-Stage Linear Programs with Recourse," Mathematics of Operations Research, INFORMS, vol. 16(3), pages 650-669, August.
    6. Wim Ackooij & Welington Oliveira & Yongjia Song, 2019. "On level regularization with normal solutions in decomposition methods for multistage stochastic programming problems," Computational Optimization and Applications, Springer, vol. 74(1), pages 1-42, September.
    7. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    8. Jeff Linderoth & Alexander Shapiro & Stephen Wright, 2006. "The empirical behavior of sampling methods for stochastic programming," Annals of Operations Research, Springer, vol. 142(1), pages 215-241, February.
    9. Oscar Dowson & Lea Kapelevich, 2021. "SDDP.jl : A Julia Package for Stochastic Dual Dynamic Programming," INFORMS Journal on Computing, INFORMS, vol. 33(1), pages 27-33, January.
    10. Z. L. Chen & W. B. Powell, 1999. "Convergent Cutting-Plane and Partial-Sampling Algorithm for Multistage Stochastic Linear Programs with Recourse," Journal of Optimization Theory and Applications, Springer, vol. 102(3), pages 497-524, September.
    11. Michael Freimer & Jeffrey Linderoth & Douglas Thomas, 2012. "The impact of sampling methods on bias and variance in stochastic linear programs," Computational Optimization and Applications, Springer, vol. 51(1), pages 51-75, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Davi Valladão & Thuener Silva & Marcus Poggi, 2019. "Time-consistent risk-constrained dynamic portfolio optimization with transactional costs and time-dependent returns," Annals of Operations Research, Springer, vol. 282(1), pages 379-405, November.
    2. Murwan Siddig & Yongjia Song, 2022. "Adaptive partition-based SDDP algorithms for multistage stochastic linear programming with fixed recourse," Computational Optimization and Applications, Springer, vol. 81(1), pages 201-250, January.
    3. Wim Ackooij & Welington Oliveira & Yongjia Song, 2019. "On level regularization with normal solutions in decomposition methods for multistage stochastic programming problems," Computational Optimization and Applications, Springer, vol. 74(1), pages 1-42, September.
    4. W. Ackooij & X. Warin, 2020. "On conditional cuts for stochastic dual dynamic programming," EURO Journal on Computational Optimization, Springer;EURO - The Association of European Operational Research Societies, vol. 8(2), pages 173-199, June.
    5. D. Ávila & A. Papavasiliou & N. Löhndorf, 2022. "Parallel and distributed computing for stochastic dual dynamic programming," Computational Management Science, Springer, vol. 19(2), pages 199-226, June.
    6. de Queiroz, Anderson Rodrigo, 2016. "Stochastic hydro-thermal scheduling optimization: An overview," Renewable and Sustainable Energy Reviews, Elsevier, vol. 62(C), pages 382-395.
    7. Saif Benjaafar & Daniel Jiang & Xiang Li & Xiaobo Li, 2022. "Dynamic Inventory Repositioning in On-Demand Rental Networks," Management Science, INFORMS, vol. 68(11), pages 7861-7878, November.
    8. Pedro Borges, 2022. "Cut-sharing across trees and efficient sequential sampling for SDDP with uncertainty in the RHS," Computational Optimization and Applications, Springer, vol. 82(3), pages 617-647, July.
    9. Zhou, Shaorui & Zhang, Hui & Shi, Ning & Xu, Zhou & Wang, Fan, 2020. "A new convergent hybrid learning algorithm for two-stage stochastic programs," European Journal of Operational Research, Elsevier, vol. 283(1), pages 33-46.
    10. Soares, Murilo Pereira & Street, Alexandre & Valladão, Davi Michel, 2017. "On the solution variability reduction of Stochastic Dual Dynamic Programming applied to energy planning," European Journal of Operational Research, Elsevier, vol. 258(2), pages 743-760.
    11. Bakker, Hannah & Dunke, Fabian & Nickel, Stefan, 2020. "A structuring review on multi-stage optimization under uncertainty: Aligning concepts from theory and practice," Omega, Elsevier, vol. 96(C).
    12. Xiaotie Chen & David L. Woodruff, 2023. "Software for Data-Based Stochastic Programming Using Bootstrap Estimation," INFORMS Journal on Computing, INFORMS, vol. 35(6), pages 1218-1224, November.
    13. Martin Biel & Mikael Johansson, 2022. "Efficient Stochastic Programming in Julia," INFORMS Journal on Computing, INFORMS, vol. 34(4), pages 1885-1902, July.
    14. Panos Parpas & Berk Ustun & Mort Webster & Quang Kha Tran, 2015. "Importance Sampling in Stochastic Programming: A Markov Chain Monte Carlo Approach," INFORMS Journal on Computing, INFORMS, vol. 27(2), pages 358-377, May.
    15. Powell, Warren B., 2019. "A unified framework for stochastic optimization," European Journal of Operational Research, Elsevier, vol. 275(3), pages 795-821.
    16. Haoxiang Yang & Harsha Nagarajan, 2022. "Optimal Power Flow in Distribution Networks Under N – 1 Disruptions: A Multistage Stochastic Programming Approach," INFORMS Journal on Computing, INFORMS, vol. 34(2), pages 690-709, March.
    17. Yunxiao Deng & Suvrajeet Sen, 2022. "Predictive stochastic programming," Computational Management Science, Springer, vol. 19(1), pages 65-98, January.
    18. Nils Löhndorf & David Wozabal & Stefan Minner, 2013. "Optimizing Trading Decisions for Hydro Storage Systems Using Approximate Dual Dynamic Programming," Operations Research, INFORMS, vol. 61(4), pages 810-823, August.
    19. Lee, Jinkyu & Bae, Sanghyeon & Kim, Woo Chang & Lee, Yongjae, 2023. "Value function gradient learning for large-scale multistage stochastic programming problems," European Journal of Operational Research, Elsevier, vol. 308(1), pages 321-335.
    20. Zéphyr, Luckny & Lang, Pascal & Lamond, Bernard F. & Côté, Pascal, 2017. "Approximate stochastic dynamic programming for hydroelectric production planning," European Journal of Operational Research, Elsevier, vol. 262(2), pages 586-601.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orijoc:v:35:y:2023:i:5:p:1161-1178. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.