IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v308y2023i1p321-335.html
   My bibliography  Save this article

Value function gradient learning for large-scale multistage stochastic programming problems

Author

Listed:
  • Lee, Jinkyu
  • Bae, Sanghyeon
  • Kim, Woo Chang
  • Lee, Yongjae

Abstract

A stagewise decomposition algorithm called “value function gradient learning” (VFGL) is proposed for large-scale multistage stochastic convex programs. VFGL finds the parameter values that best fit the gradient of the value function within a given parametric family. Widely used decomposition algorithms for multistage stochastic programming, such as stochastic dual dynamic programming (SDDP), approximate the value function by adding linear subgradient cuts at each iteration. Although this approach has been successful for linear problems, nonlinear problems may suffer from the increasing size of each subproblem as the iteration proceeds. On the other hand, VFGL has a fixed number of parameters; thus, the size of the subproblems remains constant throughout the iteration. Furthermore, VFGL can learn the parameters by means of stochastic gradient descent, which means that it can be easil0y parallelized and does not require a scenario tree approximation of the underlying uncertainties. VFGL was compared with a deterministic equivalent formulation of the multistage stochastic programming problem and SDDP approaches for three illustrative examples: production planning, hydrothermal generation, and the lifetime financial planning problem. Numerical examples show that VFGL generates high-quality solutions and is computationally efficient.

Suggested Citation

  • Lee, Jinkyu & Bae, Sanghyeon & Kim, Woo Chang & Lee, Yongjae, 2023. "Value function gradient learning for large-scale multistage stochastic programming problems," European Journal of Operational Research, Elsevier, vol. 308(1), pages 321-335.
  • Handle: RePEc:eee:ejores:v:308:y:2023:i:1:p:321-335
    DOI: 10.1016/j.ejor.2022.10.011
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221722007809
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ejor.2022.10.011?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jean-Paul Watson & David Woodruff, 2011. "Progressive hedging innovations for a class of stochastic mixed-integer resource allocation problems," Computational Management Science, Springer, vol. 8(4), pages 355-370, November.
    2. R. T. Rockafellar & Roger J.-B. Wets, 1991. "Scenarios and Policy Aggregation in Optimization Under Uncertainty," Mathematics of Operations Research, INFORMS, vol. 16(1), pages 119-147, February.
    3. Reuven Y. Rubinstein & Ruth Marcus, 1985. "Efficiency of Multivariate Control Variates in Monte Carlo Simulation," Operations Research, INFORMS, vol. 33(3), pages 661-677, June.
    4. David R. Cariño & Terry Kent & David H. Myers & Celine Stacy & Mike Sylvanus & Andrew L. Turner & Kouji Watanabe & William T. Ziemba, 1994. "The Russell-Yasuda Kasai Model: An Asset/Liability Model for a Japanese Insurance Company Using Multistage Stochastic Programming," Interfaces, INFORMS, vol. 24(1), pages 29-49, February.
    5. Merton, Robert C, 1969. "Lifetime Portfolio Selection under Uncertainty: The Continuous-Time Case," The Review of Economics and Statistics, MIT Press, vol. 51(3), pages 247-257, August.
    6. Vincent Guigues, 2014. "SDDP for some interstage dependent risk-averse problems and application to hydro-thermal planning," Computational Optimization and Applications, Springer, vol. 57(1), pages 167-203, January.
    7. Powell, Warren B., 2019. "A unified framework for stochastic optimization," European Journal of Operational Research, Elsevier, vol. 275(3), pages 795-821.
    8. Staino, Alessandro & Russo, Emilio, 2015. "A moment-matching method to generate arbitrage-free scenarios," European Journal of Operational Research, Elsevier, vol. 246(2), pages 619-630.
    9. Harvey M. Wagner & Thomson M. Whitin, 1958. "Dynamic Version of the Economic Lot Size Model," Management Science, INFORMS, vol. 5(1), pages 89-96, October.
    10. Gulpinar, Nalan & Rustem, Berc & Settergren, Reuben, 2004. "Simulation and optimization approaches to scenario tree generation," Journal of Economic Dynamics and Control, Elsevier, vol. 28(7), pages 1291-1315, April.
    11. Z. L. Chen & W. B. Powell, 1999. "Convergent Cutting-Plane and Partial-Sampling Algorithm for Multistage Stochastic Linear Programs with Recourse," Journal of Optimization Theory and Applications, Springer, vol. 102(3), pages 497-524, September.
    12. Shapiro, Alexander, 2011. "Analysis of stochastic dual dynamic programming method," European Journal of Operational Research, Elsevier, vol. 209(1), pages 63-72, February.
    13. Ponomareva, K. & Roman, D. & Date, P., 2015. "An algorithm for moment-matching scenario generation with application to financial portfolio optimisation," European Journal of Operational Research, Elsevier, vol. 240(3), pages 678-687.
    14. Karimi, B. & Fatemi Ghomi, S. M. T. & Wilson, J. M., 2003. "The capacitated lot sizing problem: a review of models and algorithms," Omega, Elsevier, vol. 31(5), pages 365-378, October.
    15. P. Girardeau & V. Leclere & A. B. Philpott, 2015. "On the Convergence of Decomposition Methods for Multistage Stochastic Convex Programs," Mathematics of Operations Research, INFORMS, vol. 40(1), pages 130-145, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Chung-Han Hsieh & Jie-Ling Lu, 2024. "On Accelerating Large-Scale Robust Portfolio Optimization," Papers 2408.07879, arXiv.org.
    2. Wu, Weitiao & Li, Yu, 2024. "Pareto truck fleet sizing for bike relocation with stochastic demand: Risk-averse multi-stage approximate stochastic programming," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 183(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Davi Valladão & Thuener Silva & Marcus Poggi, 2019. "Time-consistent risk-constrained dynamic portfolio optimization with transactional costs and time-dependent returns," Annals of Operations Research, Springer, vol. 282(1), pages 379-405, November.
    2. W. Ackooij & X. Warin, 2020. "On conditional cuts for stochastic dual dynamic programming," EURO Journal on Computational Optimization, Springer;EURO - The Association of European Operational Research Societies, vol. 8(2), pages 173-199, June.
    3. Wu, Dexiang & Wu, Desheng Dash, 2020. "A decision support approach for two-stage multi-objective index tracking using improved lagrangian decomposition," Omega, Elsevier, vol. 91(C).
    4. Soares, Murilo Pereira & Street, Alexandre & Valladão, Davi Michel, 2017. "On the solution variability reduction of Stochastic Dual Dynamic Programming applied to energy planning," European Journal of Operational Research, Elsevier, vol. 258(2), pages 743-760.
    5. Simon Thevenin & Yossiri Adulyasak & Jean-François Cordeau, 2022. "Stochastic Dual Dynamic Programming for Multiechelon Lot Sizing with Component Substitution," INFORMS Journal on Computing, INFORMS, vol. 34(6), pages 3151-3169, November.
    6. Vitor L. de Matos & David P. Morton & Erlon C. Finardi, 2017. "Assessing policy quality in a multistage stochastic program for long-term hydrothermal scheduling," Annals of Operations Research, Springer, vol. 253(2), pages 713-731, June.
    7. Hua, Yikang & Zhao, Dongfang & Wang, Xin & Li, Xiaopeng, 2019. "Joint infrastructure planning and fleet management for one-way electric car sharing under time-varying uncertain demand," Transportation Research Part B: Methodological, Elsevier, vol. 128(C), pages 185-206.
    8. Guigues, Vincent & Juditsky, Anatoli & Nemirovski, Arkadi, 2021. "Constant Depth Decision Rules for multistage optimization under uncertainty," European Journal of Operational Research, Elsevier, vol. 295(1), pages 223-232.
    9. Zhou, Shaorui & Zhang, Hui & Shi, Ning & Xu, Zhou & Wang, Fan, 2020. "A new convergent hybrid learning algorithm for two-stage stochastic programs," European Journal of Operational Research, Elsevier, vol. 283(1), pages 33-46.
    10. Huang, Zhouchun & Zheng, Qipeng Phil, 2020. "A multistage stochastic programming approach for preventive maintenance scheduling of GENCOs with natural gas contract," European Journal of Operational Research, Elsevier, vol. 287(3), pages 1036-1051.
    11. Pritchard, Geoffrey, 2015. "Stochastic inflow modeling for hydropower scheduling problems," European Journal of Operational Research, Elsevier, vol. 246(2), pages 496-504.
    12. Fan, Yingjie & Schwartz, Frank & Voß, Stefan, 2017. "Flexible supply chain planning based on variable transportation modes," International Journal of Production Economics, Elsevier, vol. 183(PC), pages 654-666.
    13. Melega, Gislaine Mara & de Araujo, Silvio Alexandre & Jans, Raf, 2018. "Classification and literature review of integrated lot-sizing and cutting stock problems," European Journal of Operational Research, Elsevier, vol. 271(1), pages 1-19.
    14. Hu, Shaolong & Han, Chuanfeng & Dong, Zhijie Sasha & Meng, Lingpeng, 2019. "A multi-stage stochastic programming model for relief distribution considering the state of road network," Transportation Research Part B: Methodological, Elsevier, vol. 123(C), pages 64-87.
    15. Zhicheng Zhu & Yisha Xiang & Bo Zeng, 2021. "Multicomponent Maintenance Optimization: A Stochastic Programming Approach," INFORMS Journal on Computing, INFORMS, vol. 33(3), pages 898-914, July.
    16. Ferstl, Robert & Weissensteiner, Alex, 2011. "Asset-liability management under time-varying investment opportunities," Journal of Banking & Finance, Elsevier, vol. 35(1), pages 182-192, January.
    17. Nadjib Brahimi & Stéphane Dauzère-Pérès & Najib M. Najid, 2006. "Capacitated Multi-Item Lot-Sizing Problems with Time Windows," Operations Research, INFORMS, vol. 54(5), pages 951-967, October.
    18. Boonmee, Atiwat & Sethanan, Kanchana, 2016. "A GLNPSO for multi-level capacitated lot-sizing and scheduling problem in the poultry industry," European Journal of Operational Research, Elsevier, vol. 250(2), pages 652-665.
    19. Bilel Jarraya & Abdelfettah Bouri, 2013. "A Theoretical Assessment on Optimal Asset Allocations in Insurance Industry," International Journal of Finance & Banking Studies, Center for the Strategic Studies in Business and Finance, vol. 2(4), pages 30-44, October.
    20. Yongxi (Eric) Huang & Yueyue Fan & Chien-Wei Chen, 2014. "An Integrated Biofuel Supply Chain to Cope with Feedstock Seasonality and Uncertainty," Transportation Science, INFORMS, vol. 48(4), pages 540-554, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:308:y:2023:i:1:p:321-335. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.