IDEAS home Printed from
   My bibliography  Save this article

A moment and sum-of-squares extension of dual dynamic programming with application to nonlinear energy storage problems


  • Hohmann, Marc
  • Warrington, Joseph
  • Lygeros, John


We present a finite-horizon optimization algorithm that extends the established concept of Dual Dynamic Programming (DDP) in two ways. First, in contrast to the linear costs, dynamics, and constraints of standard DDP, we consider problems in which all of these can be polynomial functions. Second, we allow the state trajectory to be described by probability distributions rather than point values, and return approximate value functions fitted to these. The algorithm is in part an adaptation of sum-of-squares techniques used in the approximate dynamic programming literature. It alternates between a forward simulation through the horizon, in which the moments of the state distribution are propagated through a succession of single-stage problems, and a backward recursion, in which a new polynomial function is derived for each stage using the moments of the state as fixed data. The value function approximation returned for a given stage is the point-wise maximum of all polynomials derived for that stage. This contrasts with the piecewise affine functions derived in conventional DDP. We prove key convergence properties of the new algorithm, and validate it in simulation on two case studies related to the optimal operation of energy storage devices with nonlinear characteristics. The first is a small borehole storage problem, for which multiple value function approximations can be compared. The second is a larger problem, for which conventional discretized dynamic programming is intractable.

Suggested Citation

  • Hohmann, Marc & Warrington, Joseph & Lygeros, John, 2020. "A moment and sum-of-squares extension of dual dynamic programming with application to nonlinear energy storage problems," European Journal of Operational Research, Elsevier, vol. 283(1), pages 16-32.
  • Handle: RePEc:eee:ejores:v:283:y:2020:i:1:p:16-32
    DOI: 10.1016/j.ejor.2019.10.041

    Download full text from publisher

    File URL:
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    1. Jean Lasserre & Tung Thanh, 2013. "Convex underestimators of polynomials," Journal of Global Optimization, Springer, vol. 56(1), pages 1-25, May.
    2. Nils Löhndorf & David Wozabal & Stefan Minner, 2013. "Optimizing Trading Decisions for Hydro Storage Systems Using Approximate Dual Dynamic Programming," Operations Research, INFORMS, vol. 61(4), pages 810-823, August.
    3. Jiang, X.S. & Jing, Z.X. & Li, Y.Z. & Wu, Q.H. & Tang, W.H., 2014. "Modelling and operation optimization of an integrated energy based direct district water-heating system," Energy, Elsevier, vol. 64(C), pages 375-388.
    4. Cerisola, Santiago & Latorre, Jesus M. & Ramos, Andres, 2012. "Stochastic dual dynamic programming applied to nonconvex hydrothermal models," European Journal of Operational Research, Elsevier, vol. 218(3), pages 687-697.
    5. Soares, Murilo Pereira & Street, Alexandre & Valladão, Davi Michel, 2017. "On the solution variability reduction of Stochastic Dual Dynamic Programming applied to energy planning," European Journal of Operational Research, Elsevier, vol. 258(2), pages 743-760.
    6. Guigues, Vincent & Sagastizábal, Claudia, 2012. "The value of rolling-horizon policies for risk-averse hydro-thermal planning," European Journal of Operational Research, Elsevier, vol. 217(1), pages 129-140.
    7. P. Girardeau & V. Leclere & A. B. Philpott, 2015. "On the Convergence of Decomposition Methods for Multistage Stochastic Convex Programs," Mathematics of Operations Research, INFORMS, vol. 40(1), pages 130-145, February.
    Full references (including those not matched with items on IDEAS)


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:283:y:2020:i:1:p:16-32. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Haili He). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.