IDEAS home Printed from https://ideas.repec.org/a/inm/orijoc/v30y2018i1p106-123.html
   My bibliography  Save this article

Benchmarking a Scalable Approximate Dynamic Programming Algorithm for Stochastic Control of Grid-Level Energy Storage

Author

Listed:
  • Daniel F. Salas

    (Department of Chemical and Biological Engineering, Princeton University, Princeton, New Jersey 08540)

  • Warren B. Powell

    (Department of Operations Research and Financial Engineering, Princeton University, Princeton, New Jersey 08540)

Abstract

We present and benchmark an approximate dynamic programming algorithm that is capable of designing near-optimal control policies for a portfolio of heterogenous storage devices in a time-dependent environment, where wind supply, demand, and electricity prices may evolve stochastically. We found that the algorithm was able to design storage policies that are within 0.08% of optimal on deterministic models, and within 0.86% on stochastic models. We use the algorithm to analyze a dual-storage system with different capacities and losses, and show that the policy properly uses the low-loss device (which is typically much more expensive) for high-frequency variations. We close by demonstrating the algorithm on a five-device system. The algorithm easily scales to handle heterogeneous portfolios of storage devices distributed over the grid and more complex storage networks.

Suggested Citation

  • Daniel F. Salas & Warren B. Powell, 2018. "Benchmarking a Scalable Approximate Dynamic Programming Algorithm for Stochastic Control of Grid-Level Energy Storage," INFORMS Journal on Computing, INFORMS, vol. 30(1), pages 106-123, February.
    • Handle: RePEc:inm:orijoc:v:30:y:2018:i:1:p:106-123
    DOI: 10.1287/ijoc.2017.0768
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/ijoc.2017.0768
    Download Restriction: no
    File URL: https://libkey.io/10.1287/ijoc.2017.0768?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Shapiro, Alexander & Tekaya, Wajdi & da Costa, Joari Paulo & Soares, Murilo Pereira, 2013. "Risk neutral and risk averse Stochastic Dual Dynamic Programming method," European Journal of Operational Research, Elsevier, vol. 224(2), pages 375-391.
    2. Nils Löhndorf & David Wozabal & Stefan Minner, 2013. "Optimizing Trading Decisions for Hydro Storage Systems Using Approximate Dual Dynamic Programming," Operations Research, INFORMS, vol. 61(4), pages 810-823, August.
    3. Sioshansi, Ramteen & Hurlbut, David, 2010. "Market protocols in ERCOT and their effect on wind generation," Energy Policy, Elsevier, vol. 38(7), pages 3192-3197, July.
    4. Huseyin Topaloglu & Warren B. Powell, 2005. "A Distributed Decision-Making Structure for Dynamic Resource Allocation Using Nonlinear Functional Approximations," Operations Research, INFORMS, vol. 53(2), pages 281-297, April.
    5. Kuperman, Alon & Aharon, Ilan, 2011. "Battery-ultracapacitor hybrids for pulsed current loads: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 15(2), pages 981-992, February.
    6. Warren B. Powell & Abraham George & Hugo Simão & Warren Scott & Alan Lamont & Jeffrey Stewart, 2012. "SMART: A Stochastic Multiscale Model for the Analysis of Energy Resources, Technology, and Policy," INFORMS Journal on Computing, INFORMS, vol. 24(4), pages 665-682, November.
    7. Fleten, Stein-Erik & Kristoffersen, Trine Krogh, 2007. "Stochastic programming for optimizing bidding strategies of a Nordic hydropower producer," European Journal of Operational Research, Elsevier, vol. 181(2), pages 916-928, September.
    8. Gregory A. Godfrey & Warren B. Powell, 2001. "An Adaptive, Distribution-Free Algorithm for the Newsvendor Problem with Censored Demands, with Applications to Inventory and Distribution," Management Science, INFORMS, vol. 47(8), pages 1101-1112, August.
    9. Warren Powell & Andrzej Ruszczyński & Huseyin Topaloglu, 2004. "Learning Algorithms for Separable Approximations of Discrete Stochastic Optimization Problems," Mathematics of Operations Research, INFORMS, vol. 29(4), pages 814-836, November.
    10. Huseyin Topaloglu & Warren B. Powell, 2006. "Dynamic-Programming Approximations for Stochastic Time-Staged Integer Multicommodity-Flow Problems," INFORMS Journal on Computing, INFORMS, vol. 18(1), pages 31-42, February.
    11. Philpott, A.B. & de Matos, V.L., 2012. "Dynamic sampling algorithms for multi-stage stochastic programs with risk aversion," European Journal of Operational Research, Elsevier, vol. 218(2), pages 470-483.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Finnah, Benedikt & Gönsch, Jochen & Ziel, Florian, 2022. "Integrated day-ahead and intraday self-schedule bidding for energy storage systems using approximate dynamic programming," European Journal of Operational Research, Elsevier, vol. 301(2), pages 726-746.
    2. Lin, Zhiyi & Song, Chunyue & Zhao, Jun & Yin, Huan, 2022. "Improved approximate dynamic programming for real-time economic dispatch of integrated microgrids," Energy, Elsevier, vol. 255(C).
    3. Wen, Kerui & Li, Weidong & Yu, Samson Shenglong & Li, Ping & Shi, Peng, 2022. "Optimal intra-day operations of behind-the-meter battery storage for primary frequency regulation provision: A hybrid lookahead method," Energy, Elsevier, vol. 247(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhou, Shaorui & Zhang, Hui & Shi, Ning & Xu, Zhou & Wang, Fan, 2020. "A new convergent hybrid learning algorithm for two-stage stochastic programs," European Journal of Operational Research, Elsevier, vol. 283(1), pages 33-46.
    2. Powell, Warren B., 2019. "A unified framework for stochastic optimization," European Journal of Operational Research, Elsevier, vol. 275(3), pages 795-821.
    3. Luckny Zéphyr & C. Lindsay Anderson, 2018. "Stochastic dynamic programming approach to managing power system uncertainty with distributed storage," Computational Management Science, Springer, vol. 15(1), pages 87-110, January.
    4. Daniel R. Jiang & Warren B. Powell, 2018. "Risk-Averse Approximate Dynamic Programming with Quantile-Based Risk Measures," Mathematics of Operations Research, INFORMS, vol. 43(2), pages 554-579, May.
    5. Michelle Bandarra & Vincent Guigues, 2021. "Single cut and multicut stochastic dual dynamic programming with cut selection for multistage stochastic linear programs: convergence proof and numerical experiments," Computational Management Science, Springer, vol. 18(2), pages 125-148, June.
    6. Daniel R. Jiang & Warren B. Powell, 2015. "Optimal Hour-Ahead Bidding in the Real-Time Electricity Market with Battery Storage Using Approximate Dynamic Programming," INFORMS Journal on Computing, INFORMS, vol. 27(3), pages 525-543, August.
    7. D. Ávila & A. Papavasiliou & N. Löhndorf, 2022. "Parallel and distributed computing for stochastic dual dynamic programming," Computational Management Science, Springer, vol. 19(2), pages 199-226, June.
    8. de Queiroz, Anderson Rodrigo, 2016. "Stochastic hydro-thermal scheduling optimization: An overview," Renewable and Sustainable Energy Reviews, Elsevier, vol. 62(C), pages 382-395.
    9. Saif Benjaafar & Daniel Jiang & Xiang Li & Xiaobo Li, 2022. "Dynamic Inventory Repositioning in On-Demand Rental Networks," Management Science, INFORMS, vol. 68(11), pages 7861-7878, November.
    10. Woonghee Tim Huh & Paat Rusmevichientong, 2009. "A Nonparametric Asymptotic Analysis of Inventory Planning with Censored Demand," Mathematics of Operations Research, INFORMS, vol. 34(1), pages 103-123, February.
    11. Woonghee Tim Huh & Ganesh Janakiraman & John A. Muckstadt & Paat Rusmevichientong, 2009. "An Adaptive Algorithm for Finding the Optimal Base-Stock Policy in Lost Sales Inventory Systems with Censored Demand," Mathematics of Operations Research, INFORMS, vol. 34(2), pages 397-416, May.
    12. Cong Shi & Weidong Chen & Izak Duenyas, 2016. "Technical Note—Nonparametric Data-Driven Algorithms for Multiproduct Inventory Systems with Censored Demand," Operations Research, INFORMS, vol. 64(2), pages 362-370, April.
    13. Boxiao Chen & Xiuli Chao & Cong Shi, 2021. "Nonparametric Learning Algorithms for Joint Pricing and Inventory Control with Lost Sales and Censored Demand," Mathematics of Operations Research, INFORMS, vol. 46(2), pages 726-756, May.
    14. Davi Valladão & Thuener Silva & Marcus Poggi, 2019. "Time-consistent risk-constrained dynamic portfolio optimization with transactional costs and time-dependent returns," Annals of Operations Research, Springer, vol. 282(1), pages 379-405, November.
    15. Rudloff, Birgit & Street, Alexandre & Valladão, Davi M., 2014. "Time consistency and risk averse dynamic decision models: Definition, interpretation and practical consequences," European Journal of Operational Research, Elsevier, vol. 234(3), pages 743-750.
    16. Lin An & Andrew A. Li & Benjamin Moseley & R. Ravi, 2023. "The Nonstationary Newsvendor with (and without) Predictions," Papers 2305.07993, arXiv.org, revised Oct 2023.
    17. Andre Luiz Diniz & Maria Elvira P. Maceira & Cesar Luis V. Vasconcellos & Debora Dias J. Penna, 2020. "A combined SDDP/Benders decomposition approach with a risk-averse surface concept for reservoir operation in long term power generation planning," Annals of Operations Research, Springer, vol. 292(2), pages 649-681, September.
    18. Schur, Rouven & Gönsch, Jochen & Hassler, Michael, 2019. "Time-consistent, risk-averse dynamic pricing," European Journal of Operational Research, Elsevier, vol. 277(2), pages 587-603.
    19. Dias, Bruno Henriques & Tomim, Marcelo Aroca & Marcato, André Luís Marques & Ramos, Tales Pulinho & Brandi, Rafael Bruno S. & Junior, Ivo Chaves da Silva & Filho, João Alberto Passos, 2013. "Parallel computing applied to the stochastic dynamic programming for long term operation planning of hydrothermal power systems," European Journal of Operational Research, Elsevier, vol. 229(1), pages 212-222.
    20. Pan, Zhenning & Yu, Tao & Li, Jie & Qu, Kaiping & Yang, Bo, 2020. "Risk-averse real-time dispatch of integrated electricity and heat system using a modified approximate dynamic programming approach," Energy, Elsevier, vol. 198(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orijoc:v:30:y:2018:i:1:p:106-123. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.