IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v319y2022ics030626192200469x.html
   My bibliography  Save this article

Data-driven online interactive bidding strategy for demand response

Author

Listed:
  • Lee, Kuan-Cheng
  • Yang, Hong-Tzer
  • Tang, Wenjun

Abstract

Demand response (DR), as one of the important energy resources in the future’s grid, provides the services of peak shaving, enhancing the efficiency of renewable energy utilization with a short response period, and low cost. Various categories of DR are established, e.g. automated DR, incentive DR, emergency DR, and demand bidding. However, with the practical issue of the unawareness of residential and commercial consumers’ utility models, the researches about demand bidding aggregator involved in the electricity market are just at the beginning stage. For this issue, the bidding price and bidding quantity are two required decision variables while considering the uncertainties due to the market and participants. In this paper, we determine the bidding and purchasing strategy simultaneously employing the smart meter data and functions. A two-agent deep deterministic policy gradient method is developed to optimize the decisions through learning historical bidding experiences. The online learning further utilizes the daily newest bidding experience attained to ensure trend tracing and self-adaptation. Two environment simulators are adopted for testifying the robustness of the model. The results prove that when facing diverse situations the proposed model can earn the optimal profit via off/online learning the bidding rules and robustly making the proper bid.

Suggested Citation

  • Lee, Kuan-Cheng & Yang, Hong-Tzer & Tang, Wenjun, 2022. "Data-driven online interactive bidding strategy for demand response," Applied Energy, Elsevier, vol. 319(C).
    • Handle: RePEc:eee:appene:v:319:y:2022:i:c:s030626192200469x
    DOI: 10.1016/j.apenergy.2022.119082
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S030626192200469X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2022.119082?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lu, Renzhi & Hong, Seung Ho, 2019. "Incentive-based demand response for smart grid with reinforcement learning and deep neural network," Applied Energy, Elsevier, vol. 236(C), pages 937-949.
    2. Utama, Christian & Troitzsch, Sebastian & Thakur, Jagruti, 2021. "Demand-side flexibility and demand-side bidding for flexible loads in air-conditioned buildings," Applied Energy, Elsevier, vol. 285(C).
    3. Iria, José & Soares, Filipe & Matos, Manuel, 2018. "Optimal supply and demand bidding strategy for an aggregator of small prosumers," Applied Energy, Elsevier, vol. 213(C), pages 658-669.
    4. Li, Gong & Shi, Jing & Qu, Xiuli, 2011. "Modeling methods for GenCo bidding strategy optimization in the liberalized electricity spot market–A state-of-the-art review," Energy, Elsevier, vol. 36(8), pages 4686-4700.
    5. Vázquez-Canteli, José R. & Nagy, Zoltán, 2019. "Reinforcement learning for demand response: A review of algorithms and modeling techniques," Applied Energy, Elsevier, vol. 235(C), pages 1072-1089.
    6. Yan, Xing & Ozturk, Yusuf & Hu, Zechun & Song, Yonghua, 2018. "A review on price-driven residential demand response," Renewable and Sustainable Energy Reviews, Elsevier, vol. 96(C), pages 411-419.
    7. Fazlalipour, Pary & Ehsan, Mehdi & Mohammadi-Ivatloo, Behnam, 2019. "Risk-aware stochastic bidding strategy of renewable micro-grids in day-ahead and real-time markets," Energy, Elsevier, vol. 171(C), pages 689-700.
    8. Korkas, Christos D. & Baldi, Simone & Michailidis, Iakovos & Kosmatopoulos, Elias B., 2016. "Occupancy-based demand response and thermal comfort optimization in microgrids with renewable energy sources and energy storage," Applied Energy, Elsevier, vol. 163(C), pages 93-104.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wu, Shengyang & Ding, Zhaohao & Wang, Jingyu & Shi, Dongyuan, 2023. "Unveiling bidding uncertainties in electricity markets: A Bayesian deep learning framework based on accurate variational inference," Energy, Elsevier, vol. 276(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wen, Lulu & Zhou, Kaile & Li, Jun & Wang, Shanyong, 2020. "Modified deep learning and reinforcement learning for an incentive-based demand response model," Energy, Elsevier, vol. 205(C).
    2. Carlos Cruz & Esther Palomar & Ignacio Bravo & Alfredo Gardel, 2020. "Cooperative Demand Response Framework for a Smart Community Targeting Renewables: Testbed Implementation and Performance Evaluation," Energies, MDPI, vol. 13(11), pages 1-20, June.
    3. Silva, Ana R. & Pousinho, H.M.I. & Estanqueiro, Ana, 2022. "A multistage stochastic approach for the optimal bidding of variable renewable energy in the day-ahead, intraday and balancing markets," Energy, Elsevier, vol. 258(C).
    4. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    5. Correa-Jullian, Camila & López Droguett, Enrique & Cardemil, José Miguel, 2020. "Operation scheduling in a solar thermal system: A reinforcement learning-based framework," Applied Energy, Elsevier, vol. 268(C).
    6. Ibrahim, Muhammad Sohail & Dong, Wei & Yang, Qiang, 2020. "Machine learning driven smart electric power systems: Current trends and new perspectives," Applied Energy, Elsevier, vol. 272(C).
    7. Pinto, Giuseppe & Piscitelli, Marco Savino & Vázquez-Canteli, José Ramón & Nagy, Zoltán & Capozzoli, Alfonso, 2021. "Coordinated energy management for a cluster of buildings through deep reinforcement learning," Energy, Elsevier, vol. 229(C).
    8. Pallonetto, Fabiano & De Rosa, Mattia & Milano, Federico & Finn, Donal P., 2019. "Demand response algorithms for smart-grid ready residential buildings using machine learning models," Applied Energy, Elsevier, vol. 239(C), pages 1265-1282.
    9. Xu, Fangyuan & Zhu, Weidong & Wang, Yi Fei & Lai, Chun Sing & Yuan, Haoliang & Zhao, Yujia & Guo, Siming & Fu, Zhengxin, 2022. "A new deregulated demand response scheme for load over-shifting city in regulated power market," Applied Energy, Elsevier, vol. 311(C).
    10. Debin Fang & Qiyu Ren & Qian Yu, 2018. "How Elastic Demand Affects Bidding Strategy in Electricity Market: An Auction Approach," Energies, MDPI, vol. 12(1), pages 1-13, December.
    11. Pinto, Giuseppe & Deltetto, Davide & Capozzoli, Alfonso, 2021. "Data-driven district energy management with surrogate models and deep reinforcement learning," Applied Energy, Elsevier, vol. 304(C).
    12. Seongwoo Lee & Joonho Seon & Byungsun Hwang & Soohyun Kim & Youngghyu Sun & Jinyoung Kim, 2024. "Recent Trends and Issues of Energy Management Systems Using Machine Learning," Energies, MDPI, vol. 17(3), pages 1-24, January.
    13. Park, Keonwoo & Moon, Ilkyeong, 2022. "Multi-agent deep reinforcement learning approach for EV charging scheduling in a smart grid," Applied Energy, Elsevier, vol. 328(C).
    14. Máximo A. Domínguez-Garabitos & Víctor S. Ocaña-Guevara & Félix Santos-García & Adriana Arango-Manrique & Miguel Aybar-Mejía, 2022. "A Methodological Proposal for Implementing Demand-Shifting Strategies in the Wholesale Electricity Market," Energies, MDPI, vol. 15(4), pages 1-28, February.
    15. Golmohamadi, Hessam, 2022. "Demand-side management in industrial sector: A review of heavy industries," Renewable and Sustainable Energy Reviews, Elsevier, vol. 156(C).
    16. Antonopoulos, Ioannis & Robu, Valentin & Couraud, Benoit & Kirli, Desen & Norbu, Sonam & Kiprakis, Aristides & Flynn, David & Elizondo-Gonzalez, Sergio & Wattam, Steve, 2020. "Artificial intelligence and machine learning approaches to energy demand-side response: A systematic review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 130(C).
    17. Charbonnier, Flora & Morstyn, Thomas & McCulloch, Malcolm D., 2022. "Coordination of resources at the edge of the electricity grid: Systematic review and taxonomy," Applied Energy, Elsevier, vol. 318(C).
    18. Xu, Jiuping & Liu, Tingting, 2020. "Technological paradigm-based approaches towards challenges and policy shifts for sustainable wind energy development," Energy Policy, Elsevier, vol. 142(C).
    19. Davide Deltetto & Davide Coraci & Giuseppe Pinto & Marco Savino Piscitelli & Alfonso Capozzoli, 2021. "Exploring the Potentialities of Deep Reinforcement Learning for Incentive-Based Demand Response in a Cluster of Small Commercial Buildings," Energies, MDPI, vol. 14(10), pages 1-25, May.
    20. Yi Kuang & Xiuli Wang & Hongyang Zhao & Yijun Huang & Xianlong Chen & Xifan Wang, 2020. "Agent-Based Energy Sharing Mechanism Using Deep Deterministic Policy Gradient Algorithm," Energies, MDPI, vol. 13(19), pages 1-20, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:319:y:2022:i:c:s030626192200469x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.