IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v315y2022ics0306261922002896.html
   My bibliography  Save this article

DSM pricing method based on A3C and LSTM under cloud-edge environment

Author

Listed:
  • Sun, Fangyuan
  • Kong, Xiangyu
  • Wu, Jianzhong
  • Gao, Bixuan
  • Chen, Ke
  • Lu, Ning

Abstract

Demand-side management (DSM) could realize “peak cutting and valley filling” of power load and improve the stability and efficiency of power system. With the development of information systems, more smart devices are being deployed on the demand side. It has become a challenge for DSM service providers to take full use of the edge computing capacity and demand-side information to improve the accuracy of DSM decision-making, without revealing user privacy. This paper proposes a distributed DSM pricing method for service provider, based on asynchronous advantage actor-critic (A3C) algorithm and long short-term memory (LSTM) network under cloud-edge environment. The on-site utilization of user information is realized through distributed training and centralized decision-making structure of A3C algorithm. The training process is accelerated by LSTM based virtual environment, which greatly reduces the training cost of the algorithm. Case study results shows that the proposed method is able to make pricing decision for DSM service provider under cloud-edge environment. Moreover, through the combination of LSTM based virtual environment and A3C algorithm, the proposed method requires less historical data than other methods and improves the profit of service providers.

Suggested Citation

  • Sun, Fangyuan & Kong, Xiangyu & Wu, Jianzhong & Gao, Bixuan & Chen, Ke & Lu, Ning, 2022. "DSM pricing method based on A3C and LSTM under cloud-edge environment," Applied Energy, Elsevier, vol. 315(C).
    • Handle: RePEc:eee:appene:v:315:y:2022:i:c:s0306261922002896
    DOI: 10.1016/j.apenergy.2022.118853
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261922002896
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2022.118853?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Xi-liang Chen & Lei Cao & Chen-xi Li & Zhi-xiong Xu & Jun Lai, 2018. "Ensemble Network Architecture for Deep Reinforcement Learning," Mathematical Problems in Engineering, Hindawi, vol. 2018, pages 1-6, April.
    2. Fernández-Blanco, Ricardo & Morales, Juan Miguel & Pineda, Salvador, 2021. "Forecasting the price-response of a pool of buildings via homothetic inverse optimization," Applied Energy, Elsevier, vol. 290(C).
    3. Wang, Zhe & Hong, Tianzhen & Piette, Mary Ann, 2020. "Building thermal load prediction through shallow machine learning and deep learning," Applied Energy, Elsevier, vol. 263(C).
    4. Wang, Zhe & Hong, Tianzhen, 2020. "Reinforcement learning for building controls: The opportunities and challenges," Applied Energy, Elsevier, vol. 269(C).
    5. Zhipeng Liang & Hao Chen & Junhao Zhu & Kangkang Jiang & Yanran Li, 2018. "Adversarial Deep Reinforcement Learning in Portfolio Management," Papers 1808.09940, arXiv.org, revised Nov 2018.
    6. Bhatti, Bilal Ahmad & Broadwater, Robert, 2019. "Energy trading in the distribution system using a non-model based game theoretic approach," Applied Energy, Elsevier, vol. 253(C), pages 1-1.
    7. Lu, Renzhi & Bai, Ruichang & Huang, Yuan & Li, Yuting & Jiang, Junhui & Ding, Yuemin, 2021. "Data-driven real-time price-based demand response for industrial facilities energy management," Applied Energy, Elsevier, vol. 283(C).
    8. Wang, Yi & Gan, Dahua & Sun, Mingyang & Zhang, Ning & Lu, Zongxiang & Kang, Chongqing, 2019. "Probabilistic individual load forecasting using pinball loss guided LSTM," Applied Energy, Elsevier, vol. 235(C), pages 10-20.
    9. Lu, Renzhi & Hong, Seung Ho & Zhang, Xiongfeng, 2018. "A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach," Applied Energy, Elsevier, vol. 220(C), pages 220-230.
    10. Lork, Clement & Li, Wen-Tai & Qin, Yan & Zhou, Yuren & Yuen, Chau & Tushar, Wayes & Saha, Tapan K., 2020. "An uncertainty-aware deep reinforcement learning framework for residential air conditioning energy management," Applied Energy, Elsevier, vol. 276(C).
    11. Ruan, Guangchun & Zhong, Haiwang & Wang, Jianxiao & Xia, Qing & Kang, Chongqing, 2020. "Neural-network-based Lagrange multiplier selection for distributed demand response in smart grid," Applied Energy, Elsevier, vol. 264(C).
    12. Vázquez-Canteli, José R. & Nagy, Zoltán, 2019. "Reinforcement learning for demand response: A review of algorithms and modeling techniques," Applied Energy, Elsevier, vol. 235(C), pages 1072-1089.
    13. Hafeez, Ghulam & Alimgeer, Khurram Saleem & Khan, Imran, 2020. "Electric load forecasting based on deep learning and optimized by heuristic algorithm in smart grid," Applied Energy, Elsevier, vol. 269(C).
    14. Kong, Xiangyu & Sun, Fangyuan & Huo, Xianxu & Li, Xue & Shen, Yu, 2020. "Hierarchical optimal scheduling method of heat-electricity integrated energy system based on Power Internet of Things," Energy, Elsevier, vol. 210(C).
    15. Kong, Xiangyu & Kong, Deqian & Yao, Jingtao & Bai, Linquan & Xiao, Jie, 2020. "Online pricing of demand response based on long short-term memory and reinforcement learning," Applied Energy, Elsevier, vol. 271(C).
    16. Yang, Ting & Zhao, Yingjie & Pen, Haibo & Wang, Zhaoxia, 2018. "Data center holistic demand response algorithm to smooth microgrid tie-line power fluctuation," Applied Energy, Elsevier, vol. 231(C), pages 277-287.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mudhafar Al-Saadi & Maher Al-Greer & Michael Short, 2023. "Reinforcement Learning-Based Intelligent Control Strategies for Optimal Power Management in Advanced Power Distribution Systems: A Survey," Energies, MDPI, vol. 16(4), pages 1-38, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    2. Xie, Jiahan & Ajagekar, Akshay & You, Fengqi, 2023. "Multi-Agent attention-based deep reinforcement learning for demand response in grid-responsive buildings," Applied Energy, Elsevier, vol. 342(C).
    3. Zhang, Xiongfeng & Lu, Renzhi & Jiang, Junhui & Hong, Seung Ho & Song, Won Seok, 2021. "Testbed implementation of reinforcement learning-based demand response energy management system," Applied Energy, Elsevier, vol. 297(C).
    4. Saima Akhtar & Sulman Shahzad & Asad Zaheer & Hafiz Sami Ullah & Heybet Kilic & Radomir Gono & Michał Jasiński & Zbigniew Leonowicz, 2023. "Short-Term Load Forecasting Models: A Review of Challenges, Progress, and the Road Ahead," Energies, MDPI, vol. 16(10), pages 1-29, May.
    5. Wen, Lulu & Zhou, Kaile & Li, Jun & Wang, Shanyong, 2020. "Modified deep learning and reinforcement learning for an incentive-based demand response model," Energy, Elsevier, vol. 205(C).
    6. Qiu, Dawei & Dong, Zihang & Zhang, Xi & Wang, Yi & Strbac, Goran, 2022. "Safe reinforcement learning for real-time automatic control in a smart energy-hub," Applied Energy, Elsevier, vol. 309(C).
    7. Zeng, Huibin & Shao, Bilin & Dai, Hongbin & Tian, Ning & Zhao, Wei, 2023. "Incentive-based demand response strategies for natural gas considering carbon emissions and load volatility," Applied Energy, Elsevier, vol. 348(C).
    8. Eduardo J. Salazar & Mauro Jurado & Mauricio E. Samper, 2023. "Reinforcement Learning-Based Pricing and Incentive Strategy for Demand Response in Smart Grids," Energies, MDPI, vol. 16(3), pages 1-33, February.
    9. Perera, A.T.D. & Kamalaruban, Parameswaran, 2021. "Applications of reinforcement learning in energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    10. Nik, Vahid M. & Hosseini, Mohammad, 2023. "CIRLEM: a synergic integration of Collective Intelligence and Reinforcement learning in Energy Management for enhanced climate resilience and lightweight computation," Applied Energy, Elsevier, vol. 350(C).
    11. Shafqat Jawad & Junyong Liu, 2020. "Electrical Vehicle Charging Services Planning and Operation with Interdependent Power Networks and Transportation Networks: A Review of the Current Scenario and Future Trends," Energies, MDPI, vol. 13(13), pages 1-24, July.
    12. Gokhale, Gargya & Claessens, Bert & Develder, Chris, 2022. "Physics informed neural networks for control oriented thermal modeling of buildings," Applied Energy, Elsevier, vol. 314(C).
    13. Langer, Lissy & Volling, Thomas, 2022. "A reinforcement learning approach to home energy management for modulating heat pumps and photovoltaic systems," Applied Energy, Elsevier, vol. 327(C).
    14. Correa-Jullian, Camila & López Droguett, Enrique & Cardemil, José Miguel, 2020. "Operation scheduling in a solar thermal system: A reinforcement learning-based framework," Applied Energy, Elsevier, vol. 268(C).
    15. Ibrahim, Muhammad Sohail & Dong, Wei & Yang, Qiang, 2020. "Machine learning driven smart electric power systems: Current trends and new perspectives," Applied Energy, Elsevier, vol. 272(C).
    16. Pinto, Giuseppe & Piscitelli, Marco Savino & Vázquez-Canteli, José Ramón & Nagy, Zoltán & Capozzoli, Alfonso, 2021. "Coordinated energy management for a cluster of buildings through deep reinforcement learning," Energy, Elsevier, vol. 229(C).
    17. Zhang, Li & Gao, Yan & Zhu, Hongbo & Tao, Li, 2022. "Bi-level stochastic real-time pricing model in multi-energy generation system: A reinforcement learning approach," Energy, Elsevier, vol. 239(PA).
    18. Pallonetto, Fabiano & De Rosa, Mattia & Milano, Federico & Finn, Donal P., 2019. "Demand response algorithms for smart-grid ready residential buildings using machine learning models," Applied Energy, Elsevier, vol. 239(C), pages 1265-1282.
    19. Sabarathinam Srinivasan & Suresh Kumarasamy & Zacharias E. Andreadakis & Pedro G. Lind, 2023. "Artificial Intelligence and Mathematical Models of Power Grids Driven by Renewable Energy Sources: A Survey," Energies, MDPI, vol. 16(14), pages 1-56, July.
    20. Xu, Fangyuan & Zhu, Weidong & Wang, Yi Fei & Lai, Chun Sing & Yuan, Haoliang & Zhao, Yujia & Guo, Siming & Fu, Zhengxin, 2022. "A new deregulated demand response scheme for load over-shifting city in regulated power market," Applied Energy, Elsevier, vol. 311(C).

    More about this item

    Keywords

    Demand-side management; LSTM; A3C; Cloud-edge environment;
    All these keywords.

    JEL classification:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:315:y:2022:i:c:s0306261922002896. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.