IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v220y2018icp220-230.html
   My bibliography  Save this article

A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach

Author

Listed:
  • Lu, Renzhi
  • Hong, Seung Ho
  • Zhang, Xiongfeng

Abstract

With the modern advanced information and communication technologies in smart grid systems, demand response (DR) has become an effective method for improving grid reliability and reducing energy costs due to the ability to react quickly to supply-demand mismatches by adjusting flexible loads on the demand side. This paper proposes a dynamic pricing DR algorithm for energy management in a hierarchical electricity market that considers both service provider’s (SP) profit and customers’ (CUs) costs. Reinforcement learning (RL) is used to illustrate the hierarchical decision-making framework, in which the dynamic pricing problem is formulated as a discrete finite Markov decision process (MDP), and Q-learning is adopted to solve this decision-making problem. Using RL, the SP can adaptively decide the retail electricity price during the on-line learning process where the uncertainty of CUs’ load demand profiles and the flexibility of wholesale electricity prices are addressed. Simulation results show that this proposed DR algorithm, can promote SP profitability, reduce energy costs for CUs, balance energy supply and demand in the electricity market, and improve the reliability of electric power systems, which can be regarded as a win-win strategy for both SP and CUs.

Suggested Citation

  • Lu, Renzhi & Hong, Seung Ho & Zhang, Xiongfeng, 2018. "A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach," Applied Energy, Elsevier, vol. 220(C), pages 220-230.
    • Handle: RePEc:eee:appene:v:220:y:2018:i:c:p:220-230
    DOI: 10.1016/j.apenergy.2018.03.072
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261918304112
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2018.03.072?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dagoumas, Athanasios S. & Polemis, Michael L., 2017. "An integrated model for assessing electricity retailer’s profitability with demand response," Applied Energy, Elsevier, vol. 198(C), pages 49-64.
    2. Yu, Mengmeng & Hong, Seung Ho, 2017. "Incentive-based demand response considering hierarchical electricity market: A Stackelberg game approach," Applied Energy, Elsevier, vol. 203(C), pages 267-279.
    3. Nojavan, Sayyad & Zare, Kazem & Mohammadi-Ivatloo, Behnam, 2017. "Optimal stochastic energy management of retailer based on selling price determination under smart grid environment in the presence of demand response program," Applied Energy, Elsevier, vol. 187(C), pages 449-464.
    4. Siano, Pierluigi, 2014. "Demand response and smart grids—A survey," Renewable and Sustainable Energy Reviews, Elsevier, vol. 30(C), pages 461-478.
    5. Rana, Rupal & Oliveira, Fernando S., 2014. "Real-time dynamic pricing in a non-stationary environment using model-free reinforcement learning," Omega, Elsevier, vol. 47(C), pages 116-126.
    6. Li, Xiao Hui & Hong, Seung Ho, 2014. "User-expected price-based demand response algorithm for a home-to-grid system," Energy, Elsevier, vol. 64(C), pages 437-449.
    7. Shen, Bo & Ghatikar, Girish & Lei, Zeng & Li, Jinkai & Wikler, Greg & Martin, Phil, 2014. "The role of regulatory reforms, market changes, and technology development to make demand response a viable resource in meeting energy challenges," Applied Energy, Elsevier, vol. 130(C), pages 814-823.
    8. Jin, Ming & Feng, Wei & Marnay, Chris & Spanos, Costas, 2018. "Microgrid to enable optimal distributed energy retail and end-user demand response," Applied Energy, Elsevier, vol. 210(C), pages 1321-1335.
    9. Massimo, Filippini, 2011. "Short- and long-run time-of-use price elasticities in Swiss residential electricity demand," Energy Policy, Elsevier, vol. 39(10), pages 5811-5817, October.
    10. Zhe Luo & Seung-Ho Hong & Jong-Beom Kim, 2016. "A Price-Based Demand Response Scheme for Discrete Manufacturing in Smart Grids," Energies, MDPI, vol. 9(8), pages 1-18, August.
    11. Behboodi, Sahand & Chassin, David P. & Djilali, Ned & Crawford, Curran, 2018. "Transactive control of fast-acting demand response based on thermostatic loads in real-time retail electricity markets," Applied Energy, Elsevier, vol. 210(C), pages 1310-1320.
    12. Faria, P. & Vale, Z., 2011. "Demand response in electrical energy supply: An optimal real time pricing approach," Energy, Elsevier, vol. 36(8), pages 5374-5384.
    13. Yu, Mengmeng & Hong, Seung Ho, 2016. "Supply–demand balancing for power management in smart grid: A Stackelberg game approach," Applied Energy, Elsevier, vol. 164(C), pages 702-710.
    14. Torriti, Jacopo, 2012. "Price-based demand side management: Assessing the impacts of time-of-use tariffs on residential electricity demand and peak shifting in Northern Italy," Energy, Elsevier, vol. 44(1), pages 576-583.
    15. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    16. Gyamfi, Samuel & Krumdieck, Susan & Urmee, Tania, 2013. "Residential peak electricity demand response—Highlights of some behavioural issues," Renewable and Sustainable Energy Reviews, Elsevier, vol. 25(C), pages 71-77.
    17. Miller, Mark & Alberini, Anna, 2016. "Sensitivity of price elasticity of demand to aggregation, unobserved heterogeneity, price trends, and price endogeneity: Evidence from U.S. Data," Energy Policy, Elsevier, vol. 97(C), pages 235-249.
    18. Yu, Mengmeng & Lu, Renzhi & Hong, Seung Ho, 2016. "A real-time decision model for industrial load management in a smart grid," Applied Energy, Elsevier, vol. 183(C), pages 1488-1497.
    19. Kuznetsova, Elizaveta & Li, Yan-Fu & Ruiz, Carlos & Zio, Enrico & Ault, Graham & Bell, Keith, 2013. "Reinforcement learning for microgrid energy management," Energy, Elsevier, vol. 59(C), pages 133-146.
    20. Nolan, Sheila & O’Malley, Mark, 2015. "Challenges and barriers to demand response deployment and evaluation," Applied Energy, Elsevier, vol. 152(C), pages 1-10.
    21. Gao, Dian-ce & Sun, Yongjun & Lu, Yuehong, 2015. "A robust demand response control of commercial buildings for smart grid under load prediction uncertainty," Energy, Elsevier, vol. 93(P1), pages 275-283.
    22. Katrina Jessoe & David Rapson, 2015. "Commercial and Industrial Demand Response Under Mandatory Time-of-Use Electricity Pricing," Journal of Industrial Economics, Wiley Blackwell, vol. 63(3), pages 397-421, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ahmed Ismail & Mustafa Baysal, 2023. "Dynamic Pricing Based on Demand Response Using Actor–Critic Agent Reinforcement Learning," Energies, MDPI, vol. 16(14), pages 1-19, July.
    2. Lu, Renzhi & Hong, Seung Ho, 2019. "Incentive-based demand response for smart grid with reinforcement learning and deep neural network," Applied Energy, Elsevier, vol. 236(C), pages 937-949.
    3. Haider, Haider Tarish & See, Ong Hang & Elmenreich, Wilfried, 2016. "Residential demand response scheme based on adaptive consumption level pricing," Energy, Elsevier, vol. 113(C), pages 301-308.
    4. Yang, Changhui & Meng, Chen & Zhou, Kaile, 2018. "Residential electricity pricing in China: The context of price-based demand response," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P2), pages 2870-2878.
    5. Meyabadi, A. Fattahi & Deihimi, M.H., 2017. "A review of demand-side management: Reconsidering theoretical framework," Renewable and Sustainable Energy Reviews, Elsevier, vol. 80(C), pages 367-379.
    6. Mohseni, Soheil & Brent, Alan C. & Kelly, Scott & Browne, Will N., 2022. "Demand response-integrated investment and operational planning of renewable and sustainable energy systems considering forecast uncertainties: A systematic review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 158(C).
    7. Haider, Haider Tarish & See, Ong Hang & Elmenreich, Wilfried, 2016. "A review of residential demand response of smart grid," Renewable and Sustainable Energy Reviews, Elsevier, vol. 59(C), pages 166-178.
    8. Märkle-Huß, Joscha & Feuerriegel, Stefan & Neumann, Dirk, 2018. "Large-scale demand response and its implications for spot prices, load and policies: Insights from the German-Austrian electricity market," Applied Energy, Elsevier, vol. 210(C), pages 1290-1298.
    9. Fan, Songli & Ai, Qian & Piao, Longjian, 2018. "Bargaining-based cooperative energy trading for distribution company and demand response," Applied Energy, Elsevier, vol. 226(C), pages 469-482.
    10. Aftab Ahmed Almani & Xueshan Han, 2023. "Real-Time Pricing-Enabled Demand Response Using Long Short-Time Memory Deep Learning," Energies, MDPI, vol. 16(5), pages 1-13, March.
    11. Yan, Xing & Ozturk, Yusuf & Hu, Zechun & Song, Yonghua, 2018. "A review on price-driven residential demand response," Renewable and Sustainable Energy Reviews, Elsevier, vol. 96(C), pages 411-419.
    12. Zhou, Kaile & Peng, Ning & Yin, Hui & Hu, Rong, 2023. "Urban virtual power plant operation optimization with incentive-based demand response," Energy, Elsevier, vol. 282(C).
    13. Zhang, Xiongfeng & Lu, Renzhi & Jiang, Junhui & Hong, Seung Ho & Song, Won Seok, 2021. "Testbed implementation of reinforcement learning-based demand response energy management system," Applied Energy, Elsevier, vol. 297(C).
    14. Xu, Bing & Nayak, Amar & Gray, David & Ouenniche, Jamal, 2016. "Assessing energy business cases implemented in the North Sea Region and strategy recommendations," Applied Energy, Elsevier, vol. 172(C), pages 360-371.
    15. Wen, Lulu & Zhou, Kaile & Li, Jun & Wang, Shanyong, 2020. "Modified deep learning and reinforcement learning for an incentive-based demand response model," Energy, Elsevier, vol. 205(C).
    16. Lu, Renzhi & Bai, Ruichang & Ding, Yuemin & Wei, Min & Jiang, Junhui & Sun, Mingyang & Xiao, Feng & Zhang, Hai-Tao, 2021. "A hybrid deep learning-based online energy management scheme for industrial microgrid," Applied Energy, Elsevier, vol. 304(C).
    17. Paterakis, Nikolaos G. & Erdinç, Ozan & Catalão, João P.S., 2017. "An overview of Demand Response: Key-elements and international experience," Renewable and Sustainable Energy Reviews, Elsevier, vol. 69(C), pages 871-891.
    18. Madia Safdar & Ghulam Amjad Hussain & Matti Lehtonen, 2019. "Costs of Demand Response from Residential Customers’ Perspective," Energies, MDPI, vol. 12(9), pages 1-16, April.
    19. Soares, Ana & Antunes, Carlos Henggeler & Oliveira, Carlos & Gomes, Álvaro, 2014. "A multi-objective genetic approach to domestic load scheduling in an energy management system," Energy, Elsevier, vol. 77(C), pages 144-152.
    20. Konstantakopoulos, Ioannis C. & Barkan, Andrew R. & He, Shiying & Veeravalli, Tanya & Liu, Huihan & Spanos, Costas, 2019. "A deep learning and gamification approach to improving human-building interaction and energy efficiency in smart infrastructure," Applied Energy, Elsevier, vol. 237(C), pages 810-821.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:220:y:2018:i:c:p:220-230. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.