IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v256y2022ics0360544222015602.html
   My bibliography  Save this article

Strategic bidding in a competitive electricity market: An intelligent method using Multi-Agent Transfer Learning based on reinforcement learning

Author

Listed:
  • Wu, Jiahui
  • Wang, Jidong
  • Kong, Xiangyu

Abstract

The electricity market will tend to be diverse and competitive to realize Carbon Neutrality goals under Energy Internet. Moreover, bidding strategies and methods are essential for the stable and benign operation of the electricity market. With the development of artificial intelligence and computer simulation technology, multi-agent simulation has gradually become a significant method for electricity market bidding. Among them, Multi-Agent Reinforcement Learning (MARL) can help agents adapt to changing environments. In contrast, Multi-Agent Transfer Learning (MATL) can help agents learn from not only the target task but also other similar tasks. This paper proposes an intelligent strategic bidding theoretical framework in a competitive electricity market using MATL based on MARL and studies four MATL algorithms, including RNN, LSTM, GRU and BGRU. An intelligent bidding simulation model based on the four MATL algorithms is established, and the performance of the intelligent bidding simulation model in the electricity market using the four MATL algorithms based on the MARL Q-learning algorithm is compared and analyzed from the perspective of accuracy and convergence speed. And based on the multi-agent simulation model, examples of bidding strategies are carried out to verify the rationality and effectiveness of the intelligent bidding method using MATL based on MARL.

Suggested Citation

  • Wu, Jiahui & Wang, Jidong & Kong, Xiangyu, 2022. "Strategic bidding in a competitive electricity market: An intelligent method using Multi-Agent Transfer Learning based on reinforcement learning," Energy, Elsevier, vol. 256(C).
    • Handle: RePEc:eee:energy:v:256:y:2022:i:c:s0360544222015602
    DOI: 10.1016/j.energy.2022.124657
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544222015602
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2022.124657?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zang, Haixiang & Xu, Ruiqi & Cheng, Lilin & Ding, Tao & Liu, Ling & Wei, Zhinong & Sun, Guoqiang, 2021. "Residential load forecasting based on LSTM fusing self-attention mechanism with pooling," Energy, Elsevier, vol. 229(C).
    2. Wang, Jidong & Wu, Jiahui & Che, Yanbo, 2019. "Agent and system dynamics-based hybrid modeling and simulation for multilateral bidding in electricity market," Energy, Elsevier, vol. 180(C), pages 444-456.
    3. Guo, Hongye & Chen, Qixin & Shahidehpour, Mohammad & Xia, Qing & Kang, Chongqing, 2022. "Bidding behaviors of GENCOs under bounded rationality with renewable energy," Energy, Elsevier, vol. 250(C).
    4. Chen, Peipei & Wu, Yi & Zou, Lele, 2019. "Distributive PV trading market in China: A design of multi-agent-based model and its forecast analysis," Energy, Elsevier, vol. 185(C), pages 423-436.
    5. Saeian, Hosein & Niknam, Taher & Zare, Mohsen & Aghaei, Jamshid, 2022. "Coordinated optimal bidding strategies methods of aggregated microgrids: A game theory-based demand side management under an electricity market environment," Energy, Elsevier, vol. 245(C).
    6. Imani, Maryam, 2021. "Electrical load-temperature CNN for residential load forecasting," Energy, Elsevier, vol. 227(C).
    7. Lu, Yakai & Tian, Zhe & Zhou, Ruoyu & Liu, Wenjing, 2021. "A general transfer learning-based framework for thermal load prediction in regional energy system," Energy, Elsevier, vol. 217(C).
    8. Ostadi, Bakhtiar & Motamedi Sedeh, Omid & Husseinzadeh Kashan, Ali, 2020. "Risk-based optimal bidding patterns in the deregulated power market using extended Markowitz model," Energy, Elsevier, vol. 191(C).
    9. Das, Saborni & Basu, Mousumi, 2020. "Day-ahead optimal bidding strategy of microgrid with demand response program considering uncertainties and outages of renewable energy resources," Energy, Elsevier, vol. 190(C).
    10. Xian, Huafeng & Che, Jinxing, 2022. "Multi-space collaboration framework based optimal model selection for power load forecasting," Applied Energy, Elsevier, vol. 314(C).
    11. Shafiekhani, Morteza & Ahmadi, Abdollah & Homaee, Omid & Shafie-khah, Miadreza & Catalão, João P.S., 2022. "Optimal bidding strategy of a renewable-based virtual power plant including wind and solar units and dispatchable loads," Energy, Elsevier, vol. 239(PD).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Xian Huang & Kun Liu, 2023. "Impact of Electricity Price Expectation in the Planning Period on the Evolution of Generation Expansion Planning in the Market Environment," Energies, MDPI, vol. 16(8), pages 1-21, April.
    2. Wu, Shengyang & Ding, Zhaohao & Wang, Jingyu & Shi, Dongyuan, 2023. "Unveiling bidding uncertainties in electricity markets: A Bayesian deep learning framework based on accurate variational inference," Energy, Elsevier, vol. 276(C).
    3. Helder Pereira & Bruno Ribeiro & Luis Gomes & Zita Vale, 2022. "Smart Grid Ecosystem Modeling Using a Novel Framework for Heterogenous Agent Communities," Sustainability, MDPI, vol. 14(23), pages 1-20, November.
    4. Alrobaian, Abdulrahman A. & Alsagri, Ali Sulaiman, 2023. "Multi-agent-based energy management for a fully electrified residential consumption," Energy, Elsevier, vol. 282(C).
    5. Marcelle Caroline Thimotheo de Brito & Amaro O. Pereira Junior & Mario Veiga Ferraz Pereira & Julio César Cahuano Simba & Sergio Granville, 2022. "Competitive Behavior of Hydroelectric Power Plants under Uncertainty in Spot Market," Energies, MDPI, vol. 15(19), pages 1-22, October.
    6. Mudhafar Al-Saadi & Maher Al-Greer & Michael Short, 2023. "Reinforcement Learning-Based Intelligent Control Strategies for Optimal Power Management in Advanced Power Distribution Systems: A Survey," Energies, MDPI, vol. 16(4), pages 1-38, February.
    7. Li, Qirui & Yang, Zhifang & Yu, Juan & Li, Wenyuan, 2023. "Impacts of previous revenues on bidding strategies in electricity market: A quantitative analysis," Applied Energy, Elsevier, vol. 345(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wu, Shengyang & Ding, Zhaohao & Wang, Jingyu & Shi, Dongyuan, 2023. "Unveiling bidding uncertainties in electricity markets: A Bayesian deep learning framework based on accurate variational inference," Energy, Elsevier, vol. 276(C).
    2. Ling, Jihong & Zhang, Bingyang & Dai, Na & Xing, Jincheng, 2023. "Coupling input feature construction methods and machine learning algorithms for hourly secondary supply temperature prediction," Energy, Elsevier, vol. 278(C).
    3. Lu, Shixiang & Xu, Qifa & Jiang, Cuixia & Liu, Yezheng & Kusiak, Andrew, 2022. "Probabilistic load forecasting with a non-crossing sparse-group Lasso-quantile regression deep neural network," Energy, Elsevier, vol. 242(C).
    4. Zhang, Xinyue & Guo, Xiaopeng & Zhang, Xingping, 2023. "Bidding modes for renewable energy considering electricity-carbon integrated market mechanism based on multi-agent hybrid game," Energy, Elsevier, vol. 263(PA).
    5. Guo, Hongye & Chen, Qixin & Shahidehpour, Mohammad & Xia, Qing & Kang, Chongqing, 2022. "Bidding behaviors of GENCOs under bounded rationality with renewable energy," Energy, Elsevier, vol. 250(C).
    6. Mostafa Darvishi & Mehrdad Tahmasebi & Ehsan Shokouhmand & Jagadeesh Pasupuleti & Pitshou Bokoro & Jwan Satei Raafat, 2023. "Optimal Operation of Sustainable Virtual Power Plant Considering the Amount of Emission in the Presence of Renewable Energy Sources and Demand Response," Sustainability, MDPI, vol. 15(14), pages 1-25, July.
    7. Fazlipour, Zahra & Mashhour, Elaheh & Joorabian, Mahmood, 2022. "A deep model for short-term load forecasting applying a stacked autoencoder based on LSTM supported by a multi-stage attention mechanism," Applied Energy, Elsevier, vol. 327(C).
    8. Mingping Liu & Xihao Sun & Qingnian Wang & Suhui Deng, 2022. "Short-Term Load Forecasting Using EMD with Feature Selection and TCN-Based Deep Learning Model," Energies, MDPI, vol. 15(19), pages 1-22, September.
    9. Johannes Dahlke & Kristina Bogner & Matthias Mueller & Thomas Berger & Andreas Pyka & Bernd Ebersberger, 2020. "Is the Juice Worth the Squeeze? Machine Learning (ML) In and For Agent-Based Modelling (ABM)," Papers 2003.11985, arXiv.org.
    10. Kandpal, Bakul & Pareek, Parikshit & Verma, Ashu, 2022. "A robust day-ahead scheduling strategy for EV charging stations in unbalanced distribution grid," Energy, Elsevier, vol. 249(C).
    11. Zhou, Yuekuan & Lund, Peter D., 2023. "Peer-to-peer energy sharing and trading of renewable energy in smart communities ─ trading pricing models, decision-making and agent-based collaboration," Renewable Energy, Elsevier, vol. 207(C), pages 177-193.
    12. Bujin Shi & Xinbo Zhou & Peilin Li & Wenyu Ma & Nan Pan, 2023. "An IHPO-WNN-Based Federated Learning System for Area-Wide Power Load Forecasting Considering Data Security Protection," Energies, MDPI, vol. 16(19), pages 1-20, October.
    13. Wang, Yubin & Dong, Wei & Yang, Qiang, 2022. "Multi-stage optimal energy management of multi-energy microgrid in deregulated electricity markets," Applied Energy, Elsevier, vol. 310(C).
    14. Zhang, Yang & Yang, Qingyu & Li, Donghe & An, Dou, 2022. "A reinforcement and imitation learning method for pricing strategy of electricity retailer with customers’ flexibility," Applied Energy, Elsevier, vol. 323(C).
    15. Alfredo Candela Esclapez & Miguel López García & Sergio Valero Verdú & Carolina Senabre Blanes, 2022. "Automatic Selection of Temperature Variables for Short-Term Load Forecasting," Sustainability, MDPI, vol. 14(20), pages 1-22, October.
    16. Ding, Kun & Chen, Xiang & Weng, Shuai & Liu, Yongjie & Zhang, Jingwei & Li, Yuanliang & Yang, Zenan, 2023. "Health status evaluation of photovoltaic array based on deep belief network and Hausdorff distance," Energy, Elsevier, vol. 262(PB).
    17. Lau, Jat-Syu & Jiang, Yihuo & Li, Ziyuan & Qian, Qian, 2023. "Stochastic trading of storage systems in short term electricity markets considering intraday demand response market," Energy, Elsevier, vol. 280(C).
    18. Jun Dong & Yuanyuan Wang & Xihao Dou & Zhengpeng Chen & Yaoyu Zhang & Yao Liu, 2021. "Research on Decision Optimization Model of Microgrid Participating in Spot Market Transaction," Sustainability, MDPI, vol. 13(12), pages 1-26, June.
    19. Rezaei, Navid & Pezhmani, Yasin & Khazali, Amirhossein, 2022. "Economic-environmental risk-averse optimal heat and power energy management of a grid-connected multi microgrid system considering demand response and bidding strategy," Energy, Elsevier, vol. 240(C).
    20. Songkai Wang & Rong Jia & Xiaoyu Shi & Chang Luo & Yuan An & Qiang Huang & Pengcheng Guo & Xueyan Wang & Xuewen Lei, 2022. "Research on Capacity Allocation Optimization of Commercial Virtual Power Plant (CVPP)," Energies, MDPI, vol. 15(4), pages 1-18, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:256:y:2022:i:c:s0360544222015602. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.