IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v153y2018icp977-987.html

Smart generation control based on multi-agent reinforcement learning with the idea of the time tunnel

Author

Listed:
  • Xi, Lei
  • Chen, Jianfeng
  • Huang, Yuehua
  • Xu, Yanchun
  • Liu, Lang
  • Zhou, Yimin
  • Li, Yudan

Abstract

One of the significant solutions for hazy is to reduce carbon emission by introducing renewable energy on a large scale. However, the large-scale integration of new energy will result in stochastic disturbance to power grid. Therefore it becomes a top priority to make new energy compatible with power system. The PDWoLF-PHC(λ) based on the idea of time tunnel is to be proposed in this paper. Optimal strategy could be obtained by adopting the variable learning rate in a variety of complex operating environments, and thence it can deal with stochastic disturbance caused by massive integrations of new energy and distributed energy sources to the power grid, which is difficult for traditional centralized AGC. The proposed algorithm is simulated to be effective according to the improved IEEE standard two-area load-frequency control power system model and the Central China Power Grid model. Compared with the traditional smart ones, the proposed algorithm is characterized with faster convergence and stronger robustness, which makes it able to reduce carbon emission and enhance utilization rate of the new energy.

Suggested Citation

  • Xi, Lei & Chen, Jianfeng & Huang, Yuehua & Xu, Yanchun & Liu, Lang & Zhou, Yimin & Li, Yudan, 2018. "Smart generation control based on multi-agent reinforcement learning with the idea of the time tunnel," Energy, Elsevier, vol. 153(C), pages 977-987.
    • Handle: RePEc:eee:energy:v:153:y:2018:i:c:p:977-987
    DOI: 10.1016/j.energy.2018.04.042
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544218306479
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2018.04.042?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Hammond, G.P. & Norman, J.B., 2012. "Decomposition analysis of energy-related carbon emissions from UK manufacturing," Energy, Elsevier, vol. 41(1), pages 220-227.
    2. Vives, Xavier, 1990. "Nash equilibrium with strategic complementarities," Journal of Mathematical Economics, Elsevier, vol. 19(3), pages 305-321.
    3. Soares M.C. Borba, Bruno & Szklo, Alexandre & Schaeffer, Roberto, 2012. "Plug-in hybrid electric vehicles as a way to maximize the integration of variable renewable energy in power systems: The case of wind generation in northeastern Brazil," Energy, Elsevier, vol. 37(1), pages 469-481.
    4. Park, Jinkyoo & Law, Kincho H., 2016. "A data-driven, cooperative wind farm control to maximize the total power production," Applied Energy, Elsevier, vol. 165(C), pages 151-165.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shi, Zhongtuo & Yao, Wei & Li, Zhouping & Zeng, Lingkang & Zhao, Yifan & Zhang, Runfeng & Tang, Yong & Wen, Jinyu, 2020. "Artificial intelligence techniques for stability analysis and control in smart grids: Methodologies, applications, challenges and future directions," Applied Energy, Elsevier, vol. 278(C).
    2. Yin, Linfei & Zhang, Bin, 2021. "Time series generative adversarial network controller for long-term smart generation control of microgrids," Applied Energy, Elsevier, vol. 281(C).
    3. Xi, Lei & Shi, Yu & Quan, Yue & Liu, Zhihong, 2024. "Research on the multi-area cooperative control method for novel power systems," Energy, Elsevier, vol. 313(C).
    4. Zhao, Zeyu & Yang, Xiyun & Chen, Han & Chen, He, 2025. "An improved heuristic dynamic programming framework for multi-source frequency regulation," Applied Energy, Elsevier, vol. 387(C).
    5. Huaizhi Wang & Xian Zhang & Qing Li & Guibin Wang & Hui Jiang & Jianchun Peng, 2018. "Recursive Method for Distribution System Reliability Evaluation," Energies, MDPI, vol. 11(10), pages 1-15, October.
    6. Ochoa, Tomás & Gil, Esteban & Angulo, Alejandro & Valle, Carlos, 2022. "Multi-agent deep reinforcement learning for efficient multi-timescale bidding of a hybrid power plant in day-ahead and real-time markets," Applied Energy, Elsevier, vol. 317(C).
    7. Paul Arévalo & Francisco Jurado, 2024. "Impact of Artificial Intelligence on the Planning and Operation of Distributed Energy Systems in Smart Grids," Energies, MDPI, vol. 17(17), pages 1-22, September.
    8. Han, Kunlun & Yang, Kai & Yin, Linfei, 2022. "Lightweight actor-critic generative adversarial networks for real-time smart generation control of microgrids," Applied Energy, Elsevier, vol. 317(C).
    9. Yin, Linfei & Gao, Qi & Zhao, Lulin & Wang, Tao, 2020. "Expandable deep learning for real-time economic generation dispatch and control of three-state energies based future smart grids," Energy, Elsevier, vol. 191(C).
    10. Ju, Liwei & Zhang, Qi & Tan, Zhongfu & Wang, Wei & Xin, He & Zhang, Zehao, 2018. "Multi-agent-system-based coupling control optimization model for micro-grid group intelligent scheduling considering autonomy-cooperative operation strategy," Energy, Elsevier, vol. 157(C), pages 1035-1052.
    11. Li, Jiawen & Yu, Tao & Zhang, Xiaoshun & Li, Fusheng & Lin, Dan & Zhu, Hanxin, 2021. "Efficient experience replay based deep deterministic policy gradient for AGC dispatch in integrated energy system," Applied Energy, Elsevier, vol. 285(C).
    12. Yin, Linfei & Luo, Shikui & Ma, Chenxiao, 2021. "Expandable depth and width adaptive dynamic programming for economic smart generation control of smart grids," Energy, Elsevier, vol. 232(C).
    13. Yin, Linfei & Li, Yu, 2022. "Hybrid multi-agent emotional deep Q network for generation control of multi-area integrated energy systems," Applied Energy, Elsevier, vol. 324(C).
    14. Ziyang Yu & Bowen Zhou & Dongsheng Yang & Weirong Wu & Chen Lv & Yong Cui, 2023. "Power System Operation Mode Calculation Based on Improved Deep Reinforcement Learning," Mathematics, MDPI, vol. 12(1), pages 1-14, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pietro Tebaldi, 2015. "Estimating Equilibrium in Health Insurance Exchanges: Analysis of the Californian Market under the ACA," Discussion Papers 15-012, Stanford Institute for Economic Policy Research.
    2. López, Alberto, 2012. "Productivity effects of ICTs and organizational change: A test of the complementarity hypothesis in Spain," MPRA Paper 40400, University Library of Munich, Germany.
    3. Boyarchenko, Svetlana & Machowska, Dominika & Topolyan, Iryna, 2025. "Supermodularity and incentive reversal in teams," Games and Economic Behavior, Elsevier, vol. 150(C), pages 93-105.
    4. Zhang, Qi & Mclellan, Benjamin C. & Tezuka, Tetsuo & Ishihara, Keiichi N., 2013. "A methodology for economic and environmental analysis of electric vehicles with different operational conditions," Energy, Elsevier, vol. 61(C), pages 118-127.
    5. de Frutos, María-Ángeles & Fabra, Natalia, 2011. "Endogenous capacities and price competition: The role of demand uncertainty," International Journal of Industrial Organization, Elsevier, vol. 29(4), pages 399-411, July.
    6. Christian Ewerhart, 2020. "Ordinal potentials in smooth games," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 70(4), pages 1069-1100, November.
    7. Echenique, Federico, 2004. "A characterization of strategic complementarities," Games and Economic Behavior, Elsevier, vol. 46(2), pages 325-347, February.
    8. Tanvir Ahmad & Abdul Basit & Muneeb Ahsan & Olivier Coupiac & Nicolas Girard & Behzad Kazemtabrizi & Peter C. Matthews, 2019. "Implementation and Analyses of Yaw Based Coordinated Control of Wind Farms," Energies, MDPI, vol. 12(7), pages 1-15, April.
    9. Jacobovic, Royi & Levy, Yehuda John & Solan, Eilon, 0. "Bayesian games with nested information," Theoretical Economics, Econometric Society.
    10. Jun Honda, 2015. "Games with the Total Bandwagon Property," Department of Economics Working Papers wuwp197, Vienna University of Economics and Business, Department of Economics.
    11. Bergstrom, Theodore C & Stark, Oded, 1993. "How Altruism Can Prevail in an Evolutionary Environment," American Economic Review, American Economic Association, vol. 83(2), pages 149-155, May.
    12. , & , & ,, 2008. "Monotone methods for equilibrium selection under perfect foresight dynamics," Theoretical Economics, Econometric Society, vol. 3(2), June.
    13. Fichter, Tobias & Soria, Rafael & Szklo, Alexandre & Schaeffer, Roberto & Lucena, Andre F.P., 2017. "Assessing the potential role of concentrated solar power (CSP) for the northeast power system of Brazil using a detailed power system model," Energy, Elsevier, vol. 121(C), pages 695-715.
    14. Amir, Rabah & Bloch, Francis, 2009. "Comparative statics in a simple class of strategic market games," Games and Economic Behavior, Elsevier, vol. 65(1), pages 7-24, January.
    15. Kaivo-oja, J. & Luukkanen, J. & Panula-Ontto, J. & Vehmas, J. & Chen, Y. & Mikkonen, S. & Auffermann, B., 2014. "Are structural change and modernisation leading to convergence in the CO2 economy? Decomposition analysis of China, EU and USA," Energy, Elsevier, vol. 72(C), pages 115-125.
    16. Mitsukuni Nishida, 2015. "Estimating a Model of Strategic Network Choice: The Convenience-Store Industry in Okinawa," Marketing Science, INFORMS, vol. 34(1), pages 20-38, January.
    17. Michael T. Rauh & Giulio Seccia, 2010. "Agency and Anxiety," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 19(1), pages 87-116, March.
      • Michael T. Rauh & Giulio Seccia, 2006. "Agency and Anxiety," Working Papers 2006-02, Indiana University, Kelley School of Business, Department of Business Economics and Public Policy.
    18. Rabah Amir & Giuseppe Feo, 2014. "Endogenous timing in a mixed duopoly," International Journal of Game Theory, Springer;Game Theory Society, vol. 43(3), pages 629-658, August.
    19. Jullien, Bruno, 2001. "Competing with Network Externalities and Price Discrimination," CEPR Discussion Papers 2883, C.E.P.R. Discussion Papers.
    20. Chen, Yu & Zhao, Changyi & Chen, Shan & Chen, Wenqing & Wan, Kunyang & Wei, Jia, 2023. "Riding the green rails: Exploring the nexus between high-speed trains, green innovation, and carbon emissions," Energy, Elsevier, vol. 282(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:153:y:2018:i:c:p:977-987. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.