IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i5p1626-d755702.html
   My bibliography  Save this article

Automatic Verification Flow Shop Scheduling of Electric Energy Meters Based on an Improved Q-Learning Algorithm

Author

Listed:
  • Long Peng

    (Meteorology Center of Guangdong Power Grid Co., Ltd., Guangzhou 510600, China)

  • Jiajie Li

    (Meteorology Center of Guangdong Power Grid Co., Ltd., Guangzhou 510600, China)

  • Jingming Zhao

    (Meteorology Center of Guangdong Power Grid Co., Ltd., Guangzhou 510600, China)

  • Sanlei Dang

    (Meteorology Center of Guangdong Power Grid Co., Ltd., Guangzhou 510600, China)

  • Zhengmin Kong

    (School of Electrical Engineering and Automation, Wuhan University, Wuhan 430072, China)

  • Li Ding

    (School of Electrical Engineering and Automation, Wuhan University, Wuhan 430072, China)

Abstract

Considering the engineering problem of electric energy meter automatic verification and scheduling, this paper proposes a novel scheduling scheme based on an improved Q-learning algorithm. First, by introducing the state variables and behavior variables, the ranking problem of combinatorial optimization is transformed into a sequential decision problem. Then, a novel reward function is proposed to evaluate the pros and cons of the different strategies. In particular, this paper considers adopting the reinforcement learning algorithm to efficiently solve the problem. In addition, this paper also considers the ratio of exploration and utilization in the reinforcement learning process, and then provides reasonable exploration and utilization through an iterative updating scheme. Meanwhile, a decoupling strategy is introduced to address the restriction of over estimation. Finally, real time data from a provincial electric energy meter automatic verification center are used to verify the effectiveness of the proposed algorithm.

Suggested Citation

  • Long Peng & Jiajie Li & Jingming Zhao & Sanlei Dang & Zhengmin Kong & Li Ding, 2022. "Automatic Verification Flow Shop Scheduling of Electric Energy Meters Based on an Improved Q-Learning Algorithm," Energies, MDPI, vol. 15(5), pages 1-11, February.
  • Handle: RePEc:gam:jeners:v:15:y:2022:i:5:p:1626-:d:755702
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/5/1626/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1996-1073/15/5/1626/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lu, Renzhi & Hong, Seung Ho & Zhang, Xiongfeng, 2018. "A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach," Applied Energy, Elsevier, vol. 220(C), pages 220-230.
    2. Ruiz, Ruben & Maroto, Concepcion, 2005. "A comprehensive review and evaluation of permutation flowshop heuristics," European Journal of Operational Research, Elsevier, vol. 165(2), pages 479-494, September.
    3. Rossit, Daniel Alejandro & Tohmé, Fernando & Frutos, Mariano, 2018. "The Non-Permutation Flow-Shop scheduling problem: A literature review," Omega, Elsevier, vol. 77(C), pages 143-153.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mehravaran, Yasaman & Logendran, Rasaratnam, 2012. "Non-permutation flowshop scheduling in a supply chain with sequence-dependent setup times," International Journal of Production Economics, Elsevier, vol. 135(2), pages 953-963.
    2. Shafqat Jawad & Junyong Liu, 2020. "Electrical Vehicle Charging Services Planning and Operation with Interdependent Power Networks and Transportation Networks: A Review of the Current Scenario and Future Trends," Energies, MDPI, vol. 13(13), pages 1-24, July.
    3. Ibrahim, Muhammad Sohail & Dong, Wei & Yang, Qiang, 2020. "Machine learning driven smart electric power systems: Current trends and new perspectives," Applied Energy, Elsevier, vol. 272(C).
    4. Perez-Gonzalez, Paz & Framinan, Jose M., 2024. "A review and classification on distributed permutation flowshop scheduling problems," European Journal of Operational Research, Elsevier, vol. 312(1), pages 1-21.
    5. Xu, Fangyuan & Zhu, Weidong & Wang, Yi Fei & Lai, Chun Sing & Yuan, Haoliang & Zhao, Yujia & Guo, Siming & Fu, Zhengxin, 2022. "A new deregulated demand response scheme for load over-shifting city in regulated power market," Applied Energy, Elsevier, vol. 311(C).
    6. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    7. Ray, Manojit & Chakraborty, Basab, 2019. "Impact of evolving technology on collaborative energy access scaling," Renewable and Sustainable Energy Reviews, Elsevier, vol. 110(C), pages 13-27.
    8. Lu, Renzhi & Bai, Ruichang & Ding, Yuemin & Wei, Min & Jiang, Junhui & Sun, Mingyang & Xiao, Feng & Zhang, Hai-Tao, 2021. "A hybrid deep learning-based online energy management scheme for industrial microgrid," Applied Energy, Elsevier, vol. 304(C).
    9. Chen, Shih-Hsin & Chen, Min-Chih, 2013. "Addressing the advantages of using ensemble probabilistic models in Estimation of Distribution Algorithms for scheduling problems," International Journal of Production Economics, Elsevier, vol. 141(1), pages 24-33.
    10. Cai, Qiran & Xu, Qingyang & Qing, Jing & Shi, Gang & Liang, Qiao-Mei, 2022. "Promoting wind and photovoltaics renewable energy integration through demand response: Dynamic pricing mechanism design and economic analysis for smart residential communities," Energy, Elsevier, vol. 261(PB).
    11. Smutnicki, Czeslaw & Pempera, Jaroslaw & Bocewicz, Grzegorz & Banaszak, Zbigniew, 2022. "Cyclic flow-shop scheduling with no-wait constraints and missing operations," European Journal of Operational Research, Elsevier, vol. 302(1), pages 39-49.
    12. Huang, Pei & Sun, Yongjun, 2019. "A collaborative demand control of nearly zero energy buildings in response to dynamic pricing for performance improvements at cluster level," Energy, Elsevier, vol. 174(C), pages 911-921.
    13. Sioud, A. & Gagné, C., 2018. "Enhanced migrating birds optimization algorithm for the permutation flow shop problem with sequence dependent setup times," European Journal of Operational Research, Elsevier, vol. 264(1), pages 66-73.
    14. Zhao, Liyuan & Yang, Ting & Li, Wei & Zomaya, Albert Y., 2022. "Deep reinforcement learning-based joint load scheduling for household multi-energy system," Applied Energy, Elsevier, vol. 324(C).
    15. Muhammad Arshad Shehzad Hassan & Ussama Assad & Umar Farooq & Asif Kabir & Muhammad Zeeshan Khan & S. Sabahat H. Bukhari & Zain ul Abidin Jaffri & Judit Oláh & József Popp, 2022. "Dynamic Price-Based Demand Response through Linear Regression for Microgrids with Renewable Energy Resources," Energies, MDPI, vol. 15(4), pages 1-17, February.
    16. Antonopoulos, Ioannis & Robu, Valentin & Couraud, Benoit & Kirli, Desen & Norbu, Sonam & Kiprakis, Aristides & Flynn, David & Elizondo-Gonzalez, Sergio & Wattam, Steve, 2020. "Artificial intelligence and machine learning approaches to energy demand-side response: A systematic review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 130(C).
    17. Fatima Benbouzid-Si Tayeb & Karima Benatchba & Abd-Essalam Messiaid, 2018. "Game theory-based integration of scheduling with flexible and periodic maintenance planning in the permutation flowshop sequencing problem," Operational Research, Springer, vol. 18(1), pages 221-255, April.
    18. Luo, Zhe & Hong, SeungHo & Ding, YueMin, 2019. "A data mining-driven incentive-based demand response scheme for a virtual power plant," Applied Energy, Elsevier, vol. 239(C), pages 549-559.
    19. Ruiz, Ruben & Stutzle, Thomas, 2007. "A simple and effective iterated greedy algorithm for the permutation flowshop scheduling problem," European Journal of Operational Research, Elsevier, vol. 177(3), pages 2033-2049, March.
    20. Hatami, Sara & Ruiz, Rubén & Andrés-Romano, Carlos, 2015. "Heuristics and metaheuristics for the distributed assembly permutation flowshop scheduling problem with sequence dependent setup times," International Journal of Production Economics, Elsevier, vol. 169(C), pages 76-88.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:5:p:1626-:d:755702. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.