Multi-agent deep reinforcement learning based demand response and energy management for heavy industries with discrete manufacturing systems

Energy-centric decarbonization of heavy industries, such as steel and cement, necessitates their participation in integrating Renewable Energy Sources (RES) and effective Demand Response (DR) programs. This situation has created the opportunities to research control algorithms in diverse DR scenarios. Further, the industrial sector’s unique challenges, including the diversity of operations and the need for uninterrupted production, bring unique challenges in designing and implementing control algorithms. Reinforcement learning (RL) methods are practical solutions to the unique challenges faced by the industrial sector. Nevertheless, research in RL for industrial demand response has not yet achieved the level of standardization seen in other areas of RL research, hindering broader progress. To propel the research progress, we propose a multi-agent reinforcement learning (MARL)-based energy management system designed to optimize energy consumption in energy-intensive industrial settings by leveraging dynamic pricing DR schemes. The study highlights the creation of a MARL environment and addresses these challenges by designing a general framework that allows researchers to replicate and implement MARL environments for industrial sectors. The proposed framework incorporates a Partially Observable Markov Decision Process (POMDP) to model energy consumption and production processes while introducing buffer storage constraints and a flexible reward function that balances production efficiency and cost reduction. The paper evaluates the framework through experimental validation within a steel powder manufacturing facility. The experimental results validate our framework and also demonstrate the effectiveness of the MARL-based energy management system.