Resource Allocation in Energy Efficient Heterogeneous IoT Networks using Multi-Agent Deep Reinforcement Learning

Reconfigurable Intelligent Surfaces (RIS) provide a revolutionary passive beamforming technique that can dynamically reshape wireless communication channels. By incorporating RIS into heterogeneous Internet of Things (H-IoT) networks, spectral efficiency and energy efficiency can be improved through intelligent reflection of incoming electromagnetic waves toward desired users. However, the optimization of RIS phase-shift matrices, subcarrier allocation, transmit power allocation, and device scheduling in H-IoT networks constitutes an NP-hard problem due to its high dimensionality and non-convexity, making conventional resource management techniques insufficient for real-time large-scale deployment.Introduction: The Internet of Things (IoT) is experiencing rapid growth and is expected to exceed 29 billion connected devices by 2030worldwide by 2030, resulting in heterogeneous IoT (H-IoT) networks with diverse QoS requirements. Energy efficiency (EE) has become the primary design challenge as most IoT devices operate on limited batteries, and conventional resource management techniques fail to scale in complex, high-dimensional H-IoT environments.Novelty statement: This work proposes RIS-MADDPG, a novel Multi-Agent Deep Reinforcement Learning framework that jointly optimizes RIS passive beamforming phase shifts, subcarrier allocation, and transmit power allocation in RIS-enabled H-IoT networks —a problem not previously addressed through cooperative MADRL with an attention-based centralized critic in distributed heterogeneous IoT networks.Materialsand Methods: The proposed RIS-MADDPG employs the Centralized Training with Decentralized Execution (CTDE) paradigm, where each base station (BS) and RIS controller acts as an independent agent using an extended MADDPG algorithmwith an attention-based centralized critic and prioritized experience replay. Simulations are conducted in Python 3.11 with PyTorch 2.1 and OpenAI Gymnasium, using QuaDRiGa 2.6 for spatially consistent channel modeling, Results are averaged over 20 independent Monte Carlo runs across three H-IoT deployment scenarios(Massive IoT, V2X, IIoT).Resultsand Discussion: Simulation results demonstrate that RIS-MADDPG achieves an average EE gain of 47.31% over non-RIS baselines, 31.6% over single-agent DRL, and 22.8% over convex alternating optimization approaches. The framework converges within 1,200 training episodes, maintains inference latency below 2 ms for up to 20 agents, achieves an overall QoS satisfaction rate of 98.6%, and degrades by only 15.3% under 20% CSI estimation error,compared to 29.1% degradation for convex optimization methods.Concluding Remarks: RIS-MADDPG provides an effective, scalable, and robust solution for energy-efficient resource allocation in heterogeneous IoT networks, offering significant performance gains over both classical optimization and single-agent DRL methods.