Dynamic Energy Management for Perpetual Operation of Energy Harvesting Wireless Sensor Node Using Fuzzy Q-Learning

In an energy harvesting wireless sensor node (EHWSN), balance of energy harvested and consumption using dynamic energy management to achieve the goal of perpetual operation is one of the most important research topics. In this study, a novel fuzzy Q-learning (FQL)-based dynamic energy management (FQLDEM) is proposed in adapting its policy to the time varying environment, regarding both the harvested energy and the energy consumption of the WSN. The FQLDEM applies Q-learning to train, evaluate, and update the fuzzy rule base and then uses the fuzzy inference system (FIS) for determining the working duty cycle of the sensor of the EHWSN. Through the interaction with the energy harvesting environment, the learning agent of the FQL will be able to find the appropriate fuzzy rules in adapting the working duty cycle for the goal of energy neutrality such that the objective of perpetual operation of the EHWSN can be achieved. Experimental results show that the FQLDEM can maintain the battery charge status at a higher level than other existing methods did, such as the reinforcement learning (RL) method and dynamic duty cycle adaption (DDCA), and achieve the perpetual operation of the EHWSN. Furthermore, experimental results for required on-demand sensing measurements exhibit that the FQLDEM method can be slowly upgraded to meet 65% of the service quality control requirements in the early stage, which outperforms the RL-based and DDCA methods.