Evolutionary Reinforcement Learning: A Systematic Review and Future Directions

In response to the limitations of reinforcement learning and Evolutionary Algorithms (EAs) in complex problem-solving, Evolutionary Reinforcement Learning (EvoRL) has emerged as a synergistic solution. This systematic review aims to provide a comprehensive analysis of EvoRL, examining the symbiotic relationship between EAs and reinforcement learning algorithms and identifying critical gaps in relevant application tasks. The review begins by outlining the technological foundations of EvoRL, detailing the complementary relationship between EAs and reinforcement learning algorithms to address the limitations of reinforcement learning, such as parameter sensitivity, sparse rewards, and its susceptibility to local optima. We then delve into the challenges faced by both reinforcement learning and EvoRL, exploring the utility and limitations of EAs in EvoRL. EvoRL itself is constrained by the sampling efficiency and algorithmic complexity, which affect its application in areas like robotic control and large-scale industrial settings. Furthermore, we address significant open issues in the field, such as adversarial robustness, fairness, and ethical considerations. Finally, we propose future directions for EvoRL, emphasizing research avenues that strive to enhance self-adaptation, self-improvement, scalability, interpretability, and so on. To quantify the current state, we analyzed about 100 EvoRL studies, categorizing them based on algorithms, performance metrics, and benchmark tasks. Serving as a comprehensive resource for researchers and practitioners, this systematic review provides insights into the current state of EvoRL and offers a guide for advancing its capabilities in the ever-evolving landscape of artificial intelligence.