An improved Q-learning algorithm integrating past and future rewards

Achieving stable and efficient learning remains a core challenge in reinforcement learning. However, traditional Q-learning methods often suffer from instability and slow convergence due to an unbalanced focus on either immediate or future rewards, while neglecting the role of past rewards. To address this, we propose an improved Q-learning algorithm that systematically integrates both past and future rewards through a balancing factor. Specifically, we design three reward weighting strategies: static discount, double exponential discount, and weighted moving average, to process past rewards. Theoretical analysis proves the convergence of the algorithm and reveals that the weighted moving average method minimizes reward variance. In a series of simulation experiments, we find that the weighted moving average approach significantly outperforms other methods in terms of convergence speed, cumulative reward, and learning stability. This study provides a new perspective on reward utilization in reinforcement learning through the balancing factor and adaptive weighting mechanisms.