Study of electric vehicle charging scheduling with renewable energy: Offline and stochastic online optimization

This paper studies the Electric Vehicle Charging Scheduling (EVCS) problem in a charging station powered by grid electricity and a Renewable Energy Source (RES). The objective is to accept the maximum number of charging requests, reaching a desired state of charge of electric vehicles at their departure time, while minimizing the cost of the energy supplied by the grid. We propose an extended formulation of the EVCS problem, including charging request acceptance with a waiting queue. First, the offline problem is modeled as a Mixed-Integer Linear Program (MILP) problem to derive optimal solutions and serve as a benchmark for evaluating the performance of stochastic online optimization methods. Next, we formulate a Markov decision process for the stochastic online problem and propose several methods. These methods include rule-based algorithms, a rolling horizon approach based on the MILP formulation, and a novel deep Reinforcement Learning (RL) mechanism based on proximal policy optimization with masking of invalid actions. A comprehensive comparison between these methods is conducted using a developed open-source environment within the OpenAI-Gymnasium framework. The proposed methods are evaluated using scenarios with increasing numbers of EV charging requests. Computational experiments show that among the online strategies, the rolling horizon algorithm achieves the highest RES share, while the RL approach demonstrates strong potential for scalability in complex, high-demand scenarios. Overall, this study provides a foundation for developing advanced algorithmic solutions and contributes to refining the EVCS problem formulation in stochastic and complex decision-making contexts.