A low-carbon resource-constrained project scheduling problem with limited three-dimensional workspaces and its hierarchical deep reinforcement learning approach

Carbon cap-and-trade policies have been widely adopted across many countries, compelling enterprises to account for carbon emissions cost. Heavy equipment constitutes the primary source of carbon emissions in construction projects, and its operational speed can be adjusted within a certain range. While the faster operational speed shortens the duration of an activity, it also increases emissions and demands more workspace. However, the available workspace for executing an activity is often limited, which indirectly constrains the selection of its execution speed. Moreover, improper workspace arrangements may delay subsequent activities, extending the project makespan. To address these challenges, we propose a low-carbon resource-constrained project scheduling problem with limited three-dimensional workspaces (LCRCPSP-L3DW), extending traditional project scheduling problems by incorporating execution speed selection, workspace arrangement, and emission costs. The objective is to minimize the total cost, including the carbon-related expense. We formulate the LCRCPSP-L3DW as a hierarchical Markov Decision Process (HMDP), where the upper-level agent determines which activity to start and its execution speed, and the lower-level agent determines the workspace position of the selected activity. To solve the HMDP, a hierarchical deep reinforcement learning (HDRL) algorithm is developed, integrating an improved graph attention network (GAT) and a convolutional neural network (CNN), and trained via proximal policy optimization (PPO). Extensive experiments show that the policy learned by HDRL significantly outperforms 84 heuristic methods formed by 42 priority rule combinations and two schedule generation schemes. Ignoring spatial constraints leads to a misestimation of project performance. In addition, carbon cap-and-trade policies can effectively reduce project emissions.