Dynamic collaborative truck-drone delivery with en-route synchronization and random requests

Coordinated truck and drone delivery is gaining popularity in logistics as it can greatly reduce operation costs. However, existing studies on related operations management problems typically ignore the following important features: (i) the random appearance of requests, which require operators to dynamically respond to the requests; and (ii) the decisions of optimal launch and retrieval locations for trucks and drones instead of fixed to customer locations, which can significantly impact the overall time costs. To tackle these challenges, this study investigates the dynamic collaborative truck-drone routing problem with randomly arriving requests and synchronization on routes. We model the problem as a Markov Decision Process (MDP) and solve the MDP via a reinforcement learning (RL) approach. The proposed RL approach determines: (i) whether each request should be serviced upon arrival, (ii) which truck or drone should be assigned for the request, and (iii) the optimal en-route take-off and landing positions for paired trucks and drones. We further employ a framework of decentralized learning and centralized dispatching in RL to increase performance. Numerical experiments are conducted to assess the proposed solution approach on instances generated based on both the Solomon dataset and real-world operational data of a logistics operator in Singapore over several benchmark algorithms under various battery endurance levels of drones and distinct transportation scenarios including node-based dynamic collaborative truck-drone routing problem, dynamic non-collaborative truck and drone routing problem, and dynamic vehicle routing problem. The results show that our RL solution outperforms the benchmark algorithm in total profit by an average of 28.03 %, and our en-route takeoff and landing scenario outperforms the benchmark scenarios in total profit by an average of 8.43 % in multi-day instances. Additionally, compared to the traditional node-based landing scenario, employing our en-route takeoff and landing strategy can save 0.9 h/(drone*day) of waiting time on average.