ATDrive: Collaborative decision-making method for autonomous truck platoon considering intra-negotiation mechanism

The autonomous truck platoon control using multi-agent reinforcement learning (MARL) presents a few challenges, including social dilemmas, restricted perception information, and rigid formation structures. To overcome these challenges, this study proposes a novel collaborative decision-making approach, termed autonomous truck driving collaborative decision-making (ATDrive), which is based on QMIX, incorporating an intra-negotiation mechanism. First, this study presents a social influence reward within the framework of collaborative decision-making to address the social dilemma problem. Second, a customized partially observable Markov decision process (POMDP) model is designed, which enhances more granular state representations tailored to the roles within the truck platoon. Third, a flexible formation control strategy is proposed, featuring an interlaced structure that promotes efficient information sharing and management through adaptive leadership roles and task allocation. Finally, the proposed method is trained using the centralized training and decentralized execution (CTDE) paradigm. Experimental findings reveal that the ATDrive method outperforms baseline models across various traffic flow settings. Specifically, it achieves a 16.76% reduction in energy consumption compared with conventional IDM-MOBIL models and 30.57% compared with multi-agent imitation learning models. In addition, the proposed formation control method demonstrates a 39% reduction in collision rates compared with representative formation structures. These findings suggest that the proposed ATDrive method effectively promotes a balanced credit assignment among trucks, fosters the development of an intra-negotiation mechanism, and offers valuable insights for minimizing operational costs within the truck platoon.