Resolving causal confusion in long-tail scenarios: Robust decision-making in autonomous vehicles

In the field of autonomous driving, reinforcement learning often faces distributional shifts in long-tail traffic scenarios, that is, rare but safety-critical events in stochastic traffic dynamics. In such scenarios, excessive reliance on statistical rather than causal relationships can lead to causal confusion. To address this issue, this paper proposes a novel Causal Reinforcement Learning (CRL) paradigm. It introduces a Transition-Reward Joint Causal Deconfounding Network (TR-JCDN), which performs causal deconfounding along both the state-transition and reward-generation. TR-JCDN combines causal graphs with multilayer perceptrons and employs a gradient-based causal discovery method based on the Gumbel–Softmax sampling mechanism. To verify the effectiveness of our CRL, lane changing and pedestrian crossing simulation scenarios were constructed on the SUMO. Experimental results show that CRL significantly improves robustness and safety in long-tail scenarios. Compared with several state-of-the-art algorithms, CRL achieves a 14% increase in average speed and a 12% reduction in collision rate. The learned causal graphs confirm that CRL accurately captures the causal dependencies among states, actions, and rewards, while effectively distinguishing confusion features. Furthermore, the causal robustness analysis demonstrates that CRL maintains stable value estimation and decision-making under changes in causal confusion features, validating the effectiveness of its causal modeling mechanism.