Multivariate transfer entropy-guided self-attention for nonlinear causal interaction modeling: Evidence from energy systems

Accurately capturing nonlinear dependencies and causal interactions in multivariate time series remains a fundamental challenge in complex systems analysis. Traditional deep learning models, though effective in sequence modeling, often overlook causality and offer limited interpretability. To address this limitation, we propose the Multivariate Transfer Entropy-guided Self-Attention (MTESA) mechanism, which integrates multivariate transfer entropy (MTE) as a causality-informed prior into the self-attention framework. In this design, the pre-estimated MTE values are incorporated as learnable bias terms that guide the attention mechanism toward causally relevant dependencies while maintaining full model trainability. Rather than altering the core architecture of attention-based models, MTESA provides a flexible integration route for embedding external causal priors into deep learning systems. The proposed framework is evaluated on representative energy forecasting tasks, including coal-fired power plant fuel consumption and wind power generation. Experimental results demonstrate that MTESA consistently improves forecasting accuracy and robustness and yields attention patterns that align with the causal structures revealed by MTE analysis. Overall, this work contributes an effective engineering integration of information-theoretic causality with attention mechanisms, offering a practical pathway toward causality-aware deep learning in multivariate time series forecasting.