Causal Discovery and Classification Using Lempel–Ziv Complexity

Inferring causal relationships in the decision-making processes of machine learning models is essential for advancing explainable artificial intelligence. In this work, we propose a novel causality measure and a distance metric derived from Lempel–Ziv (LZ) complexity. We explore how these measures can be integrated into decision tree classifiers by enabling splits based on features that cause the most changes in the target variable. Specifically, we design (i) a causality-based decision tree, where feature selection is driven by the LZ-based causal score; (ii) a distance-based decision tree, using LZ-based distance measure. We compare these models against traditional decision trees constructed using Gini impurity and Shannon entropy as splitting criteria. While all models show comparable classification performance on standard datasets, the causality-based decision tree significantly outperforms all others on the Coupled Auto Regressive (AR) dataset, which is known to exhibit an underlying causal structure. This result highlights the advantage of incorporating causal information in settings where such a structure exists. Furthermore, based on the features selected in the LZ causality-based tree, we define a causal strength score for each input variable, enabling interpretable insights into the most influential causes of the observed outcomes. This makes our approach a promising step toward interpretable and causally grounded decision-making in AI systems.