Extropy rate: Properties and application in feature selection

Extropy, a complementary dual of entropy, has attracted considerable interest of researchers and scientists. In this study, our focus is on defining conditional extropy and establishing key properties of joint and conditional extropy, such as bounds, uncertainty reduction due to additional information, and Lipschitz continuity. We study the extropy rate for a stochastic process of finitely supported random variables as the entropy rate of the complementary probability distribution and show that its asymptotic behaviour is governed by the normalized logarithmic growth of the support size, making it closely related to the zeroth-order Rényi entropy and the topological entropy. We further establish that, for a stationary and ergodic finite-state process, the entropy rate is asymptotically bounded above by the extropy rate. Furthermore, a numerical study is conducted to analyse the significance of the extropy rate for finite-time stochastic processes in comparison with entropy-based measures. The real-life applicability of the extropy rate is demonstrated through a feature selection method that selects features exhibiting high support size diversity and uncertainty, quantified using the finite-time extropy rate. The proposed method has demonstrated robust, comparative performance compared with existing baselines and an entropy rate-based feature selection method across eight publicly available datasets.