Quantile-based nonparametric estimation of the Kullback-Leibler divergence

Kullback-Leibler divergence is a fundamental measure of dissimilarity between two probability distributions, widely used in statistics, information theory, and machine learning. A quantile-based definition of the Kullback-Leibler divergence is especially useful when the functional form of a probability distribution is not analytically tractable, but its quantile function is available in closed form. In this paper, we propose two nonparametric estimators for the Kullback-Leibler divergence based on its quantile-based representation. We establish the consistency of the proposed estimators and employ a nonparametric bootstrap procedure to construct confidence intervals for the Kullback-Leibler divergence. The performance of the estimators is assessed via simulation studies and illustrated with a real data example.