Estimating densities with non‐linear support by using Fisher–Gaussian kernels

Current tools for multivariate density estimation struggle when the density is concentrated near a non‐linear subspace or manifold. Most approaches require the choice of a kernel, with the multivariate Gaussian kernel by far the most commonly used. Although heavy‐tailed and skewed extensions have been proposed, such kernels cannot capture curvature in the support of the data. This leads to poor performance unless the sample size is very large relative to the dimension of the data. The paper proposes a novel generalization of the Gaussian distribution, which includes an additional curvature parameter. We refer to the proposed class as Fisher–Gaussian kernels, since they arise by sampling from a von Mises–Fisher density on the sphere and adding Gaussian noise. The Fisher–Gaussian density has an analytic form and is amenable to straightforward implementation within Bayesian mixture models by using Markov chain Monte Carlo sampling. We provide theory on large support and illustrate gains relative to competitors in simulated and real data applications.