IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2601.17860.html

The Hellinger Bounds on the Kullback-Leibler Divergence and the Bernstein Norm

Author

Listed:
  • Tetsuya Kaji

Abstract

The Kullback-Leibler divergence, the Kullback-Leibler variation, and the Bernstein "norm" are used to quantify discrepancies among probability distributions in likelihood models such as nonparametric maximum likelihood and nonparametric Bayes. They are closely related to the Hellinger distance, which is often easier to work with. Consequently, it is of interest to characterize conditions under which the Hellinger distance serves as an upper bound for these measures. This article characterizes a necessary and sufficient condition for each of the discrepancy measures to be bounded by the Hellinger distance. It accommodates unbounded likelihood ratios and generalizes all previously known results. We then apply it to relax the regularity condition for the sieve maximum likelihood estimator.

Suggested Citation

  • Tetsuya Kaji, 2026. "The Hellinger Bounds on the Kullback-Leibler Divergence and the Bernstein Norm," Papers 2601.17860, arXiv.org.
    • Handle: RePEc:arx:papers:2601.17860
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2601.17860
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ghosal,Subhashis & van der Vaart,Aad, 2017. "Fundamentals of Nonparametric Bayesian Inference," Cambridge Books, Cambridge University Press, number 9780521878265, November.
    2. Gut, Allan, 1992. "The weak law of large numbers for arrays," Statistics & Probability Letters, Elsevier, vol. 14(1), pages 49-52, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Clément de Chaisemartin & Luc Behaghel, 2020. "Estimating the Effect of Treatments Allocated by Randomized Waiting Lists," Econometrica, Econometric Society, vol. 88(4), pages 1453-1477, July.
    2. Qianwen Tan & Subhashis Ghosal, 2021. "Bayesian Analysis of Mixed-effect Regression Models Driven by Ordinary Differential Equations," Sankhya B: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 83(1), pages 3-29, May.
    3. Laura Liu & Hyungsik Roger Moon & Frank Schorfheide, 2023. "Forecasting with a panel Tobit model," Quantitative Economics, Econometric Society, vol. 14(1), pages 117-159, January.
    4. Cl'ement de Chaisemartin & Diego Ciccia & Xavier D'Haultf{oe}uille & Felix Knau, 2024. "Difference-in-Differences Estimators When No Unit Remains Untreated," Papers 2405.04465, arXiv.org, revised Apr 2026.
    5. Gael M. Martin & David T. Frazier & Christian P. Robert, 2020. "Computing Bayes: Bayesian Computation from 1763 to the 21st Century," Monash Econometrics and Business Statistics Working Papers 14/20, Monash University, Department of Econometrics and Business Statistics.
    6. A Stefano Caria & Grant Gordon & Maximilian Kasy & Simon Quinn & Soha Osman Shami & Alexander Teytelboym, 2024. "An Adaptive Targeted Field Experiment: Job Search Assistance for Refugees in Jordan," Journal of the European Economic Association, European Economic Association, vol. 22(2), pages 781-836.
    7. Adler, André & Rosalsky, Andrew & Volodin, Andrej I., 1997. "A mean convergence theorem and weak law for arrays of random elements in martingale type p Banach spaces," Statistics & Probability Letters, Elsevier, vol. 32(2), pages 167-174, March.
    8. Reiß, Markus & Schmidt-Hieber, Johannes, 2020. "Posterior contraction rates for support boundary recovery," Stochastic Processes and their Applications, Elsevier, vol. 130(11), pages 6638-6656.
    9. Dmitry B. Rokhlin, 2021. "Relative utility bounds for empirically optimal portfolios," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 93(3), pages 437-462, June.
    10. Suzanne Sniekers & Aad Vaart, 2020. "Adaptive Bayesian credible bands in regression with a Gaussian process prior," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 82(2), pages 386-425, August.
    11. Shota Gugushvili & Frank van der Meulen & Moritz Schauer & Peter Spreij, 2018. "Nonparametric Bayesian volatility estimation," Papers 1801.09956, arXiv.org, revised Mar 2019.
    12. Laurent Davezies & Xavier D'Haultf{oe}uille & Louise Laage, 2021. "Identification and Estimation of Average Causal Effects in Fixed Effects Logit Models," Papers 2105.00879, arXiv.org, revised Dec 2024.
    13. Martin Burda & Remi Daviet, 2023. "Hamiltonian sequential Monte Carlo with application to consumer choice behavior," Econometric Reviews, Taylor & Francis Journals, vol. 42(1), pages 54-77, January.
    14. Arijit Dey & Arnab Hazra, 2025. "A semiparametric generalized exponential regression model with a principled distance-based prior," Statistical Papers, Springer, vol. 66(7), pages 1-32, December.
    15. Agustín G. Nogales, 2022. "On Consistency of the Bayes Estimator of the Density," Mathematics, MDPI, vol. 10(4), pages 1-6, February.
    16. Christoph Breunig & Ruixuan Liu & Zhengfei Yu, 2025. "Robust Semiparametric Inference for Bayesian Additive Regression Trees," Papers 2509.24634, arXiv.org, revised Oct 2025.
    17. Minerva Mukhopadhyay & Didong Li & David B. Dunson, 2020. "Estimating densities with non‐linear support by using Fisher–Gaussian kernels," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(5), pages 1249-1271, December.
    18. Meier, Alexander & Kirch, Claudia & Meyer, Renate, 2020. "Bayesian nonparametric analysis of multivariate time series: A matrix Gamma Process approach," Journal of Multivariate Analysis, Elsevier, vol. 175(C).
    19. Stephen G. Walker, 2024. "Martingale posterior distributions for cumulative hazard functions," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 51(3), pages 936-955, September.
    20. Hong, Dug Hun & Cabrera, Manuel Ordóñez & Sung, Soo Hak & Volodin, Andrei I., 2000. "On the weak law for randomly indexed partial sums for arrays of random elements in martingale type p Banach spaces," Statistics & Probability Letters, Elsevier, vol. 46(2), pages 177-185, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2601.17860. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.