IDEAS home Printed from https://ideas.repec.org/a/eee/jmvana/v92y2005i2p324-342.html
   My bibliography  Save this article

Estimation of the entropy of a multivariate normal distribution

Author

Listed:
  • Misra, Neeraj
  • Singh, Harshinder
  • Demchuk, Eugene

Abstract

Motivated by problems in molecular biosciences wherein the evaluation of entropy of a molecular system is important for understanding its thermodynamic properties, we consider the efficient estimation of entropy of a multivariate normal distribution having unknown mean vector and covariance matrix. Based on a random sample, we discuss the problem of estimating the entropy under the quadratic loss function. The best affine equivariant estimator is obtained and, interestingly, it also turns out to be an unbiased estimator and a generalized Bayes estimator. It is established that the best affine equivariant estimator is admissible in the class of estimators that depend on the determinant of the sample covariance matrix alone. The risk improvements of the best affine equivariant estimator over the maximum likelihood estimator (an estimator commonly used in molecular sciences) are obtained numerically and are found to be substantial in higher dimensions, which is commonly the case for atomic coordinates in macromolecules such as proteins. We further establish that even the best affine equivariant estimator is inadmissible and obtain Stein-type and Brewster-Zidek-type estimators dominating it. The Brewster-Zidek-type estimator is shown to be generalized Bayes.

Suggested Citation

  • Misra, Neeraj & Singh, Harshinder & Demchuk, Eugene, 2005. "Estimation of the entropy of a multivariate normal distribution," Journal of Multivariate Analysis, Elsevier, vol. 92(2), pages 324-342, February.
    • Handle: RePEc:eee:jmvana:v:92:y:2005:i:2:p:324-342
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0047-259X(03)00178-7
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rukhin, Andrew L. & Ananda, Malwane M. A., 1992. "Risk behavior of variance estimators in multivariate normal distribution," Statistics & Probability Letters, Elsevier, vol. 13(2), pages 159-166, January.
    2. Ghosh M. & Sinha B. K., 1987. "Inadmissibility Of The Best Equivariant Estimators Of The Variance-Covariance Matrix, The Precision Matrix, And The Generalized Variance Under Entropy Loss," Statistics & Risk Modeling, De Gruyter, vol. 5(3-4), pages 201-228, April.
    3. Sinha, Bimal Kumar, 1976. "On improved estimators of the generalized variance," Journal of Multivariate Analysis, Elsevier, vol. 6(4), pages 617-625, December.
    4. Tatsuya Kubokawa & Yoshihiko Konno, 1990. "Estimating the covariance matrix and the generalized variance under a symmetric loss," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 42(2), pages 331-343, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kayal, Suchandan & Kumar, Somesh, 2013. "Estimation of the Shannon’s entropy of several shifted exponential populations," Statistics & Probability Letters, Elsevier, vol. 83(4), pages 1127-1135.
    2. Withers, Christopher S. & Nadarajah, Saralees, 2011. "Estimates of low bias for the multivariate normal," Statistics & Probability Letters, Elsevier, vol. 81(11), pages 1635-1647, November.
    3. Cai, T. Tony & Liang, Tengyuan & Zhou, Harrison H., 2015. "Law of log determinant of sample covariance matrix and optimal estimation of differential entropy for high-dimensional Gaussian distributions," Journal of Multivariate Analysis, Elsevier, vol. 137(C), pages 161-172.
    4. Lakshmi Kanta Patra & Suchandan Kayal & Somesh Kumar, 2020. "Estimating a function of scale parameter of an exponential population with unknown location under general loss function," Statistical Papers, Springer, vol. 61(6), pages 2511-2527, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Iliopoulos, George, 2008. "UMVU estimation of the ratio of powers of normal generalized variances under correlation," Journal of Multivariate Analysis, Elsevier, vol. 99(6), pages 1051-1069, July.
    2. Iliopoulos, George & Kourouklis, Stavros, 1999. "Improving on the Best Affine Equivariant Estimator of the Ratio of Generalized Variances," Journal of Multivariate Analysis, Elsevier, vol. 68(2), pages 176-192, February.
    3. Pal, Nabendu & Ling, Chiahua, 1995. "Improved minimax estimation of powers of the variance of a multivariate normal distribution under the entropy loss function," Statistics & Probability Letters, Elsevier, vol. 24(3), pages 205-211, August.
    4. Tsukuma, Hisayuki, 2014. "Minimax covariance estimation using commutator subgroup of lower triangular matrices," Journal of Multivariate Analysis, Elsevier, vol. 124(C), pages 333-344.
    5. Tatsuya Kubokawa & M. S. Srivastava, 1999. ""Estimating the Covariance Matrix: A New Approach", June 1999," CIRJE F-Series CIRJE-F-52, CIRJE, Faculty of Economics, University of Tokyo.
    6. Nariaki Sugiura & Yoshihiko Konno, 1988. "Entropy loss and risk of improved estimators for the generalized variance and precision," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 40(2), pages 329-341, June.
    7. Sanat Sarkar, 1991. "Stein-type improvements of confidence intervals for the generalized variance," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 43(2), pages 369-375, June.
    8. Champion, Colin J., 2003. "Empirical Bayesian estimation of normal variances and covariances," Journal of Multivariate Analysis, Elsevier, vol. 87(1), pages 60-79, October.
    9. Katori, Makoto & Konno, Norio, 1992. "On the extinction of Dickman's reaction- diffusion processes," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 186(3), pages 578-590.
    10. Panayiotis Bobotas & George Iliopoulos & Stavros Kourouklis, 2012. "Estimating the ratio of two scale parameters: a simple approach," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 64(2), pages 343-357, April.
    11. Wan, Alan T. K. & Zou, Guohua, 2003. "Optimal critical values of pre-tests when estimating the regression error variance: analytical findings under a general loss structure," Journal of Econometrics, Elsevier, vol. 114(1), pages 165-196, May.
    12. He, Daojiang & Xu, Kai, 2014. "Estimation of the Cholesky decomposition in a conditional independent normal model with missing data," Statistics & Probability Letters, Elsevier, vol. 88(C), pages 27-39.
    13. Ledoit, Olivier & Wolf, Michael, 2021. "Shrinkage estimation of large covariance matrices: Keep it simple, statistician?," Journal of Multivariate Analysis, Elsevier, vol. 186(C).
    14. Sun, Xiaoqian & Zhou, Xian, 2008. "Improved minimax estimation of the bivariate normal precision matrix under the squared loss," Statistics & Probability Letters, Elsevier, vol. 78(2), pages 127-134, February.
    15. Tatsuya Kubokawa, 1994. "Double shrinkage estimation of ratio of scale parameters," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 46(1), pages 95-116, March.
    16. Kubokawa, T. & Srivastava, M. S., 2003. "Estimating the covariance matrix: a new approach," Journal of Multivariate Analysis, Elsevier, vol. 86(1), pages 28-47, July.
    17. Sun, Xiaoqian & Sun, Dongchu, 2005. "Estimation of the Cholesky decomposition of the covariance matrix for a conditional independent normal model," Statistics & Probability Letters, Elsevier, vol. 73(1), pages 1-12, June.
    18. Tatsuya Kubokawa & M. S. Srivastava, 2002. "Estimating the Covariance Matrix: A New Approach," CIRJE F-Series CIRJE-F-162, CIRJE, Faculty of Economics, University of Tokyo.
    19. Hara, Hisayuki, 2001. "Other Classes of Minimax Estimators of Variance Covariance Matrix in Multivariate Normal Distribution," Journal of Multivariate Analysis, Elsevier, vol. 77(2), pages 175-186, May.
    20. Tatsuya Kubokawa & Yoshihiko Konno, 1990. "Estimating the covariance matrix and the generalized variance under a symmetric loss," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 42(2), pages 331-343, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jmvana:v:92:y:2005:i:2:p:324-342. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.