IDEAS home Printed from https://ideas.repec.org/a/eee/jmvana/v136y2015icp147-162.html
   My bibliography  Save this article

Bayesian structure learning in graphical models

Author

Listed:
  • Banerjee, Sayantan
  • Ghosal, Subhashis

Abstract

We consider the problem of estimating a sparse precision matrix of a multivariate Gaussian distribution, where the dimension p may be large. Gaussian graphical models provide an important tool in describing conditional independence through presence or absence of edges in the underlying graph. A popular non-Bayesian method of estimating a graphical structure is given by the graphical lasso. In this paper, we consider a Bayesian approach to the problem. We use priors which put a mixture of a point mass at zero and certain absolutely continuous distribution on off-diagonal elements of the precision matrix. Hence the resulting posterior distribution can be used for graphical structure learning. The posterior convergence rate of the precision matrix is obtained and is shown to match the oracle rate. The posterior distribution on the model space is extremely cumbersome to compute using the commonly used reversible jump Markov chain Monte Carlo methods. However, the posterior mode in each graph can be easily identified as the graphical lasso restricted to each model. We propose a fast computational method for approximating the posterior probabilities of various graphs using the Laplace approximation approach by expanding the posterior density around the posterior mode. We also provide estimates of the accuracy in the approximation.

Suggested Citation

  • Banerjee, Sayantan & Ghosal, Subhashis, 2015. "Bayesian structure learning in graphical models," Journal of Multivariate Analysis, Elsevier, vol. 136(C), pages 147-162.
  • Handle: RePEc:eee:jmvana:v:136:y:2015:i:c:p:147-162
    DOI: 10.1016/j.jmva.2015.01.015
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0047259X15000202
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ledoit, Olivier & Wolf, Michael, 2004. "A well-conditioned estimator for large-dimensional covariance matrices," Journal of Multivariate Analysis, Elsevier, vol. 88(2), pages 365-411, February.
    2. Park, Trevor & Casella, George, 2008. "The Bayesian Lasso," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 681-686, June.
    3. Aliye Atay-Kayis & Helène Massam, 2005. "A Monte Carlo method for computing the marginal likelihood in nondecomposable Gaussian graphical models," Biometrika, Biometrika Trust, vol. 92(2), pages 317-335, June.
    4. Jian Guo & Elizaveta Levina & George Michailidis & Ji Zhu, 2011. "Joint estimation of multiple graphical models," Biometrika, Biometrika Trust, vol. 98(1), pages 1-15.
    5. Ghosal, Subhashis, 2000. "Asymptotic Normality of Posterior Distributions for Exponential Families when the Number of Parameters Tends to Infinity," Journal of Multivariate Analysis, Elsevier, vol. 74(1), pages 49-68, July.
    6. Lam, Clifford & Fan, Jianqing, 2009. "Sparsistency and rates of convergence in large covariance matrix estimation," LSE Research Online Documents on Economics 31540, London School of Economics and Political Science, LSE Library.
    7. McKay Curtis, S. & Banerjee, Sayantan & Ghosal, Subhashis, 2014. "Fast Bayesian model assessment for nonparametric additive regression," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 347-358.
    8. Rothman, Adam J. & Levina, Elizaveta & Zhu, Ji, 2009. "Generalized Thresholding of Large Covariance Matrices," Journal of the American Statistical Association, American Statistical Association, vol. 104(485), pages 177-186.
    9. Ming Yuan & Yi Lin, 2007. "Model selection and estimation in the Gaussian graphical model," Biometrika, Biometrika Trust, vol. 94(1), pages 19-35.
    10. Yuan, Ming & Lin, Yi, 2005. "Efficient Empirical Bayes Variable Selection and Estimation in Linear Models," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 1215-1225, December.
    11. Dobra, Adrian & Hans, Chris & Jones, Beatrix & Nevins, J.R.Joseph R. & Yao, Guang & West, Mike, 2004. "Sparse graphical models for exploring gene expression data," Journal of Multivariate Analysis, Elsevier, vol. 90(1), pages 196-212, July.
    12. Jianhua Z. Huang & Naiping Liu & Mohsen Pourahmadi & Linxu Liu, 2006. "Covariance matrix selection and estimation via penalised normal likelihood," Biometrika, Biometrika Trust, vol. 93(1), pages 85-98, March.
    13. Cai, Tony & Liu, Weidong & Luo, Xi, 2011. "A Constrained â„“1 Minimization Approach to Sparse Precision Matrix Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 594-607.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nogales, Francisco J. & Alonso, Andrés M. & Avagyan, Vahe, 2015. "D-trace Precision Matrix Estimation Using Adaptive Lasso Penalties," DES - Working Papers. Statistics and Econometrics. WS 21775, Universidad Carlos III de Madrid. Departamento de Estadística.
    2. repec:eee:jmvana:v:173:y:2019:i:c:p:656-671 is not listed on IDEAS
    3. repec:spr:advdac:v:12:y:2018:i:2:d:10.1007_s11634-016-0272-8 is not listed on IDEAS

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jmvana:v:136:y:2015:i:c:p:147-162. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Dana Niculescu). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.