IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v13y2025i11p1812-d1667200.html
   My bibliography  Save this article

Multivariate Bayesian Global–Local Shrinkage Methods for Regularisation in the High-Dimensional Linear Model

Author

Listed:
  • Valentina Mameli

    (Department of Economics and Statistics, University of Udine, 33100 Udine, Italy)

  • Debora Slanzi

    (Venice School of Management, Ca’ Foscari University of Venice, 30121 Venice, Italy
    European Centre for Living Technology, Ca’ Foscari University of Venice, 30123 Venice, Italy)

  • Jim E. Griffin

    (Department of Statistical Science, University College London, London WC1E 6BT, UK)

  • Philip J. Brown

    (Department of Mathematics, Statistics and Actuarial Science, University of Kent, Canterbury CT2 7NZ, UK)

Abstract

This paper considers Bayesian regularisation using global–local shrinkage priors in the multivariate general linear model when there are many more explanatory variables than observations. We adopt priors’ structures used extensively in univariate problems (conjugate and non-conjugate with tail behaviour ranging from polynomial to exponential) and consider how the addition of error correlation in the multivariate set-up affects the performance of these priors. Two different datasets (from drug discovery and chemometrics) with many covariates are used for comparison, and these are supplemented by a small simulation study to corroborate the role of error correlation. We find that structural assumptions of the prior distribution on regression coefficients can be more significant than the tail behaviour. In particular, if the structural assumption of conjugacy is used, the performance of the posterior predictive distribution deteriorates relative to non-conjugate choices as the error correlation becomes stronger.

Suggested Citation

  • Valentina Mameli & Debora Slanzi & Jim E. Griffin & Philip J. Brown, 2025. "Multivariate Bayesian Global–Local Shrinkage Methods for Regularisation in the High-Dimensional Linear Model," Mathematics, MDPI, vol. 13(11), pages 1-21, May.
    • Handle: RePEc:gam:jmathe:v:13:y:2025:i:11:p:1812-:d:1667200
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/13/11/1812/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/13/11/1812/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Carlos M. Carvalho & Nicholas G. Polson & James G. Scott, 2010. "The horseshoe estimator for sparse signals," Biometrika, Biometrika Trust, vol. 97(2), pages 465-480.
    2. Leonardo Bottolo & Marco Banterle & Sylvia Richardson & Mika Ala‐Korpela & Marjo‐Riitta Järvelin & Alex Lewin, 2021. "A computationally efficient Bayesian seemingly unrelated regressions model for high‐dimensional quantitative trait loci discovery," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 70(4), pages 886-908, August.
    3. Annalisa Cadonna & Sylvia Frühwirth-Schnatter & Peter Knaus, 2020. "Triple the Gamma—A Unifying Shrinkage Prior for Variance and Variable Selection in Sparse State Space and TVP Models," Econometrics, MDPI, vol. 8(2), pages 1-36, May.
    4. Friedman, Jerome H. & Hastie, Trevor & Tibshirani, Rob, 2010. "Regularization Paths for Generalized Linear Models via Coordinate Descent," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 33(i01).
    5. P. J. Brown & M. Vannucci & T. Fearn, 1998. "Multivariate Bayesian variable selection and prediction," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 60(3), pages 627-641.
    6. Debamita Kundu & Riten Mitra & Jeremy T. Gaskins, 2021. "Bayesian variable selection for multioutcome models through shared shrinkage," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 48(1), pages 295-320, March.
    7. Leo Breiman & Jerome H. Friedman, 1997. "Predicting Multivariate Responses in Multiple Linear Regression," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 59(1), pages 3-54.
    8. P. J. Brown & M. Vannucci & T. Fearn, 2002. "Bayes model averaging with selection of regressors," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(3), pages 519-536, August.
    9. Anindya Bhadra & Bani K. Mallick, 2013. "Joint High-Dimensional Bayesian Variable and Covariance Selection with an Application to eQTL Analysis," Biometrics, The International Biometric Society, vol. 69(2), pages 447-457, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bai, Ray & Ghosh, Malay, 2018. "High-dimensional multivariate posterior consistency under global–local shrinkage priors," Journal of Multivariate Analysis, Elsevier, vol. 167(C), pages 157-170.
    2. Anastasia Dimiski, 2020. "Factors that affect Students’ performance in Science: An application using Gini-BMA methodology in PISA 2015 dataset," Working Papers 2004, University of Guelph, Department of Economics and Finance.
    3. Leonardo Bottolo & Marco Banterle & Sylvia Richardson & Mika Ala‐Korpela & Marjo‐Riitta Järvelin & Alex Lewin, 2021. "A computationally efficient Bayesian seemingly unrelated regressions model for high‐dimensional quantitative trait loci discovery," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 70(4), pages 886-908, August.
    4. Ander Wilson & Brian J. Reich, 2014. "Confounder selection via penalized credible regions," Biometrics, The International Biometric Society, vol. 70(4), pages 852-861, December.
    5. Bernardi, Mauro & Costola, Michele, 2019. "High-dimensional sparse financial networks through a regularised regression model," SAFE Working Paper Series 244, Leibniz Institute for Financial Research SAFE.
    6. Dimitris Korobilis, 2008. "Forecasting in vector autoregressions with many predictors," Advances in Econometrics, in: Bayesian Econometrics, pages 403-431, Emerald Group Publishing Limited.
    7. Yu Bai & Andrea Carriero & Todd E. Clark & Massimiliano Marcellino, 2022. "Macroeconomic forecasting in a multi‐country context," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(6), pages 1230-1255, September.
    8. Peter Knaus & Sylvia Fruhwirth-Schnatter, 2023. "The Dynamic Triple Gamma Prior as a Shrinkage Process Prior for Time-Varying Parameter Models," Papers 2312.10487, arXiv.org, revised Jan 2025.
    9. Li, Yunfan & Datta, Jyotishka & Craig, Bruce A. & Bhadra, Anindya, 2021. "Joint mean–covariance estimation via the horseshoe," Journal of Multivariate Analysis, Elsevier, vol. 183(C).
    10. Daniel Spencer & Rajarshi Guhaniyogi & Raquel Prado, 2020. "Joint Bayesian Estimation of Voxel Activation and Inter-regional Connectivity in fMRI Experiments," Psychometrika, Springer;The Psychometric Society, vol. 85(4), pages 845-869, December.
    11. Simila, Timo & Tikka, Jarkko, 2007. "Input selection and shrinkage in multiresponse linear regression," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 406-422, September.
    12. Posch, Konstantin & Arbeiter, Maximilian & Pilz, Juergen, 2020. "A novel Bayesian approach for variable selection in linear regression models," Computational Statistics & Data Analysis, Elsevier, vol. 144(C).
    13. Uddin, Md Nazir & Gaskins, Jeremy T., 2023. "Shared Bayesian variable shrinkage in multinomial logistic regression," Computational Statistics & Data Analysis, Elsevier, vol. 177(C).
    14. van Erp, Sara & Oberski, Daniel L. & Mulder, Joris, 2018. "Shrinkage priors for Bayesian penalized regression," OSF Preprints cg8fq, Center for Open Science.
    15. ter Braak, Cajo J.F., 2006. "Bayesian sigmoid shrinkage with improper variance priors and an application to wavelet denoising," Computational Statistics & Data Analysis, Elsevier, vol. 51(2), pages 1232-1242, November.
    16. David Kohns & Tibor Szendrei, 2021. "Decoupling Shrinkage and Selection for the Bayesian Quantile Regression," Papers 2107.08498, arXiv.org.
    17. Yunxi Zhang & Soeun Kim, 2024. "Gaussian Graphical Model Estimation and Selection for High-Dimensional Incomplete Data Using Multiple Imputation and Horseshoe Estimators," Mathematics, MDPI, vol. 12(12), pages 1-15, June.
    18. Ouysse, Rachida & Kohn, Robert, 2010. "Bayesian variable selection and model averaging in the arbitrage pricing theory model," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3249-3268, December.
    19. repec:osf:osfxxx:cg8fq_v1 is not listed on IDEAS
    20. Sylvia Fruhwirth-Schnatter & Peter Knaus, 2022. "Sparse Bayesian State-Space and Time-Varying Parameter Models," Papers 2207.12147, arXiv.org.
    21. Niko Hauzenberger, 2020. "Flexible Mixture Priors for Large Time-varying Parameter Models," Papers 2006.10088, arXiv.org, revised Nov 2020.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:13:y:2025:i:11:p:1812-:d:1667200. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.