Unconstrained Metropolis–Hastings Sampling of Covariance Matrices

Markov chain Monte Carlo (MCMC), the predominant algorithm for fitting hierarchal models to data in a Bayesian setting, relies on the ability to sample from the full conditional distributions of unobserved parameters. Covariance or precision matrices offer a unique sampling challenge due to the constraints on the elements of symmetric positive-definite matrices. Historically, MCMC algorithms have relied upon conjugate prior-likelihood forms for such matrices to sample directly from the conditional distribution. In the absence of conjugacy, Metropolis–Hastings sampling is problematic due to the challenge of generating suitable proposal values. Here, we develop a general Metropolis–Hastings sampling algorithm for covariance matrices, which makes proposals on an unconstrained domain through use of a bijective transformation. Our main contribution is a derivation of the Metropolis–Hastings acceptance probability for proposals made using this transformation. Pseudocode is provided for implementing our algorithm, and we also present a simulation study analyzing asymptotic performance under varying matrix dimensions and correlation structures. We present a real data example using this methodology to fit a hierarchical model for species abundance and detection using correlated random effects, for several bird species in the White Mountain National Forest, New Hampshire. Our method provides a simple and general-purpose approach for MCMC sampling of nonconjugate covariance matrices in arbitrary hierarchical model structures.