Toward a diagnostic toolkit for linear models with Gaussian‐process distributed random effects

Gaussian processes (GPs) are widely used as distributions of random effects in linear mixed models, which are fit using the restricted likelihood or the closely related Bayesian analysis. This article addresses two problems. First, we propose tools for understanding how data determine estimates in these models, using a spectral basis approximation to the GP under which the restricted likelihood is formally identical to the likelihood for a gamma‐errors GLM with identity link. Second, to examine the data's support for a covariate and to understand how adding that covariate moves variation in the outcome y out of the GP and error parts of the fit, we apply a linear‐model diagnostic, the added variable plot (AVP), both to the original observations and to projections of the data onto the spectral basis functions. The spectral‐ and observation‐domain AVPs estimate the same coefficient for a covariate but emphasize low‐ and high‐frequency data features respectively and thus highlight the covariate's effect on the GP and error parts of the fit, respectively. The spectral approximation applies to data observed on a regular grid; for data observed at irregular locations, we propose smoothing the data to a grid before applying our methods. The methods are illustrated using the forest‐biomass data of Finley et al. (2008).