IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v213y2026ics0167947325001306.html

Random effects misspecification and its consequences for prediction in generalized linear mixed models

Author

Listed:
  • Vu, Quan
  • Hui, Francis K.C.
  • Muller, Samuel
  • Welsh, A.H.

Abstract

When fitting generalized linear mixed models, choosing the random effects distribution is an important decision. As random effects are unobserved, misspecification of their distribution is a real possibility. Thus, the consequences of random effects misspecification for point prediction and prediction inference of random effects in generalized linear mixed models need to be investigated. A combination of theory, simulation, and a real application is used to explore the effect of using the common normality assumption for the random effects distribution when the correct specification is a mixture of normal distributions, focusing on the impacts on point prediction, mean squared prediction errors, and prediction intervals. Results show that the level of shrinkage for the predicted random effects can differ greatly under the two random effect distributions, and so is susceptible to misspecification. Also, the unconditional mean squared prediction errors for the random effects are almost always larger under the misspecified normal random effects distribution, while results for the mean squared prediction errors conditional on the random effects are more complicated but remain generally larger under the misspecified distribution (especially when the true random effect is close to the mean of one of the component distributions in the true mixture distribution). Results for prediction intervals indicate that the overall coverage probability is, in contrast, not greatly impacted by misspecification. It is concluded that misspecifying the random effects distribution can affect prediction of random effects, and greater caution is recommended when adopting the normality assumption in generalized linear mixed models.

Suggested Citation

  • Vu, Quan & Hui, Francis K.C. & Muller, Samuel & Welsh, A.H., 2026. "Random effects misspecification and its consequences for prediction in generalized linear mixed models," Computational Statistics & Data Analysis, Elsevier, vol. 213(C).
  • Handle: RePEc:eee:csdana:v:213:y:2026:i:c:s0167947325001306
    DOI: 10.1016/j.csda.2025.108254
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947325001306
    Download Restriction: Full text for ScienceDirect subscribers only.

    File URL: https://libkey.io/10.1016/j.csda.2025.108254?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Francis K.C. Hui & Nicole A. Hill & A.H. Welsh, 2022. "Assuming independence in spatial latent variable models: Consequences and implications of misspecification," Biometrics, The International Biometric Society, vol. 78(1), pages 85-99, March.
    2. Flores-Agreda, Daniel & Cantoni, Eva, 2019. "Bootstrap estimation of uncertainty in prediction for generalized linear mixed models," Computational Statistics & Data Analysis, Elsevier, vol. 130(C), pages 1-17.
    3. Hien D. Nguyen & Geoffrey McLachlan, 2019. "On approximations via convolution-defined mixture models," Communications in Statistics - Theory and Methods, Taylor & Francis Journals, vol. 48(16), pages 3945-3955, August.
    4. Verbeke, Geert & Lesaffre, Emmanuel, 1997. "The effect of misspecifying the random-effects distribution in linear mixed models for longitudinal data," Computational Statistics & Data Analysis, Elsevier, vol. 23(4), pages 541-556, February.
    5. Bates, Douglas & Mächler, Martin & Bolker, Ben & Walker, Steve, 2015. "Fitting Linear Mixed-Effects Models Using lme4," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 67(i01).
    6. Francis K.C. Hui & David I. Warton & Scott D. Foster, 2015. "Order selection in finite mixture models: complete or observed likelihood information criteria?," Biometrika, Biometrika Trust, vol. 102(3), pages 724-730.
    7. James R. Carpenter & Harvey Goldstein & Jon Rasbash, 2003. "A novel bootstrap procedure for assessing the relationship between class size and achievement," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 52(4), pages 431-443, October.
    8. Komárek, Arnost & Lesaffre, Emmanuel, 2008. "Generalized linear mixed model with a penalized Gaussian mixture as a random effects distribution," Computational Statistics & Data Analysis, Elsevier, vol. 52(7), pages 3441-3458, March.
    9. Zheng, Nan & Cadigan, Noel, 2021. "Frequentist delta-variance approximations with mixed-effects models and TMB," Computational Statistics & Data Analysis, Elsevier, vol. 160(C).
    10. Ha, Il Do & Sylvester, Richard & Legrand, Catherine & MacKenzie, Gilbert, 2011. "Frailty modelling for survival data from multi-centre clinical trials," LIDAM Reprints ISBA 2011060, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    11. Youngjo Lee & Myoungjin Jang & Woojoo Lee, 2011. "Prediction interval for disease mapping using hierarchical likelihood," Computational Statistics, Springer, vol. 26(1), pages 159-179, March.
    12. Ziyang Lyu & A.H. Welsh, 2022. "Asymptotics for EBLUPs: Nested Error Regression Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 117(540), pages 2028-2042, October.
    13. Francis K. C. Hui & Samuel Müller & Alan H. Welsh, 2021. "Random Effects Misspecification Can Have Severe Consequences for Random Effects Inference in Linear Mixed Models," International Statistical Review, International Statistical Institute, vol. 89(1), pages 186-206, April.
    14. Ray Chambers & Nikos Tzavidis, 2006. "M-quantile models for small area estimation," Biometrika, Biometrika Trust, vol. 93(2), pages 255-268, June.
    15. Andreea L. Erciulescu & Jean D. Opsomer, 2022. "A model‐based approach to predict employee compensation components," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(5), pages 1503-1520, November.
    16. Anders Skrondal & Sophia Rabe‐Hesketh, 2009. "Prediction in multilevel generalized linear models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(3), pages 659-687, June.
    17. Charles E. McCulloch & John M. Neuhaus, 2011. "Prediction of Random Effects in Linear and Generalized Linear Models under Model Misspecification," Biometrics, The International Biometric Society, vol. 67(1), pages 270-279, March.
    18. Agresti, Alan & Caffo, Brian & Ohman-Strickland, Pamela, 2004. "Examples in which misspecification of a random effects distribution reduces efficiency, and possible remedies," Computational Statistics & Data Analysis, Elsevier, vol. 47(3), pages 639-653, October.
    19. Pan, Lanfeng & Li, Yehua & He, Kevin & Li, Yanming & Li, Yi, 2020. "Generalized linear mixed models with Gaussian mixture random effects: Inference and application," Journal of Multivariate Analysis, Elsevier, vol. 175(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Francis K. C. Hui & Samuel Müller & Alan H. Welsh, 2021. "Random Effects Misspecification Can Have Severe Consequences for Random Effects Inference in Linear Mixed Models," International Statistical Review, International Statistical Institute, vol. 89(1), pages 186-206, April.
    2. Anders Skrondal & Sophia Rabe-Hesketh, 2022. "The Role of Conditional Likelihoods in Latent Variable Modeling," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 799-834, September.
    3. Freddy Hernández & Viviana Giampaoli, 2018. "The Impact of Misspecified Random Effect Distribution in a Weibull Regression Mixed Model," Stats, MDPI, vol. 1(1), pages 1-29, May.
    4. Leonardo Grilli & Carla Rampichini, 2015. "Specification of random effects in multilevel models: a review," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(3), pages 967-976, May.
    5. Reluga, Katarzyna & Lombardía, María-José & Sperlich, Stefan, 2024. "Bootstrap-based statistical inference for linear mixed effects under misspecifications," Computational Statistics & Data Analysis, Elsevier, vol. 199(C).
    6. Shun Yu & Xianzheng Huang, 2017. "Random-intercept misspecification in generalized linear mixed models for binary responses," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 26(3), pages 333-359, August.
    7. Valéry Dongmo Jiongo & Pierre Nguimkeu, 2018. "Bootstrapping Mean Squared Errors of Robust Small-Area Estimators: Application to the Method-of-Payments Data," Staff Working Papers 18-28, Bank of Canada.
    8. Huang, Xianzheng, 2011. "Detecting random-effects model misspecification via coarsened data," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 703-714, January.
    9. Stefano Marchetti & Maciej Beręsewicz & Nicola Salvati & Marcin Szymkowiak & Łukasz Wawrowski, 2018. "The use of a three‐level M‐quantile model to map poverty at local administrative unit 1 in Poland," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(4), pages 1077-1104, October.
    10. Antonello Maruotti & Pierfrancesco Alaimo Di Loro, 2023. "CO2 emissions and growth: A bivariate bidimensional mean‐variance random effects model," Environmetrics, John Wiley & Sons, Ltd., vol. 34(5), August.
    11. Saskia Litière & Ariel Alonso & Geert Molenberghs, 2007. "Type I and Type II Error Under Random-Effects Misspecification in Generalized Linear Mixed Models," Biometrics, The International Biometric Society, vol. 63(4), pages 1038-1044, December.
    12. Changli Lu & Yuqin Sun & Yongge Tian, 2018. "Two competing linear random-effects models and their connections," Statistical Papers, Springer, vol. 59(3), pages 1101-1115, September.
    13. Flores-Agreda, Daniel & Cantoni, Eva, 2019. "Bootstrap estimation of uncertainty in prediction for generalized linear mixed models," Computational Statistics & Data Analysis, Elsevier, vol. 130(C), pages 1-17.
    14. Philip S. Boonstra & Bhramar Mukherjee & Jeremy M. G. Taylor & Mef Nilbert & Victor Moreno & Stephen B. Gruber, 2011. "Bayesian Modeling for Genetic Anticipation in Presence of Mutational Heterogeneity: A Case Study in Lynch Syndrome," Biometrics, The International Biometric Society, vol. 67(4), pages 1627-1637, December.
    15. Frink, Nicolas & Schmid, Timo, 2025. "Small area prediction of counts under machine learning-type mixed models," Computational Statistics & Data Analysis, Elsevier, vol. 211(C).
    16. George Leckie, 2018. "Avoiding Bias When Estimating the Consistency and Stability of Value-Added School Effects," Journal of Educational and Behavioral Statistics, , vol. 43(4), pages 440-468, August.
    17. Nan Zheng & Noel Cadigan, 2025. "Improved confidence intervals for nonlinear mixed-effects and nonparametric regression models," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 77(1), pages 105-126, February.
    18. Anna Gottard & Giulia Vannucci & Leonardo Grilli & Carla Rampichini, 2023. "Mixed-effect models with trees," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 17(2), pages 431-461, June.
    19. Tomasz .Zk{a}d{l}o & Adam Chwila, 2024. "A step towards the integration of machine learning and classic model-based survey methods," Papers 2402.07521, arXiv.org, revised Jul 2025.
    20. Charles E. McCulloch & John M. Neuhaus, 2011. "Prediction of Random Effects in Linear and Generalized Linear Models under Model Misspecification," Biometrics, The International Biometric Society, vol. 67(1), pages 270-279, March.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:213:y:2026:i:c:s0167947325001306. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.