IDEAS home Printed from
   My bibliography  Save this article

Assessing the predictive ability of a multilevel binary regression model


  • Van Oirbeek, R.
  • Lesaffre, E.


An adaptation of the Brier score and the concordance probability is proposed for the two-level and the three-level random intercept binary regression model. This results in 2 different Brier scores and 3 different C-indices for the two-level binary regression model and 4 different Brier scores and 7 different C-indices for the three-level binary regression model. The ensemble of these measures offers a better view on how the different elements of the random effects model, i.e. the covariates and the random effects, affect the predictive ability of the model separately, evaluated on a within-cluster, between-cluster and global level. For all measures, an estimation procedure using Bayesian and likelihood estimation methods was developed, including a percentile and a BCa non-parametric bootstrap step to construct credible/confidence intervals. In a simulation study, the likelihood estimation procedure showed difficulties in estimating unbiasedly the predictive ability of the random effects, while the Bayesian estimation procedure resulted in good estimation properties for all of the developed measures. The BCa non-parametric bootstrap method resulted in confidence/credible intervals with better coverage properties than the percentile non-parametric bootstrap method. The proposals are applied to a real-life binary data set with a three-level structure using the Bayesian estimation procedure.

Suggested Citation

  • Van Oirbeek, R. & Lesaffre, E., 2012. "Assessing the predictive ability of a multilevel binary regression model," Computational Statistics & Data Analysis, Elsevier, vol. 56(6), pages 1966-1980.
  • Handle: RePEc:eee:csdana:v:56:y:2012:i:6:p:1966-1980
    DOI: 10.1016/j.csda.2011.11.023

    Download full text from publisher

    File URL:
    Download Restriction: Full text for ScienceDirect subscribers only.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    1. Germán Rodríguez & Noreen Goldman, 2001. "Improved estimation procedures for multilevel models with binary response: a case-study," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 164(2), pages 339-355.
    2. Anders Skrondal & Sophia Rabe-Hesketh, 2009. "Prediction in multilevel generalized linear models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(3), pages 659-687.
    3. C. A. Field & A. H. Welsh, 2007. "Bootstrapping clustered data," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(3), pages 369-390.
    Full references (including those not matched with items on IDEAS)


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:56:y:2012:i:6:p:1966-1980. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Dana Niculescu). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.