IDEAS home Printed from
   My bibliography  Save this paper

A Monte Carlo analysis of multilevel binary logit model estimator performance


  • Stephen P. Jenkins

    (London School of Economics)


Social scientists are increasingly fitting multilevel models to datasets in which a large number of individuals (N ~ several thousands) are nested within each of a small number of countries (C ~ 25). The researchers are particularly interested in “country effects†, as summarized by either the coefficients on country-level predictors (or cross-level interactions) or the variance of the country-level random effects. Although questions have been raised about the potentially poor performance of estimators of these “country effects†when C is “small†, this issue appears not to be widely appreciated by many social scientist researchers. Using Monte Carlo analysis, I examine the performance of two estimators of a binary-dependent two-level model using a design in which C = 5(5)50 100 and N = 1000 for each country. The results point to i) the superior performance of adaptive quadrature estimators compared with PQL2 estimators, and ii) poor coverage of estimates of “country effects†in models in which C ~ 25, regardless of estimator. The analysis makes extensive use of xtmelogit and simulate and user-written commands such as runmlwin, parmby, and eclplot. Issues associated with having extremely long runtimes are also discussed.

Suggested Citation

  • Stephen P. Jenkins, 2013. "A Monte Carlo analysis of multilevel binary logit model estimator performance," United Kingdom Stata Users' Group Meetings 2013 04, Stata Users Group.
  • Handle: RePEc:boc:usug13:04

    Download full text from publisher

    File URL:
    Download Restriction: no

    References listed on IDEAS

    1. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, July.
    2. Brunell, Thomas L. & DiNardo, John, 2004. "A Propensity Score Reweighting Approach to Estimating the Partisan Effects of Full Turnout in American Presidential Elections," Political Analysis, Cambridge University Press, vol. 12(01), pages 28-45, December.
    3. Alberto Abadie & Guido W. Imbens, 2008. "On the Failure of the Bootstrap for Matching Estimators," Econometrica, Econometric Society, vol. 76(6), pages 1537-1557, November.
    4. Li, Qi & Racine, Jeffrey S. & Wooldridge, Jeffrey M., 2009. "Efficient Estimation of Average Treatment Effects with Mixed Categorical and Continuous Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(2), pages 206-223.
    5. Wooldridge, Jeffrey M., 2007. "Inverse probability weighted estimation for general missing data problems," Journal of Econometrics, Elsevier, vol. 141(2), pages 1281-1301, December.
    6. Austin Nichols, 2007. "Causal inference with observational data," Stata Journal, StataCorp LP, vol. 7(4), pages 507-541, December.
    7. Giovanni Cerulli, 2012. "Ivtreatreg: a new STATA routine for estimating binary treatment models with heterogeneous response to treatment under observable and unobservable selection," CERIS Working Paper 201203, Institute for Economic Research on Firms and Growth - Moncalieri (TO) ITALY -NOW- Research Institute on Sustainable Economic Growth - Moncalieri (TO) ITALY.
    Full references (including those not matched with items on IDEAS)

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:boc:usug13:04. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Christopher F Baum). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.