A Monte Carlo analysis of multilevel binary logit model estimator performance

Social scientists are increasingly fitting multilevel models to datasets in which a large number of individuals (N ~ several thousands) are nested within each of a small number of countries (C ~ 25). The researchers are particularly interested in “country effects†, as summarized by either the coefficients on country-level predictors (or cross-level interactions) or the variance of the country-level random effects. Although questions have been raised about the potentially poor performance of estimators of these “country effects†when C is “small†, this issue appears not to be widely appreciated by many social scientist researchers. Using Monte Carlo analysis, I examine the performance of two estimators of a binary-dependent two-level model using a design in which C = 5(5)50 100 and N = 1000 for each country. The results point to i) the superior performance of adaptive quadrature estimators compared with PQL2 estimators, and ii) poor coverage of estimates of “country effects†in models in which C ~ 25, regardless of estimator. The analysis makes extensive use of xtmelogit and simulate and user-written commands such as runmlwin, parmby, and eclplot. Issues associated with having extremely long runtimes are also discussed.