IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v36y2011i1p6-32.html
   My bibliography  Save this article

Performance of Random Effects Model Estimators Under Complex Sampling Designs

Author

Listed:
  • Yue Jia
  • Lynne Stokes
  • Ian Harris
  • Yan Wang

Abstract

In this article, we consider estimation of parameters of random effects models from samples collected via complex multistage designs. Incorporation of sampling weights is one way to reduce estimation bias due to unequal probabilities of selection. Several weighting methods have been proposed in the literature for estimating the parameters of hierarchical models, of which random effects models are a special case. We evaluate the bias of the weighted analysis of variance (ANOVA) estimators of the variance components for a two-level, one-way random effects model. For these estimators, analytic bias expressions are developed and the accuracy of the expressions is evaluated through Monte Carlo simulation. The expressions are used to examine the impact of sample size, the size of the intraclass correlation coefficient (ICC), and the sampling design on the estimators' performance. The sampling designs considered are two-stage, with a general probability design at Level 2 and simple random sampling without replacement (SRS) at Level 1. The study shows that variance component estimators using only first-order weights perform well when both cluster size and ICC are moderate. However, this weighting method should be used with caution for small cluster sizes (less than 20), particularly when ICC is small (less than 0.2). In such scenarios, scaled first-order weighted (SFW) estimators provide an alternative to the difficult-to-use second-order weighted estimators for designs in which SRS is used at the ultimate sampling unit level (Level 1). This article is discussed in the context of large educational survey assessments.

Suggested Citation

  • Yue Jia & Lynne Stokes & Ian Harris & Yan Wang, 2011. "Performance of Random Effects Model Estimators Under Complex Sampling Designs," Journal of Educational and Behavioral Statistics, , vol. 36(1), pages 6-32, February.
  • Handle: RePEc:sae:jedbes:v:36:y:2011:i:1:p:6-32
    DOI: 10.3102/1076998609359793
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998609359793
    Download Restriction: no

    File URL: https://libkey.io/10.3102/1076998609359793?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sophia Rabe‐Hesketh & Anders Skrondal, 2006. "Multilevel modelling of complex survey data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(4), pages 805-827, October.
    2. D. Pfeffermann & C. J. Skinner & D. J. Holmes & H. Goldstein & J. Rasbash, 1998. "Weighting for unequal selection probabilities in multilevel models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 60(1), pages 23-40.
    3. Robert Mislevy, 1991. "Randomization-based inference about latent variables from complex samples," Psychometrika, Springer;The Psychometric Society, vol. 56(2), pages 177-196, June.
    4. Edward L. Korn & Barry I. Graubard, 2003. "Estimating variance components by using survey data," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 65(1), pages 175-190, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yergeau, Marie-Eve, 2020. "Tourism and local welfare: A multilevel analysis in Nepal’s protected areas," World Development, Elsevier, vol. 127(C).
    2. Woodward, Albert & Das, Abhik & Raskin, Ira E. & Morgan-Lopez, Antonio A., 2006. "An exploratory analysis of treatment completion and client and organizational factors using hierarchical linear modeling," Evaluation and Program Planning, Elsevier, vol. 29(4), pages 335-351, November.
    3. Woojin Chung & Roeul Kim, 2020. "A Reversal of the Association between Education Level and Obesity Risk during Ageing: A Gender-Specific Longitudinal Study in South Korea," IJERPH, MDPI, vol. 17(18), pages 1-19, September.
    4. Amini, Chiara & Commander, Simon, 2012. "Educational scores: How does Russia fare?," Journal of Comparative Economics, Elsevier, vol. 40(3), pages 508-527.
    5. Patricia Dörr & Jan Pablo Burgard, 2019. "Data-driven transformations and survey-weighting for linear mixed models," Research Papers in Economics 2019-16, University of Trier, Department of Economics.
    6. Joseph L Dieleman & Tara Templin, 2014. "Random-Effects, Fixed-Effects and the within-between Specification for Clustered Data in Observational Health Studies: A Simulation Study," PLOS ONE, Public Library of Science, vol. 9(10), pages 1-17, October.
    7. Woojin Chung & Roeul Kim, 2020. "Differential Risk of Cognitive Impairment across Paid and Unpaid Occupations in the Middle-Age Population: Evidence from the Korean Longitudinal Study of Aging, 2006–2016," IJERPH, MDPI, vol. 17(9), pages 1-14, April.
    8. Laura M. Stapleton & Yoonjeong Kang, 2018. "Design Effects of Multilevel Estimates From National Probability Samples," Sociological Methods & Research, , vol. 47(3), pages 430-457, August.
    9. Woojin Chung & Roeul Kim, 2020. "Which Occupation is Highly Associated with Cognitive Impairment? A Gender-Specific Longitudinal Study of Paid and Unpaid Occupations in South Korea," IJERPH, MDPI, vol. 17(21), pages 1-17, October.
    10. Nora Würz & Timo Schmid & Nikos Tzavidis, 2022. "Estimating regional income indicators under transformations and access to limited population auxiliary information," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(4), pages 1679-1706, October.
    11. Sophia Rabe‐Hesketh & Anders Skrondal, 2006. "Multilevel modelling of complex survey data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(4), pages 805-827, October.
    12. Robert G. Clark & David G. Steel, 2022. "Sample design for analysis using high‐influence probability sampling," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(4), pages 1733-1756, October.
    13. Francesco Schirripa Spagnolo & Nicola Salvati & Antonella D’Agostino & Ides Nicaise, 2020. "The use of sampling weights in M‐quantile random‐effects regression: an application to Programme for International Student Assessment mathematics scores," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 69(4), pages 991-1012, August.
    14. Bowen, Mary Elizabeth, 2009. "Childhood socioeconomic status and racial differences in disability: Evidence from the Health and Retirement Study (1998-2006)," Social Science & Medicine, Elsevier, vol. 69(3), pages 433-441, August.
    15. Ana Maria Osorio & Catalina Bolancé & Nyovane Madise & Katharina Rathmann, 2013. "Social Determinants of Child Health in Colombia: Can Community Education Moderate the Effect of Family Characteristics?," Working Papers XREAP2013-02, Xarxa de Referència en Economia Aplicada (XREAP), revised Mar 2013.
    16. Amini, Chiara & Nivorozhkin, Eugene, 2015. "The urban–rural divide in educational outcomes: Evidence from Russia," International Journal of Educational Development, Elsevier, vol. 44(C), pages 118-133.
    17. Glen McGee & Jonathan Schildcrout & Sharon‐Lise Normand & Sebastien Haneuse, 2020. "Outcome‐dependent sampling in cluster‐correlated data settings with application to hospital profiling," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(1), pages 379-402, January.
    18. Oǧuz-Alper, Melike & Berger, Yves G., 2020. "Modelling multilevel data under complex sampling designs: An empirical likelihood approach," Computational Statistics & Data Analysis, Elsevier, vol. 145(C).
    19. M. Giovanna Ranalli & Giorgio E. Montanari & Cecilia Vicarelli, 2018. "Estimation of small area counts with the benchmarking property," METRON, Springer;Sapienza Università di Roma, vol. 76(3), pages 349-378, December.
    20. Carla Cristina Rosa de Almeida & João Policarpo Rodrigues Lima & Maria Fernanda Freire Gatto, 2020. "Expenditure on cultural events: preferences or opportunities? An analysis of Brazilian consumer data," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 44(3), pages 451-480, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:36:y:2011:i:1:p:6-32. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.