IDEAS home Printed from https://ideas.repec.org/a/bla/stanee/v64y2010i2p157-170.html
   My bibliography  Save this article

The effect of estimation method and sample size in multilevel structural equation modeling

Author

Listed:
  • Joop J. Hox
  • Cora J. M. Maas
  • Matthieu J. S. Brinkhuis

Abstract

Multilevel structural equation modeling (multilevel SEM) has become an established method to analyze multilevel multivariate data. The first useful estimation method was the pseudobalanced method. This method is approximate because it assumes that all groups have the same size, and ignores unbalance when it exists. In addition, full information maximum likelihood (ML) estimation is now available, which is often combined with robust chi‐squares and standard errors to accommodate unmodeled heterogeneity (MLR). In addition, diagonally weighted least squares (DWLS) methods have become available as estimation methods. This article compares the pseudobalanced estimation method, ML(R), and two DWLS methods by simulating a multilevel factor model with unbalanced data. The simulations included different sample sizes at the individual and group levels and different intraclass correlation (ICC). The within‐group part of the model posed no problems. In the between part of the model, the different ICC sizes had no effect. There is a clear interaction effect between number of groups and estimation method. ML reaches unbiasedness fastest, then the two DWLS methods, then MLR, and then the pseudobalanced method (which needs more than 200 groups). We conclude that both ML(R) and DWLS are genuine improvements on the pseudobalanced approximation. With small sample sizes, the robust methods are not recommended.

Suggested Citation

  • Joop J. Hox & Cora J. M. Maas & Matthieu J. S. Brinkhuis, 2010. "The effect of estimation method and sample size in multilevel structural equation modeling," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 64(2), pages 157-170, May.
    • Handle: RePEc:bla:stanee:v:64:y:2010:i:2:p:157-170
    DOI: 10.1111/j.1467-9574.2009.00445.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1467-9574.2009.00445.x
    Download Restriction: no
    File URL: https://libkey.io/10.1111/j.1467-9574.2009.00445.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. RODERICK P. McDONALD, 1994. "The Bilevel Reticular Action Model for Path Analysis with Latent Variables," Sociological Methods & Research, , vol. 22(3), pages 399-413, February.
    2. Stephen W. Raudenbush & Robert Sampson, 1999. "Assessing Direct and Indirect Effects in Multilevel Designs with Latent Variables," Sociological Methods & Research, , vol. 28(2), pages 123-153, November.
    3. Ke-Hai Yuan & Kentaro Hayashi, 2005. "On muthén’s maximum likelihood for two-level covariance structure models," Psychometrika, Springer;The Psychometric Society, vol. 70(1), pages 147-167, March.
    4. Harvey Goldstein & Roderick McDonald, 1988. "A general model for the analysis of multilevel data," Psychometrika, Springer;The Psychometric Society, vol. 53(4), pages 455-467, December.
    5. Maas, Cora J. M. & Hox, J.J.Joop J., 2004. "The influence of violations of assumptions on multilevel parameter estimates and their standard errors," Computational Statistics & Data Analysis, Elsevier, vol. 46(3), pages 427-440, June.
    6. Johannes Berkhof & Tom A. B. Snijders, 2001. "Variance Component Testing in Multilevel Models," Journal of Educational and Behavioral Statistics, , vol. 26(2), pages 133-152, June.
    7. Jiajuan Liang & Peter Bentler, 2004. "An EM algorithm for fitting two-level structural equation models," Psychometrika, Springer;The Psychometric Society, vol. 69(1), pages 101-122, March.
    8. White, Halbert, 1982. "Maximum Likelihood Estimation of Misspecified Models," Econometrica, Econometric Society, vol. 50(1), pages 1-25, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Domenico Sardanelli & Agostino Vollero & Alfonso Siano & Gianmaria Bottoni, 2019. "Lowering the pirate flag: a TPB study of the factors influencing the intention to pay for movie streaming services," Electronic Commerce Research, Springer, vol. 19(3), pages 549-574, September.
    2. Dave Bouckenooghe & Asma Zafar & Usman Raja, 2015. "How Ethical Leadership Shapes Employees’ Job Performance: The Mediating Roles of Goal Congruence and Psychological Capital," Journal of Business Ethics, Springer, vol. 129(2), pages 251-264, June.
    3. Esposito, Tonino & Chabot, Martin & Caldwell, Johanna & Webb, Calum & Delaye, Ashleigh & Fluke, John D. & Trocmé, Nico & Bywaters, Paul, 2022. "The differential association of socioeconomic vulnerabilities and neglect-related child protection involvement across geographies: Multilevel structural equation modeling," Children and Youth Services Review, Elsevier, vol. 138(C).
    4. Matthias N. Sweet & Kailey Laidlaw, 2020. "No longer in the driver’s seat: How do affective motivations impact consumer interest in automated vehicles?," Transportation, Springer, vol. 47(5), pages 2601-2634, October.
    5. Francisco Caballero & Marta Miret & Beatriz Olaya & Jaime Perales & Ruy López-Ridaura & Josep Haro & Somnath Chatterji & José Ayuso-Mateos, 2014. "Evaluation of Affect in Mexico and Spain: Psychometric Properties and Usefulness of an Abbreviated Version of the Day Reconstruction Method," Journal of Happiness Studies, Springer, vol. 15(4), pages 915-935, August.
    6. Blanca Mellor-Marsá & Marta Miret & Francisco J. Abad & Somnath Chatterji & Beatriz Olaya & Beata Tobiasz-Adamczyk & Seppo Koskinen & Matilde Leonardi & Josep Maria Haro & José Luis Ayuso-Mateos & Fra, 2016. "Measurement Invariance of the Day Reconstruction Method: Results from the COURAGE in Europe Project," Journal of Happiness Studies, Springer, vol. 17(5), pages 1769-1787, October.
    7. Ferdinando Paolo Santarpia & Laura Borgogni & Chiara Consiglio & Pietro Menatta, 2021. "The Bright and Dark Sides of Resources for Cross-Role Interrupting Behaviors and Work–Family Conflict: Preliminary Multigroup Findings on Remote and Traditional Working," IJERPH, MDPI, vol. 18(22), pages 1-20, November.
    8. An Zhou & Shenhan Wu & Zhujie Chu & Wei-Chiao Huang, 2019. "Regional Differences in Municipal Solid Waste Collection Quantities in China," Sustainability, MDPI, vol. 11(15), pages 1-12, July.
    9. Zaitun Mohd Saman & Ab Hamid Siti-Azrin & Azizah Othman & Yee Cheng Kueh, 2021. "The Validity and Reliability of the Malay Version of the Cyberbullying Scale among Secondary School Adolescents in Malaysia," IJERPH, MDPI, vol. 18(21), pages 1-12, November.
    10. Habtie Alemnew Belay & Fentaye Kasaa Hailu & Gedif Tessema Sinshaw, 2024. "Linking employees' work orientation and corporate social responsibility practices: Multilevel and multistakeholder cross‐sectional study," Corporate Social Responsibility and Environmental Management, John Wiley & Sons, vol. 31(1), pages 676-696, January.
    11. Louise Black & Margarita Panayiotou & Neil Humphrey, 2019. "The dimensionality and latent structure of mental health difficulties and wellbeing in early adolescence," PLOS ONE, Public Library of Science, vol. 14(2), pages 1-24, February.
    12. Jongmoo Jay Choi & Jimi Kim & Oded Shenkar, 2023. "Temporal Orientation and Corporate Social Responsibility: Global Evidence," Journal of Management Studies, Wiley Blackwell, vol. 60(1), pages 82-119, January.
    13. Davaei, Mahboobeh & Gunkel, Marjaana & Veglio, Valerio & Taras, Vas, 2022. "The influence of cultural intelligence and emotional intelligence on conflict occurrence and performance in global virtual teams," Journal of International Management, Elsevier, vol. 28(4).
    14. Imran Shafique & Masood N. Kalyar & Bashir Ahmad, 2018. "The Nexus of Ethical Leadership, Job Performance, and Turnover Intention: The Mediating Role of Job Satisfaction," Interdisciplinary Description of Complex Systems - scientific journal, Croatian Interdisciplinary Society Provider Homepage: http://indecs.eu, vol. 16(1), pages 71-87.
    15. Yu-Jin Chang & Byung-Hee Lee, 2022. "The Impact of ESG Activities on Firm Value: Multi-Level Analysis of Industrial Characteristics," Sustainability, MDPI, vol. 14(21), pages 1-17, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wai-Yin Poon & Hai-Bin Wang, 2010. "Analysis of a Two-Level Structural Equation Model With Missing Data," Sociological Methods & Research, , vol. 39(1), pages 25-55, August.
    2. Nicholas J. Rockwood, 2020. "Maximum Likelihood Estimation of Multilevel Structural Equation Models with Random Slopes for Latent Covariates," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 275-300, June.
    3. Ke-Hai Yuan & Kentaro Hayashi, 2005. "On muthén’s maximum likelihood for two-level covariance structure models," Psychometrika, Springer;The Psychometric Society, vol. 70(1), pages 147-167, March.
    4. Yahia S El-Horbaty & Eman M Hanafy, 2018. "Some Estimation Methods and Their Assessment in Multilevel Models: A Review," Biostatistics and Biometrics Open Access Journal, Juniper Publishers Inc., vol. 5(3), pages 69-76, February.
    5. Piotr Tarka, 2018. "An overview of structural equation modeling: its beginnings, historical development, usefulness and controversies in the social sciences," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(1), pages 313-354, January.
    6. Hirche, Martin & Greenacre, Luke & Nenycz-Thiel, Magda & Loose, Simone & Lockshin, Larry, 2021. "SKU performance and distribution: A large-scale analysis of the role of product characteristics with store scanner data," Journal of Retailing and Consumer Services, Elsevier, vol. 61(C).
    7. Ke-Hai Yuan & Peter Bentler, 2004. "On the asymptotic distributions of two statistics for two-level covariance structure models within the class of elliptical distributions," Psychometrika, Springer;The Psychometric Society, vol. 69(3), pages 437-457, September.
    8. Yuan, Ke-Hai & Bentler, Peter M., 2006. "Asymptotic robustness of standard errors in multilevel structural equation models," Journal of Multivariate Analysis, Elsevier, vol. 97(5), pages 1121-1141, May.
    9. Das, Debojyoti & Bhatia, Vaneet & Kumar, Surya Bhushan & Basu, Sankarshan, 2022. "Do precious metals hedge crude oil volatility jumps?," International Review of Financial Analysis, Elsevier, vol. 83(C).
    10. P.A.V.B. Swamy & I-Lok Chang & Jatinder S. Mehta & William H. Greene & Stephen G. Hall & George S. Tavlas, 2016. "Removing Specification Errors from the Usual Formulation of Binary Choice Models," Econometrics, MDPI, vol. 4(2), pages 1-21, June.
    11. Carlo Altavilla & Raffaella Giacomini & Giuseppe Ragusa, 2017. "Anchoring the yield curve using survey expectations," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(6), pages 1055-1068, September.
    12. Fernando Rios-Avila & Gustavo Canavire-Bacarreza, 2018. "Standard-error correction in two-stage optimization models: A quasi–maximum likelihood estimation approach," Stata Journal, StataCorp LP, vol. 18(1), pages 206-222, March.
    13. Sandy Fréret & Denis Maguain, 2017. "The effects of agglomeration on tax competition: evidence from a two-regime spatial panel model on French data," International Tax and Public Finance, Springer;International Institute of Public Finance, vol. 24(6), pages 1100-1140, December.
    14. Ai, Chunrong & Chen, Xiaohong, 2007. "Estimation of possibly misspecified semiparametric conditional moment restriction models with different conditioning variables," Journal of Econometrics, Elsevier, vol. 141(1), pages 5-43, November.
    15. Ayouz, Mourad K. & Remaud, Herve, 2003. "The Internationalization Determinants Of The Small Agro-Food Firms: Hypotheses And Statistical Tests," International Food and Agribusiness Management Review, International Food and Agribusiness Management Association, vol. 5(2), pages 1-27.
    16. Broze, Laurence & Gourieroux, Christian, 1998. "Pseudo-maximum likelihood method, adjusted pseudo-maximum likelihood method and covariance estimators," Journal of Econometrics, Elsevier, vol. 85(1), pages 75-98, July.
    17. Sridhar, Shrihari & Naik, Prasad A. & Kelkar, Ajay, 2017. "Metrics unreliability and marketing overspending," International Journal of Research in Marketing, Elsevier, vol. 34(4), pages 761-779.
    18. Yen, Steven T. & Chern, Wen S. & Lee, Hwang-Jaw, 1991. "Effects Of Income Sources On Household Food Expenditures," 1991 Annual Meeting, August 4-7, Manhattan, Kansas 271167, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    19. Ruoxuan Xiong & Allison Koenecke & Michael Powell & Zhu Shen & Joshua T. Vogelstein & Susan Athey, 2021. "Federated Causal Inference in Heterogeneous Observational Data," Papers 2107.11732, arXiv.org, revised Apr 2023.
    20. Posch, Olaf, 2009. "Structural estimation of jump-diffusion processes in macroeconomics," Journal of Econometrics, Elsevier, vol. 153(2), pages 196-210, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:stanee:v:64:y:2010:i:2:p:157-170. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0039-0402 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.