IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v13y2025i6p1001-d1615948.html
   My bibliography  Save this article

Structural Equation Modeling Approaches to Estimating Score Dependability Within Generalizability Theory-Based Univariate, Multivariate, and Bifactor Designs

Author

Listed:
  • Walter P. Vispoel

    (Department of Psychological and Quantitative Foundations, University of Iowa, Iowa City, IA 52242, USA)

  • Hyeryung Lee

    (Department of Psychological and Quantitative Foundations, University of Iowa, Iowa City, IA 52242, USA)

  • Tingting Chen

    (Department of Psychological and Quantitative Foundations, University of Iowa, Iowa City, IA 52242, USA)

Abstract

Generalizability theory (GT) provides an all-encompassing framework for estimating accuracy of scores and effects of multiple sources of measurement error when using measures intended for either norm- or criterion-referencing purposes. Structural equation models (SEMs) can replicate results from GT-based ANOVA procedures while extending those analyses to account for scale coarseness, generate Monte Carlo-based confidence intervals for key parameters, partition universe score variance into general and group factor effects, and assess subscale score viability. We apply these techniques in R to univariate, multivariate, and bifactor designs using a novel indicator-mean approach to estimate absolute error. When representing responses to items from the shortened form of the Music Self-Perception Inventory (MUSPI-S) using 2-, 4-, and 8-point response metrics over two occasions, SEMs reproduced results from the ANOVA-based mGENOVA package for univariate and multivariate designs with score accuracy and subscale viability indices within bifactor designs comparable to those from corresponding multivariate designs. Adjusting for scale coarseness improved the accuracy of scores across all response metrics, with dichotomous observed scores least approximating truly continuous scales. Although general-factor effects were dominant, subscale viability was supported in all cases, with transient measurement error leading to the greatest reductions in score accuracy. Key implications are discussed.

Suggested Citation

  • Walter P. Vispoel & Hyeryung Lee & Tingting Chen, 2025. "Structural Equation Modeling Approaches to Estimating Score Dependability Within Generalizability Theory-Based Univariate, Multivariate, and Bifactor Designs," Mathematics, MDPI, vol. 13(6), pages 1-49, March.
    • Handle: RePEc:gam:jmathe:v:13:y:2025:i:6:p:1001-:d:1615948
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/13/6/1001/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/13/6/1001/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Goldine Gleser & Lee Cronbach & Nageswari Rajaratnam, 1965. "Generalizability of scores influenced by multiple sources of variance," Psychometrika, Springer;The Psychometric Society, vol. 30(4), pages 395-418, December.
    2. Karl Holzinger & Harry Harman, 1938. "Comparison of two factorial analyses," Psychometrika, Springer;The Psychometric Society, vol. 3(1), pages 45-60, March.
    3. Shelby Haberman & Sandip Sinharay, 2010. "Reporting of Subscores Using Multidimensional Item Response Theory," Psychometrika, Springer;The Psychometric Society, vol. 75(2), pages 209-227, June.
    4. Honaker, James & King, Gary & Blackwell, Matthew, 2011. "Amelia II: A Program for Missing Data," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 45(i07).
    5. Walter Peter Vispoel & Hyeryung Lee & Tingting Chen, 2024. "Multivariate Structural Equation Modeling Techniques for Estimating Reliability, Measurement Error, and Subscale Viability When Using Both Composite and Subscale Scores in Practice," Mathematics, MDPI, vol. 12(8), pages 1-25, April.
    6. John Schmid & John Leiman, 1957. "The development of hierarchical factor solutions," Psychometrika, Springer;The Psychometric Society, vol. 22(1), pages 53-61, March.
    7. Rosseel, Yves, 2012. "lavaan: An R Package for Structural Equation Modeling," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i02).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Walter Peter Vispoel & Hyeryung Lee & Tingting Chen, 2024. "Multivariate Structural Equation Modeling Techniques for Estimating Reliability, Measurement Error, and Subscale Viability When Using Both Composite and Subscale Scores in Practice," Mathematics, MDPI, vol. 12(8), pages 1-25, April.
    2. Kan, Kees-Jan & van der Maas, Han L.J. & Levine, Stephen Z., 2019. "Extending psychometric network analysis: Empirical evidence against g in favor of mutualism?," Intelligence, Elsevier, vol. 73(C), pages 52-62.
    3. Shenghai Dai & Dubravka Svetina & Xiaolin Wang, 2017. "Reporting Subscores Using R: A Software Review," Journal of Educational and Behavioral Statistics, , vol. 42(5), pages 617-638, October.
    4. Frank Rijmen & Minjeong Jeon & Matthias von Davier & Sophia Rabe-Hesketh, 2014. "A Third-Order Item Response Theory Model for Modeling the Effects of Domains and Subdomains in Large-Scale Educational Assessment Surveys," Journal of Educational and Behavioral Statistics, , vol. 39(4), pages 235-256, August.
    5. Werber Borut & Baggia Alenka & Žnidaršič Anja, 2018. "Factors Affecting the Intentions to Use RFID Subcutaneous Microchip Implants for Healthcare Purposes," Organizacija, Sciendo, vol. 51(2), pages 121-133, May.
    6. Hartung, Johanna & Doebler, Philipp & Schroeders, Ulrich & Wilhelm, Oliver, 2018. "Dedifferentiation and differentiation of intelligence in adults across age and years of education," Intelligence, Elsevier, vol. 69(C), pages 37-49.
    7. Gignac, Gilles E. & Kretzschmar, André, 2017. "Evaluating dimensional distinctness with correlated-factor models: Limitations and suggestions," Intelligence, Elsevier, vol. 62(C), pages 138-147.
    8. Elliott S. Lee & Myles A. Maillet & Frederick M. E. Grouzet, 2022. "Why Do Individuals Engage with the Natural World? A Self-Determination Theory Perspective on the Effect of Nature Engagement and Well-Being," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 17(3), pages 1501-1532, June.
    9. Md. Mominur Rahman & Bilkis Akhter, 2021. "The impact of investment in human capital on bank performance: evidence from Bangladesh," Future Business Journal, Springer, vol. 7(1), pages 1-13, December.
    10. César Merino-Soto & Gina Chávez-Ventura & Verónica López-Fernández & Guillermo M. Chans & Filiberto Toledano-Toledano, 2022. "Learning Self-Regulation Questionnaire (SRQ-L): Psychometric and Measurement Invariance Evidence in Peruvian Undergraduate Students," Sustainability, MDPI, vol. 14(18), pages 1-17, September.
    11. Nathaniel Oliver Iotti & Damiano Menin & Tomas Jungert, 2022. "Early Adolescents’ Motivations to Defend Victims of Cyberbullying," IJERPH, MDPI, vol. 19(14), pages 1-9, July.
    12. Robert A. Jackson & Matthew Pietryka, 2022. "The influence of becoming a parent on political participation in the United States," Social Science Quarterly, Southwestern Social Science Association, vol. 103(3), pages 565-580, May.
    13. Christoph Dworschak, 2024. "Bias mitigation in empirical peace and conflict studies: A short primer on posttreatment variables," Journal of Peace Research, Peace Research Institute Oslo, vol. 61(3), pages 462-476, May.
    14. Andreea-Ionela Puiu & Anca Monica Ardeleanu & Camelia Cojocaru & Anca Bratu, 2021. "Exploring the Effect of Status Quo, Innovativeness, and Involvement Tendencies on Luxury Fashion Innovations: The Mediation Role of Status Consumption," Mathematics, MDPI, vol. 9(9), pages 1-18, May.
    15. Slupphaug, KJell & Mehmetoglu, Mehmet & Mittner, Matthias, 2024. "modsem: An R package for estimating latent interactions and quadratic effects," OSF Preprints h3rpw, Center for Open Science.
    16. Cohen, Joseph N, 2010. "Neoliberalism’s relationship with economic growth in the developing world: Was it the power of the market or the resolution of financial crisis?," MPRA Paper 24527, University Library of Munich, Germany.
    17. Allen, Jaime & Muñoz, Juan Carlos & Ortúzar, Juan de Dios, 2019. "On evasion behaviour in public transport: Dissatisfaction or contagion?," Transportation Research Part A: Policy and Practice, Elsevier, vol. 130(C), pages 626-651.
    18. Merkle, Edgar C. & Steyvers, Mark & Mellers, Barbara & Tetlock, Philip E., 2017. "A neglected dimension of good forecasting judgment: The questions we choose also matter," International Journal of Forecasting, Elsevier, vol. 33(4), pages 817-832.
    19. Sai-fu Fung & Esther Oi-wah Chow & Chau-kiu Cheung, 2020. "Development and Evaluation of the Psychometric Properties of a Brief Wisdom Development Scale," IJERPH, MDPI, vol. 17(8), pages 1-14, April.
    20. Zou, Yuxuan & Wang, Donggen, 2025. "Differences in the influence of the built environment and physical activity on obesity in urban and suburban contexts," Social Science & Medicine, Elsevier, vol. 372(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:13:y:2025:i:6:p:1001-:d:1615948. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.