IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v39y2014i4p235-256.html
   My bibliography  Save this article

A Third-Order Item Response Theory Model for Modeling the Effects of Domains and Subdomains in Large-Scale Educational Assessment Surveys

Author

Listed:
  • Frank Rijmen

    (CTB/McGraw-Hill)

  • Minjeong Jeon

    (Ohio State University)

  • Matthias von Davier

    (Educational Testing Service)

  • Sophia Rabe-Hesketh

    (University of California, Berkeley)

Abstract

Second-order item response theory models have been used for assessments consisting of several domains, such as content areas. We extend the second-order model to a third-order model for assessments that include subdomains nested in domains. Using a graphical model framework, it is shown how the model does not suffer from the curse of multidimensionality. We apply unidimensional, second-order, and third-order item response models to the 2007 Trends in International Mathematics and Science Study. Our findings suggest that deviations from unidimensionality are more pronounced at the content domain level than at the cognitive domain level and that deviations from unidimensionality at the content domain level become negligible after taking into account topic areas.

Suggested Citation

  • Frank Rijmen & Minjeong Jeon & Matthias von Davier & Sophia Rabe-Hesketh, 2014. "A Third-Order Item Response Theory Model for Modeling the Effects of Domains and Subdomains in Large-Scale Educational Assessment Surveys," Journal of Educational and Behavioral Statistics, , vol. 39(4), pages 235-256, August.
    • Handle: RePEc:sae:jedbes:v:39:y:2014:i:4:p:235-256
    DOI: 10.3102/1076998614531045
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998614531045
    Download Restriction: no
    File URL: https://libkey.io/10.3102/1076998614531045?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Yiu-Fai Yung & David Thissen & Lori McLeod, 1999. "On the relationship between the higher-order factor model and the hierarchical factor model," Psychometrika, Springer;The Psychometric Society, vol. 64(2), pages 113-128, June.
    2. Harvey Goldstein & Jon Rasbash, 1996. "Improved Approximations for Multilevel Models with Binary Responses," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 159(3), pages 505-513, May.
    3. Geoff Masters, 1982. "A rasch model for partial credit scoring," Psychometrika, Springer;The Psychometric Society, vol. 47(2), pages 149-174, June.
    4. R. Bock & Murray Aitkin, 1981. "Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm," Psychometrika, Springer;The Psychometric Society, vol. 46(4), pages 443-459, December.
    5. Lauritzen, Steffen L., 1995. "The EM algorithm for graphical association models with missing data," Computational Statistics & Data Analysis, Elsevier, vol. 19(2), pages 191-201, February.
    6. Eric Bradlow & Howard Wainer & Xiaohui Wang, 1999. "A Bayesian random effects model for testlets," Psychometrika, Springer;The Psychometric Society, vol. 64(2), pages 153-168, June.
    7. Robert Gibbons & Donald Hedeker, 1992. "Full-information item bi-factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 57(3), pages 423-436, September.
    8. Rabe-Hesketh, Sophia & Skrondal, Anders & Pickles, Andrew, 2005. "Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects," Journal of Econometrics, Elsevier, vol. 128(2), pages 301-323, October.
    9. Frank Rijmen & Kristof Vansteelandt & Paul Boeck, 2008. "Latent Class Models for Diary Method Data: Parameter Estimation by Local Computations," Psychometrika, Springer;The Psychometric Society, vol. 73(2), pages 167-182, June.
    10. John Schmid & John Leiman, 1957. "The development of hierarchical factor solutions," Psychometrika, Springer;The Psychometric Society, vol. 22(1), pages 53-61, March.
    11. Shelby Haberman & Sandip Sinharay, 2010. "Reporting of Subscores Using Multidimensional Item Response Theory," Psychometrika, Springer;The Psychometric Society, vol. 75(2), pages 209-227, June.
    12. Jean-Paul Fox & Cees Glas, 2001. "Bayesian estimation of a multilevel IRT model using gibbs sampling," Psychometrika, Springer;The Psychometric Society, vol. 66(2), pages 271-288, June.
    13. Minjeong Jeon & Frank Rijmen & Sophia Rabe-Hesketh, 2013. "Modeling Differential Item Functioning Using a Generalization of the Multiple-Group Bifactor Model," Journal of Educational and Behavioral Statistics, , vol. 38(1), pages 32-60, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Richard A. Feinberg & Matthias von Davier, 2020. "Conditional Subscore Reporting Using Iterated Discrete Convolutions," Journal of Educational and Behavioral Statistics, , vol. 45(5), pages 515-533, October.
    2. Sayed H. Kadhem & Aristidis K. Nikoloulopoulos, 2023. "Bi-factor and Second-Order Copula Models for Item Response Data," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 132-157, March.
    3. Minjeong Jeon & Frank Rijmen & Sophia Rabe-Hesketh, 2017. "A Variational Maximization–Maximization Algorithm for Generalized Linear Mixed Models with Crossed Random Effects," Psychometrika, Springer;The Psychometric Society, vol. 82(3), pages 693-716, September.
    4. Minjeong Jeon & Frank Rijmen & Sophia Rabe-Hesketh, 2018. "CFA Models with a General Factor and Multiple Sets of Secondary Factors," Psychometrika, Springer;The Psychometric Society, vol. 83(4), pages 785-808, December.
    5. Simon Grund & Oliver Lüdtke & Alexander Robitzsch, 2021. "On the Treatment of Missing Data in Background Questionnaires in Educational Large-Scale Assessments: An Evaluation of Different Procedures," Journal of Educational and Behavioral Statistics, , vol. 46(4), pages 430-465, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Minjeong Jeon & Frank Rijmen & Sophia Rabe-Hesketh, 2013. "Modeling Differential Item Functioning Using a Generalization of the Multiple-Group Bifactor Model," Journal of Educational and Behavioral Statistics, , vol. 38(1), pages 32-60, February.
    2. Li Cai, 2015. "Lord–Wingersky Algorithm Version 2.0 for Hierarchical Item Factor Models with Applications in Test Scoring, Scale Alignment, and Model Fit Testing," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 535-559, June.
    3. Michael Edwards, 2010. "A Markov Chain Monte Carlo Approach to Confirmatory Item Factor Analysis," Psychometrika, Springer;The Psychometric Society, vol. 75(3), pages 474-497, September.
    4. Li Cai, 2010. "A Two-Tier Full-Information Item Factor Analysis Model with Applications," Psychometrika, Springer;The Psychometric Society, vol. 75(4), pages 581-612, December.
    5. Minjeong Jeon & Sophia Rabe-Hesketh, 2016. "An autoregressive growth model for longitudinal item analysis," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 830-850, September.
    6. Jean-Paul Fox & Cees Glas, 2001. "Bayesian estimation of a multilevel IRT model using gibbs sampling," Psychometrika, Springer;The Psychometric Society, vol. 66(2), pages 271-288, June.
    7. Chun Wang & Steven W. Nydick, 2020. "On Longitudinal Item Response Theory Models: A Didactic," Journal of Educational and Behavioral Statistics, , vol. 45(3), pages 339-368, June.
    8. C. Glas & Anna Dagohoy, 2007. "A Person Fit Test For Irt Models For Polytomous Items," Psychometrika, Springer;The Psychometric Society, vol. 72(2), pages 159-180, June.
    9. Sun-Joo Cho & Paul Boeck & Susan Embretson & Sophia Rabe-Hesketh, 2014. "Additive Multilevel Item Structure Models with Random Residuals: Item Modeling for Explanation and Item Generation," Psychometrika, Springer;The Psychometric Society, vol. 79(1), pages 84-104, January.
    10. Gregory Camilli & Jean-Paul Fox, 2015. "An Aggregate IRT Procedure for Exploratory Factor Analysis," Journal of Educational and Behavioral Statistics, , vol. 40(4), pages 377-401, August.
    11. Peida Zhan & Hong Jiao & Dandan Liao & Feiming Li, 2019. "A Longitudinal Higher-Order Diagnostic Classification Model," Journal of Educational and Behavioral Statistics, , vol. 44(3), pages 251-281, June.
    12. Minjeong Jeon & Sophia Rabe-Hesketh, 2012. "Profile-Likelihood Approach for Estimating Generalized Linear Mixed Models With Factor Structures," Journal of Educational and Behavioral Statistics, , vol. 37(4), pages 518-542, August.
    13. Heleno Bolfarine & Jorge Luis Bazan, 2010. "Bayesian Estimation of the Logistic Positive Exponent IRT Model," Journal of Educational and Behavioral Statistics, , vol. 35(6), pages 693-713, December.
    14. Singh, Jagdip, 2004. "Tackling measurement problems with Item Response Theory: Principles, characteristics, and assessment, with an illustrative example," Journal of Business Research, Elsevier, vol. 57(2), pages 184-208, February.
    15. Minjeong Jeon & Frank Rijmen & Sophia Rabe-Hesketh, 2018. "CFA Models with a General Factor and Multiple Sets of Secondary Factors," Psychometrika, Springer;The Psychometric Society, vol. 83(4), pages 785-808, December.
    16. Minjeong Jeon & Frank Rijmen & Sophia Rabe-Hesketh, 2017. "A Variational Maximization–Maximization Algorithm for Generalized Linear Mixed Models with Crossed Random Effects," Psychometrika, Springer;The Psychometric Society, vol. 82(3), pages 693-716, September.
    17. Anders Skrondal & Sophia Rabe‐Hesketh, 2007. "Latent Variable Modelling: A Survey," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 34(4), pages 712-745, December.
    18. repec:jss:jstsof:34:i03 is not listed on IDEAS
    19. Cho, S.-J. & Rabe-Hesketh, S., 2011. "Alternating imputation posterior estimation of models with crossed random effects," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 12-25, January.
    20. Javier Revuelta, 2008. "The generalized Logit-Linear Item Response Model for Binary-Designed Items," Psychometrika, Springer;The Psychometric Society, vol. 73(3), pages 385-405, September.
    21. Li C. Liu & Donald Hedeker, 2006. "A Mixed-Effects Regression Model for Longitudinal Multivariate Ordinal Data," Biometrics, The International Biometric Society, vol. 62(1), pages 261-268, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:39:y:2014:i:4:p:235-256. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.