IDEAS home Printed from https://ideas.repec.org/a/pal/palcom/v12y2025i1d10.1057_s41599-025-04959-w.html
   My bibliography  Save this article

Two-stage polytomous attribute estimation for cognitive diagnostic models: overcoming computational challenges in large-scale assessments with many polytomous attributes

Author

Listed:
  • Yuting Han

    (Beijing Language and Culture University
    Beijing Language and Culture University
    Beijing Language and Culture University)

  • Feng Ji

    (University of Toronto)

  • Zhehan Jiang

    (Peking University
    Peking University
    Peking University Health Science Center-Chaoxing Joint Laboratory for Digital and Smart Medical Education)

Abstract

Cognitive diagnosis models (CDMs) have been advocated as a useful tool in calibrating large-scale assessments, yet the computational challenges are inevitably amplified when the modeling complexity (e.g., the number and the levels of attributes) increases. This study presents a critical scenario, a large-scale national medical certification exam, where CDM with many polytomous attributes (mpCDM) is of great utility, but poses great computational challenges to many popular open-source CDM software packages. We developed a novel two-stage estimation method and assessed its performance through a Monte Carlo simulation study under various conditions of attribute number, item number, item quality, and sample size. Results indicate that the proposed method maintains high accuracy in handling large-scale data while effectively overcoming computational capacity limitations, especially in scenarios with many polytomous attributes, large numbers of items, and substantial sample sizes. Furthermore, we applied the proposed method to a large-scale health examination dataset, demonstrating its effectiveness in practice. This study contributes to the field of psychometrics by offering a simple yet effective solution to the computational challenges inherent in implementing mpCDMs for large-scale assessments, providing a practical tool for diagnostic analyses in educational and professional certification contexts.

Suggested Citation

  • Yuting Han & Feng Ji & Zhehan Jiang, 2025. "Two-stage polytomous attribute estimation for cognitive diagnostic models: overcoming computational challenges in large-scale assessments with many polytomous attributes," Palgrave Communications, Palgrave Macmillan, vol. 12(1), pages 1-14, December.
    • Handle: RePEc:pal:palcom:v:12:y:2025:i:1:d:10.1057_s41599-025-04959-w
    DOI: 10.1057/s41599-025-04959-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1057/s41599-025-04959-w
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1057/s41599-025-04959-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jimmy Torre & Jeffrey Douglas, 2004. "Higher-order latent trait models for cognitive diagnosis," Psychometrika, Springer;The Psychometric Society, vol. 69(3), pages 333-353, September.
    2. Peida Zhan & Wen-Chung Wang & Xiaomin Li, 2020. "A Partial Mastery, Higher-Order Latent Structural Model for Polytomous Attributes in Cognitive Diagnostic Assessments," Journal of Classification, Springer;The Classification Society, vol. 37(2), pages 328-351, July.
    3. Jimmy de la Torre, 2011. "The Generalized DINA Model Framework," Psychometrika, Springer;The Psychometric Society, vol. 76(2), pages 179-199, April.
    4. Shiyu Wang & Yinghan Chen, 2020. "Using Response Times and Response Accuracy to Measure Fluency Within Cognitive Diagnosis Models," Psychometrika, Springer;The Psychometric Society, vol. 85(3), pages 600-629, September.
    5. Jimmy Torre, 2011. "Erratum to: The Generalized DINA Model Framework," Psychometrika, Springer;The Psychometric Society, vol. 76(3), pages 510-510, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Youn Seon Lim, 2024. "Nonparametric Cognitive Diagnosis When Attributes Are Polytomous," Journal of Classification, Springer;The Classification Society, vol. 41(1), pages 94-128, March.
    2. Peida Zhan & Xin Qiao, 2022. "DIAGNOSTIC Classification Analysis of Problem-Solving Competence using Process Data: An Item Expansion Method," Psychometrika, Springer;The Psychometric Society, vol. 87(4), pages 1529-1547, December.
    3. Matthew S. Johnson & Sandip Sinharay, 2020. "The Reliability of the Posterior Probability of Skill Attainment in Diagnostic Classification Models," Journal of Educational and Behavioral Statistics, , vol. 45(1), pages 5-31, February.
    4. Xin Xu & Guanhua Fang & Jinxin Guo & Zhiliang Ying & Susu Zhang, 2024. "Diagnostic Classification Models for Testlets: Methods and Theory," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 851-876, September.
    5. Peida Zhan & Hong Jiao & Dandan Liao & Feiming Li, 2019. "A Longitudinal Higher-Order Diagnostic Classification Model," Journal of Educational and Behavioral Statistics, , vol. 44(3), pages 251-281, June.
    6. Chen-Wei Liu & Björn Andersson & Anders Skrondal, 2020. "A Constrained Metropolis–Hastings Robbins–Monro Algorithm for Q Matrix Estimation in DINA Models," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 322-357, June.
    7. Hans Friedrich Köhn & Chia-Yi Chiu, 2021. "A Unified Theory of the Completeness of Q-Matrices for the DINA Model," Journal of Classification, Springer;The Classification Society, vol. 38(3), pages 500-518, October.
    8. Jimmy de la Torre & Xue-Lan Qiu & Kevin Carl Santos, 2022. "An Empirical Q-Matrix Validation Method for the Polytomous G-DINA Model," Psychometrika, Springer;The Psychometric Society, vol. 87(2), pages 693-724, June.
    9. Motonori Oka & Kensuke Okada, 2023. "Scalable Bayesian Approach for the Dina Q-Matrix Estimation Combining Stochastic Optimization and Variational Inference," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 302-331, March.
    10. Qianru Liang & Jimmy de la Torre & Nancy Law, 2023. "Latent Transition Cognitive Diagnosis Model With Covariates: A Three-Step Approach," Journal of Educational and Behavioral Statistics, , vol. 48(6), pages 690-718, December.
    11. Junhuan Wei & Liufen Luo & Yan Cai & Dongbo Tu, 2024. "A Multistrategy Cognitive Diagnosis Model Incorporating Item Response Times Based on Strategy Selection Theories," Journal of Educational and Behavioral Statistics, , vol. 49(4), pages 658-686, August.
    12. James Joseph Balamuta & Steven Andrew Culpepper, 2022. "Exploratory Restricted Latent Class Models with Monotonicity Requirements under PÒLYA–GAMMA Data Augmentation," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 903-945, September.
    13. Kazuhiro Yamaguchi & Kensuke Okada, 2020. "Variational Bayes Inference for the DINA Model," Journal of Educational and Behavioral Statistics, , vol. 45(5), pages 569-597, October.
    14. Peida Zhan & Wen-Chung Wang & Xiaomin Li, 2020. "A Partial Mastery, Higher-Order Latent Structural Model for Polytomous Attributes in Cognitive Diagnostic Assessments," Journal of Classification, Springer;The Classification Society, vol. 37(2), pages 328-351, July.
    15. Yuqi Gu, 2024. "Going Deep in Diagnostic Modeling: Deep Cognitive Diagnostic Models (DeepCDMs)," Psychometrika, Springer;The Psychometric Society, vol. 89(1), pages 118-150, March.
    16. Chun Wang & Jing Lu, 2021. "Learning Attribute Hierarchies From Data: Two Exploratory Approaches," Journal of Educational and Behavioral Statistics, , vol. 46(1), pages 58-84, February.
    17. Xuliang Gao & Wenchao Ma & Daxun Wang & Yan Cai & Dongbo Tu, 2021. "A Class of Cognitive Diagnosis Models for Polytomous Data," Journal of Educational and Behavioral Statistics, , vol. 46(3), pages 297-322, June.
    18. Wenyi Wang & Lihong Song & Teng Wang & Peng Gao & Jian Xiong, 2020. "A Note on the Relationship of the Shannon Entropy Procedure and the Jensen–Shannon Divergence in Cognitive Diagnostic Computerized Adaptive Testing," SAGE Open, , vol. 10(1), pages 21582440198, January.
    19. Kazuhiro Yamaguchi & Jonathan Templin, 2022. "Direct Estimation of Diagnostic Classification Model Attribute Mastery Profiles via a Collapsed Gibbs Sampling Algorithm," Psychometrika, Springer;The Psychometric Society, vol. 87(4), pages 1390-1421, December.
    20. Ragip Terzi & Sedat Sen, 2019. "A Nondiagnostic Assessment for Diagnostic Purposes: Q-Matrix Validation and Item-Based Model Fit Evaluation for the TIMSS 2011 Assessment," SAGE Open, , vol. 9(1), pages 21582440198, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pal:palcom:v:12:y:2025:i:1:d:10.1057_s41599-025-04959-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: https://www.nature.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.