IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v50y2025i1p5-43.html

Modeling Partial Knowledge in Multiple-Choice Cognitive Diagnostic Assessment

Author

Listed:
  • Kentaro Fukushima
  • Nao Uchida

    (The University of Tokyo
    Japan Society for the Promotion of Science)

  • Kensuke Okada

    (The University of Tokyo)

Abstract

Diagnostic tests are typically administered in a multiple-choice (MC) format due to their advantages of objectivity and time efficiency. The MC-deterministic input, noisy “and†gate (DINA) family of models, a representative class of cognitive diagnostic models for MC items, efficiently and parsimoniously estimates the mastery profiles of examinees. However, the existing models often overestimate the latent traits of examinees when they respond with partial knowledge, which is often observed in educational assessment. Therefore, the novel models of the MC-DINA family that can appropriately handle such responses were developed in this study. Unlike the existing models, the proposed models placed no restrictions on the Q-vector, which represents attribute specifications. Simulation and empirical studies verified that the proposed approach could resolve the overestimation problem.

Suggested Citation

  • Kentaro Fukushima & Nao Uchida & Kensuke Okada, 2025. "Modeling Partial Knowledge in Multiple-Choice Cognitive Diagnostic Assessment," Journal of Educational and Behavioral Statistics, , vol. 50(1), pages 5-43, February.
    • Handle: RePEc:sae:jedbes:v:50:y:2025:i:1:p:5-43
    DOI: 10.3102/10769986241245707
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/10769986241245707
    Download Restriction: no
    File URL: https://libkey.io/10.3102/10769986241245707?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Hans-Friedrich Köhn & Chia-Yi Chiu, 2017. "A Procedure for Assessing the Completeness of the Q-Matrices of Cognitively Diagnostic Tests," Psychometrika, Springer;The Psychometric Society, vol. 82(1), pages 112-132, March.
    2. Tsai, Tsung-han & Lin, Chang-chih, 2017. "Modeling Guessing Components in the Measurement of Political Knowledge," Political Analysis, Cambridge University Press, vol. 25(4), pages 483-504, October.
    3. Laine Bradshaw & Jonathan Templin, 2014. "Combining Item Response Theory and Diagnostic Classification Models: A Psychometric Model for Scaling Ability and Diagnosing Misconceptions," Psychometrika, Springer;The Psychometric Society, vol. 79(3), pages 403-425, July.
    4. Kazuhiro Yamaguchi & Kensuke Okada, 2020. "Variational Bayes Inference for the DINA Model," Journal of Educational and Behavioral Statistics, , vol. 45(5), pages 569-597, October.
    5. Peida Zhan & Wen-Chung Wang & Xiaomin Li, 2020. "A Partial Mastery, Higher-Order Latent Structural Model for Polytomous Attributes in Cognitive Diagnostic Assessments," Journal of Classification, Springer;The Classification Society, vol. 37(2), pages 328-351, July.
    6. Peida Zhan & Hong Jiao & Kaiwen Man & Lijun Wang, 2019. "Using JAGS for Bayesian Cognitive Diagnosis Modeling: A Tutorial," Journal of Educational and Behavioral Statistics, , vol. 44(4), pages 473-503, August.
    7. Susan Embretson (Whitely), 1984. "A general latent trait model for response processes," Psychometrika, Springer;The Psychometric Society, vol. 49(2), pages 175-186, June.
    8. Jinsong Chen & Hui Zhou, 2017. "Test designs and modeling under the general nominal diagnosis model framework," PLOS ONE, Public Library of Science, vol. 12(6), pages 1-14, June.
    9. Jimmy de la Torre, 2011. "The Generalized DINA Model Framework," Psychometrika, Springer;The Psychometric Society, vol. 76(2), pages 179-199, April.
    10. Qian Wu & Monique Vanerum & Anouk Agten & Andrés Christiansen & Frank Vandenabeele & Jean-Michel Rigo & Rianne Janssen, 2021. "Certainty-Based Marking on Multiple-Choice Items: Psychometrics Meets Decision Theory," Psychometrika, Springer;The Psychometric Society, vol. 86(2), pages 518-543, June.
    11. Matthew J. Madison & Laine P. Bradshaw, 2018. "Assessing Growth in a Diagnostic Classification Model Framework," Psychometrika, Springer;The Psychometric Society, vol. 83(4), pages 963-990, December.
    12. Jimmy Torre, 2011. "Erratum to: The Generalized DINA Model Framework," Psychometrika, Springer;The Psychometric Society, vol. 76(3), pages 510-510, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kazuhiro Yamaguchi, 2025. "Bayesian Diagnostic Classification Models for a Partially Known Q-Matrix," Journal of Educational and Behavioral Statistics, , vol. 50(2), pages 331-382, April.
    2. Kazuhiro Yamaguchi & Kensuke Okada, 2020. "Variational Bayes Inference Algorithm for the Saturated Diagnostic Classification Model," Psychometrika, Springer;The Psychometric Society, vol. 85(4), pages 973-995, December.
    3. Cheng-Hsuan Li & Yi-Jin Ju & Pei-Jyun Hsieh, 2022. "A Nonparametric Weighted Cognitive Diagnosis Model and Its Application on Remedial Instruction in a Small-Class Situation," Sustainability, MDPI, vol. 14(10), pages 1-17, May.
    4. Kazuhiro Yamaguchi & Jonathan Templin, 2022. "Direct Estimation of Diagnostic Classification Model Attribute Mastery Profiles via a Collapsed Gibbs Sampling Algorithm," Psychometrika, Springer;The Psychometric Society, vol. 87(4), pages 1390-1421, December.
    5. Kazuhiro Yamaguchi, 2023. "Bayesian Analysis Methods for Two-Level Diagnosis Classification Models," Journal of Educational and Behavioral Statistics, , vol. 48(6), pages 773-809, December.
    6. Ying Liu & Steven Andrew Culpepper, 2024. "Restricted Latent Class Models for Nominal Response Data: Identifiability and Estimation," Psychometrika, Springer;The Psychometric Society, vol. 89(2), pages 592-625, June.
    7. Kazuhiro Yamaguchi & Jonathan Templin, 2022. "A Gibbs Sampling Algorithm with Monotonicity Constraints for Diagnostic Classification Models," Journal of Classification, Springer;The Classification Society, vol. 39(1), pages 24-54, March.
    8. Ling Ling Wang & Tao Xin & Xiao Jian Sun, 2026. "Using the Bayesian Network’s Structural Learning Algorithm to Estimate the Q-Matrix in Cognitive Diagnosis Models," Journal of Educational and Behavioral Statistics, , vol. 51(2), pages 419-450, April.
    9. Peida Zhan* & Kaiwen Man* & Stefanie A. Wind & Jonathan Malone, 2022. "Cognitive Diagnosis Modeling Incorporating Response Times and Fixation Counts: Providing Comprehensive Feedback and Accurate Diagnosis," Journal of Educational and Behavioral Statistics, , vol. 47(6), pages 736-776, December.
    10. Youn Seon Lim, 2024. "Nonparametric Cognitive Diagnosis When Attributes Are Polytomous," Journal of Classification, Springer;The Classification Society, vol. 41(1), pages 94-128, March.
    11. Motonori Oka & Kensuke Okada, 2023. "Scalable Bayesian Approach for the Dina Q-Matrix Estimation Combining Stochastic Optimization and Variational Inference," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 302-331, March.
    12. Yuqi Gu, 2023. "Generic Identifiability of the DINA Model and Blessing of Latent Dependence," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 117-131, March.
    13. Qianru Liang & Jimmy de la Torre & Nancy Law, 2023. "Latent Transition Cognitive Diagnosis Model With Covariates: A Three-Step Approach," Journal of Educational and Behavioral Statistics, , vol. 48(6), pages 690-718, December.
    14. Chun Wang, 2021. "Using Penalized EM Algorithm to Infer Learning Trajectories in Latent Transition CDM," Psychometrika, Springer;The Psychometric Society, vol. 86(1), pages 167-189, March.
    15. Yuting Han & Feng Ji & Zhehan Jiang, 2025. "Two-stage polytomous attribute estimation for cognitive diagnostic models: overcoming computational challenges in large-scale assessments with many polytomous attributes," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 12(1), pages 1-14, December.
    16. Yinghan Chen & Ying Liu & Steven Andrew Culpepper & Yuguo Chen, 2021. "Inferring the Number of Attributes for the Exploratory DINA Model," Psychometrika, Springer;The Psychometric Society, vol. 86(1), pages 30-64, March.
    17. Steven Andrew Culpepper, 2023. "A Note on Weaker Conditions for Identifying Restricted Latent Class Models for Binary Responses," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 158-174, March.
    18. Peida Zhan & Hong Jiao & Kaiwen Man & Lijun Wang, 2019. "Using JAGS for Bayesian Cognitive Diagnosis Modeling: A Tutorial," Journal of Educational and Behavioral Statistics, , vol. 44(4), pages 473-503, August.
    19. Chun Wang, 2024. "A Diagnostic Facet Status Model (DFSM) for Extracting Instructionally Useful Information from Diagnostic Assessment," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 747-773, September.
    20. Junhuan Wei & Liufen Luo & Yan Cai & Dongbo Tu, 2024. "A Multistrategy Cognitive Diagnosis Model Incorporating Item Response Times Based on Strategy Selection Theories," Journal of Educational and Behavioral Statistics, , vol. 49(4), pages 658-686, August.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:50:y:2025:i:1:p:5-43. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.