IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v47y2022i6p736-776.html

Cognitive Diagnosis Modeling Incorporating Response Times and Fixation Counts: Providing Comprehensive Feedback and Accurate Diagnosis

Author

Listed:
  • Peida Zhan*

    (Zhejiang Normal University)

  • Kaiwen Man*
  • Stefanie A. Wind

    (University of Alabama)

  • Jonathan Malone

    (University of Maryland)

Abstract

Respondents’ problem-solving behaviors comprise behaviors that represent complicated cognitive processes that are frequently systematically tied to one another. Biometric data, such as visual fixation counts (FCs), which are an important eye-tracking indicator, can be combined with other types of variables that reflect different aspects of problem-solving behavior to quantify variability in problem-solving behavior. To provide comprehensive feedback and accurate diagnosis when using such multimodal data, the present study proposes a multimodal joint cognitive diagnosis model that accounts for latent attributes, latent ability, processing speed, and visual engagement by simultaneously modeling response accuracy (RA), response times, and FCs. We used two simulation studies to test the feasibility of the proposed model. Findings mainly suggest that the parameters of the proposed model can be well recovered and that modeling FCs, in addition to RA and response times, could increase the comprehensiveness of feedback on problem-solving-related cognitive characteristics as well as the accuracy of knowledge structure diagnosis. An empirical example is used to demonstrate the applicability and benefits of the proposed model. We discuss the implications of our findings as they relate to research and practice.

Suggested Citation

  • Peida Zhan* & Kaiwen Man* & Stefanie A. Wind & Jonathan Malone, 2022. "Cognitive Diagnosis Modeling Incorporating Response Times and Fixation Counts: Providing Comprehensive Feedback and Accurate Diagnosis," Journal of Educational and Behavioral Statistics, , vol. 47(6), pages 736-776, December.
    • Handle: RePEc:sae:jedbes:v:47:y:2022:i:6:p:736-776
    DOI: 10.3102/10769986221111085
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/10769986221111085
    Download Restriction: no
    File URL: https://libkey.io/10.3102/10769986221111085?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Wim van der Linden, 2007. "A Hierarchical Framework for Modeling Speed and Accuracy on Test Items," Psychometrika, Springer;The Psychometric Society, vol. 72(3), pages 287-308, September.
    2. Jochen Ranger, 2013. "A Note on the Hierarchical Model for Responses and Response Times in Tests of van der Linden (2007)," Psychometrika, Springer;The Psychometric Society, vol. 78(3), pages 538-544, July.
    3. Inhan Kang & Paul Boeck & Roger Ratcliff, 2022. "Modeling Conditional Dependence of Response Accuracy and Response Time with the Diffusion Item Response Theory Model," Psychometrika, Springer;The Psychometric Society, vol. 87(2), pages 725-748, June.
    4. Peida Zhan & Hong Jiao & Kaiwen Man & Lijun Wang, 2019. "Using JAGS for Bayesian Cognitive Diagnosis Modeling: A Tutorial," Journal of Educational and Behavioral Statistics, , vol. 44(4), pages 473-503, August.
    5. Jimmy de la Torre, 2011. "The Generalized DINA Model Framework," Psychometrika, Springer;The Psychometric Society, vol. 76(2), pages 179-199, April.
    6. Shiyu Wang & Yinghan Chen, 2020. "Using Response Times and Response Accuracy to Measure Fluency Within Cognitive Diagnosis Models," Psychometrika, Springer;The Psychometric Society, vol. 85(3), pages 600-629, September.
    7. Jimmy Torre & Jeffrey Douglas, 2004. "Higher-order latent trait models for cognitive diagnosis," Psychometrika, Springer;The Psychometric Society, vol. 69(3), pages 333-353, September.
    8. Minjeong Jeon & Paul Boeck & Jevan Luo & Xiangrui Li & Zhong-Lin Lu, 2021. "Modeling Within-Item Dependencies in Parallel Data on Test Responses and Brain Activation," Psychometrika, Springer;The Psychometric Society, vol. 86(1), pages 239-271, March.
    9. Jimmy Torre, 2011. "Erratum to: The Generalized DINA Model Framework," Psychometrika, Springer;The Psychometric Society, vol. 76(3), pages 510-510, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhimou Wang & Yaohui Liu & Peida Zhan, 2025. "Using a Deep Learning-Based Visual Computational Model to Identify Cognitive Strategies in Matrix Reasoning," Journal of Educational and Behavioral Statistics, , vol. 50(5), pages 806-832, October.
    2. Kazuhiro Yamaguchi, 2023. "Bayesian Analysis Methods for Two-Level Diagnosis Classification Models," Journal of Educational and Behavioral Statistics, , vol. 48(6), pages 773-809, December.
    3. Seunghyun Lee & Yuqi Gu, 2024. "New Paradigm of Identifiable General-response Cognitive Diagnostic Models: Beyond Categorical Data," Psychometrika, Springer;The Psychometric Society, vol. 89(4), pages 1304-1336, December.
    4. Liu, Yaohui & Zhan, Peida & Fu, Yanbin & Chen, Qipeng & Man, Kaiwen & Luo, Yikun, 2023. "Using a multi-strategy eye-tracking psychometric model to measure intelligence and identify cognitive strategy in Raven's advanced progressive matrices," Intelligence, Elsevier, vol. 100(C).
    5. Xin Qiao & Cornelis Potgieter, 2026. "A Quasi-Poisson Item Response Theory Model for Heterogeneous Dispersion in Count Data," Journal of Educational and Behavioral Statistics, , vol. 51(1), pages 60-90, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peida Zhan & Xin Qiao, 2022. "DIAGNOSTIC Classification Analysis of Problem-Solving Competence using Process Data: An Item Expansion Method," Psychometrika, Springer;The Psychometric Society, vol. 87(4), pages 1529-1547, December.
    2. Kazuhiro Yamaguchi, 2025. "Bayesian Diagnostic Classification Models for a Partially Known Q-Matrix," Journal of Educational and Behavioral Statistics, , vol. 50(2), pages 331-382, April.
    3. Kazuhiro Yamaguchi & Jonathan Templin, 2022. "Direct Estimation of Diagnostic Classification Model Attribute Mastery Profiles via a Collapsed Gibbs Sampling Algorithm," Psychometrika, Springer;The Psychometric Society, vol. 87(4), pages 1390-1421, December.
    4. Yuting Han & Feng Ji & Zhehan Jiang, 2025. "Two-stage polytomous attribute estimation for cognitive diagnostic models: overcoming computational challenges in large-scale assessments with many polytomous attributes," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 12(1), pages 1-14, December.
    5. Junhuan Wei & Liufen Luo & Yan Cai & Dongbo Tu, 2024. "A Multistrategy Cognitive Diagnosis Model Incorporating Item Response Times Based on Strategy Selection Theories," Journal of Educational and Behavioral Statistics, , vol. 49(4), pages 658-686, August.
    6. Tian Shu & Guanzhong Luo & Zhaosheng Luo & Xiaofeng Yu & Xiaojun Guo & Yujun Li, 2023. "An Explicit Form With Continuous Attribute Profile of the Partial Mastery DINA Model," Journal of Educational and Behavioral Statistics, , vol. 48(5), pages 573-602, October.
    7. Seunghyun Lee & Yuqi Gu, 2024. "New Paradigm of Identifiable General-response Cognitive Diagnostic Models: Beyond Categorical Data," Psychometrika, Springer;The Psychometric Society, vol. 89(4), pages 1304-1336, December.
    8. Kazuhiro Yamaguchi & Jonathan Templin, 2022. "A Gibbs Sampling Algorithm with Monotonicity Constraints for Diagnostic Classification Models," Journal of Classification, Springer;The Classification Society, vol. 39(1), pages 24-54, March.
    9. Matthew S. Johnson & Sandip Sinharay, 2020. "The Reliability of the Posterior Probability of Skill Attainment in Diagnostic Classification Models," Journal of Educational and Behavioral Statistics, , vol. 45(1), pages 5-31, February.
    10. James Joseph Balamuta & Steven Andrew Culpepper, 2022. "Exploratory Restricted Latent Class Models with Monotonicity Requirements under PÒLYA–GAMMA Data Augmentation," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 903-945, September.
    11. Kazuhiro Yamaguchi & Kensuke Okada, 2020. "Variational Bayes Inference for the DINA Model," Journal of Educational and Behavioral Statistics, , vol. 45(5), pages 569-597, October.
    12. Xin Xu & Guanhua Fang & Jinxin Guo & Zhiliang Ying & Susu Zhang, 2024. "Diagnostic Classification Models for Testlets: Methods and Theory," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 851-876, September.
    13. Peida Zhan & Wen-Chung Wang & Xiaomin Li, 2020. "A Partial Mastery, Higher-Order Latent Structural Model for Polytomous Attributes in Cognitive Diagnostic Assessments," Journal of Classification, Springer;The Classification Society, vol. 37(2), pages 328-351, July.
    14. Yuqi Gu, 2024. "Going Deep in Diagnostic Modeling: Deep Cognitive Diagnostic Models (DeepCDMs)," Psychometrika, Springer;The Psychometric Society, vol. 89(1), pages 118-150, March.
    15. Peida Zhan & Hong Jiao & Dandan Liao & Feiming Li, 2019. "A Longitudinal Higher-Order Diagnostic Classification Model," Journal of Educational and Behavioral Statistics, , vol. 44(3), pages 251-281, June.
    16. Kazuhiro Yamaguchi & Kensuke Okada, 2020. "Variational Bayes Inference Algorithm for the Saturated Diagnostic Classification Model," Psychometrika, Springer;The Psychometric Society, vol. 85(4), pages 973-995, December.
    17. Daoxuan Fu & Chunying Qin & Zhaosheng Luo & Yujun Li & Xiaofeng Yu & Ziyu Ye, 2025. "Using Regularized Methods to Validate Q-Matrix in Cognitive Diagnostic Assessment," Journal of Educational and Behavioral Statistics, , vol. 50(1), pages 149-179, February.
    18. Chen-Wei Liu & Björn Andersson & Anders Skrondal, 2020. "A Constrained Metropolis–Hastings Robbins–Monro Algorithm for Q Matrix Estimation in DINA Models," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 322-357, June.
    19. Chun Wang & Jing Lu, 2021. "Learning Attribute Hierarchies From Data: Two Exploratory Approaches," Journal of Educational and Behavioral Statistics, , vol. 46(1), pages 58-84, February.
    20. Hans Friedrich Köhn & Chia-Yi Chiu, 2021. "A Unified Theory of the Completeness of Q-Matrices for the DINA Model," Journal of Classification, Springer;The Classification Society, vol. 38(3), pages 500-518, October.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:47:y:2022:i:6:p:736-776. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.