IDEAS home Printed from https://ideas.repec.org/a/rnd/arimbr/v16y2024i2p63-72.html

Conundrum and Considerations in Cognitive Diagnostic Assessment for Language Proficiency Evaluation

Author

Listed:
  • Muhamad Firdaus Mohd Noh
  • Mohd Effendi Ewan bin Mohd Matore

Abstract

Since its first appearance in the field of language testing, cognitive diagnostic assessment (CDA) has attracted attention for its ability to extract the intricacies of students' cognitive abilities. However limited research has discussed the issues in the implementation of CDA. Therefore, this article offers an overview of CDA's implementation in language proficiency evaluation. The article also engages in a comprehensive discussion on the conundrum and considerations within CDA, particularly the ongoing debate between distinct classifications of cognitive diagnostic models. It elaborates on the distinctions between the models and their implications for assessment depth and diagnostic insights. Additionally, this article delves into the clash between retrofitting existing items and developing new diagnostic items, highlighting the strategic considerations in each approach. Apart from that, the contentious issue of validating Q-matrices, crucial in CDA, is thoroughly examined, presenting the battle between expert-based and empirical validation methods. The persistent challenges in CDA have profound implications for both theoretical frameworks and practical applications. The theoretical debate not only influences our understanding of cognitive processes but also shapes the conceptualization of diagnostic information extraction. In practical terms, decisions regarding item development, retrofitting strategies, and Q-matrix validation methods directly impact the effectiveness of CDA in providing targeted interventions and personalized learning strategies in real-world educational contexts. Future research directions are also presented, emphasizing the need for more development of entirely new diagnostic items, hybrid CDMs, and adaptive cognitive diagnostic assessments. Practical recommendations are provided for practitioners, encouraging a strategic approach based on specific assessment goals.

Suggested Citation

  • Muhamad Firdaus Mohd Noh & Mohd Effendi Ewan bin Mohd Matore, 2024. "Conundrum and Considerations in Cognitive Diagnostic Assessment for Language Proficiency Evaluation," Information Management and Business Review, AMH International, vol. 16(2), pages 63-72.
    • Handle: RePEc:rnd:arimbr:v:16:y:2024:i:2:p:63-72
    DOI: 10.22610/imbr.v16i2(I).3690
    as

    Download full text from publisher

    File URL: https://ojs.amhinternational.com/index.php/imbr/article/view/3690/2506
    Download Restriction: no
    File URL: https://ojs.amhinternational.com/index.php/imbr/article/view/3690
    Download Restriction: no
    File URL: https://libkey.io/10.22610/imbr.v16i2(I).3690?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jonathan Templin & Laine Bradshaw, 2013. "Measuring the Reliability of Diagnostic Classification Model Examinee Estimates," Journal of Classification, Springer;The Classification Society, vol. 30(2), pages 251-275, July.
    2. Jimmy de la Torre, 2011. "The Generalized DINA Model Framework," Psychometrika, Springer;The Psychometric Society, vol. 76(2), pages 179-199, April.
    3. Jimmy Torre, 2011. "Erratum to: The Generalized DINA Model Framework," Psychometrika, Springer;The Psychometric Society, vol. 76(3), pages 510-510, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Matthew S. Johnson & Sandip Sinharay, 2020. "The Reliability of the Posterior Probability of Skill Attainment in Diagnostic Classification Models," Journal of Educational and Behavioral Statistics, , vol. 45(1), pages 5-31, February.
    2. Laine Bradshaw & Jonathan Templin, 2014. "Combining Item Response Theory and Diagnostic Classification Models: A Psychometric Model for Scaling Ability and Diagnosing Misconceptions," Psychometrika, Springer;The Psychometric Society, vol. 79(3), pages 403-425, July.
    3. Kazuhiro Yamaguchi & Kensuke Okada, 2020. "Variational Bayes Inference Algorithm for the Saturated Diagnostic Classification Model," Psychometrika, Springer;The Psychometric Society, vol. 85(4), pages 973-995, December.
    4. Zhemin Zhu & Bing Jia, 2025. "Study on Mathematics Skill of Sixth-Grade Students in Rural China Based on Cognitive Diagnostic Models," SAGE Open, , vol. 15(4), pages 21582440251, December.
    5. Jonathan Templin & Laine Bradshaw, 2014. "Hierarchical Diagnostic Classification Models: A Family of Models for Estimating and Testing Attribute Hierarchies," Psychometrika, Springer;The Psychometric Society, vol. 79(2), pages 317-339, April.
    6. Junhuan Wei & Liufen Luo & Yan Cai & Dongbo Tu, 2024. "A Multistrategy Cognitive Diagnosis Model Incorporating Item Response Times Based on Strategy Selection Theories," Journal of Educational and Behavioral Statistics, , vol. 49(4), pages 658-686, August.
    7. Xueqin Zhang & Yu Jiang & Tao Xin & Yanlou Liu, 2025. "Iterative Attribute Hierarchy Exploration Methods for Cognitive Diagnosis Models," Journal of Educational and Behavioral Statistics, , vol. 50(4), pages 682-713, August.
    8. Chenchen Ma & Jing Ouyang & Gongjun Xu, 2023. "Learning Latent and Hierarchical Structures in Cognitive Diagnosis Models," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 175-207, March.
    9. Ling Ling Wang & Tao Xin & Xiao Jian Sun, 2026. "Using the Bayesian Network’s Structural Learning Algorithm to Estimate the Q-Matrix in Cognitive Diagnosis Models," Journal of Educational and Behavioral Statistics, , vol. 51(2), pages 419-450, April.
    10. James Joseph Balamuta & Steven Andrew Culpepper, 2022. "Exploratory Restricted Latent Class Models with Monotonicity Requirements under PÒLYA–GAMMA Data Augmentation," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 903-945, September.
    11. Kazuhiro Yamaguchi & Kensuke Okada, 2020. "Variational Bayes Inference for the DINA Model," Journal of Educational and Behavioral Statistics, , vol. 45(5), pages 569-597, October.
    12. Qingrong Tan & Yan Cai & Fen Luo & Dongbo Tu, 2023. "Development of a High-Accuracy and Effective Online Calibration Method in CD-CAT Based on Gini Index," Journal of Educational and Behavioral Statistics, , vol. 48(1), pages 103-141, February.
    13. Xin Xu & Guanhua Fang & Jinxin Guo & Zhiliang Ying & Susu Zhang, 2024. "Diagnostic Classification Models for Testlets: Methods and Theory," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 851-876, September.
    14. Peida Zhan & Wen-Chung Wang & Xiaomin Li, 2020. "A Partial Mastery, Higher-Order Latent Structural Model for Polytomous Attributes in Cognitive Diagnostic Assessments," Journal of Classification, Springer;The Classification Society, vol. 37(2), pages 328-351, July.
    15. Yuqi Gu, 2024. "Going Deep in Diagnostic Modeling: Deep Cognitive Diagnostic Models (DeepCDMs)," Psychometrika, Springer;The Psychometric Society, vol. 89(1), pages 118-150, March.
    16. Peida Zhan & Hong Jiao & Dandan Liao & Feiming Li, 2019. "A Longitudinal Higher-Order Diagnostic Classification Model," Journal of Educational and Behavioral Statistics, , vol. 44(3), pages 251-281, June.
    17. Kazuhiro Yamaguchi, 2025. "Bayesian Diagnostic Classification Models for a Partially Known Q-Matrix," Journal of Educational and Behavioral Statistics, , vol. 50(2), pages 331-382, April.
    18. Peida Zhan* & Kaiwen Man* & Stefanie A. Wind & Jonathan Malone, 2022. "Cognitive Diagnosis Modeling Incorporating Response Times and Fixation Counts: Providing Comprehensive Feedback and Accurate Diagnosis," Journal of Educational and Behavioral Statistics, , vol. 47(6), pages 736-776, December.
    19. Chia-Yi Chiu & Yuan-Pei Chang, 2021. "Advances in CD-CAT: The General Nonparametric Item Selection Method," Psychometrika, Springer;The Psychometric Society, vol. 86(4), pages 1039-1057, December.
    20. Daoxuan Fu & Chunying Qin & Zhaosheng Luo & Yujun Li & Xiaofeng Yu & Ziyu Ye, 2025. "Using Regularized Methods to Validate Q-Matrix in Cognitive Diagnostic Assessment," Journal of Educational and Behavioral Statistics, , vol. 50(1), pages 149-179, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:rnd:arimbr:v:16:y:2024:i:2:p:63-72. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Muhammad Tayyab (email available below). General contact details of provider: https://ojs.amhinternational.com/index.php/imbr .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.