IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v88y2023i4d10.1007_s11336-022-09885-3.html
   My bibliography  Save this article

Three Psychometric-Model-Based Option-Scored Multiple Choice Item Design Principles that Enhance Instruction by Improving Quiz Diagnostic Classification of Knowledge Attributes

Author

Listed:
  • William Stout

    (University of Illinois at Urbana-Champaign: (Statistics: Emeritus)
    UNIVERSITY OF ILLINOIS CHICAGO (LEARNING SCIENCES RESEARCH INSTITUTE: EMERITUS))

  • Robert Henson

    (University of North Carolina Greensboro (Education))

  • Lou DiBello

    (UNIVERSITY OF ILLINOIS CHICAGO (LEARNING SCIENCES RESEARCH INSTITUTE: EMERITUS))

Abstract

Three IRT diagnostic-classification-modeling (DCM)-based multiple choice (MC) item design principles are stated that improve classroom quiz student diagnostic classification. Using proven-optimal maximum likelihood-based student classification, example items demonstrate that adherence to these item design principles increases attribute (skills and especially misconceptions) correct classification rates (CCRs). Simple formulas compute these needed item CCRs. By use of these psychometrically driven item design principles, hopefully enough attributes can be accurately diagnosed by necessarily short MC-item-based quizzes to be widely instructionally useful. These results should then stimulate increased use of well-designed MC item quizzes that target accurately diagnosing skills/misconceptions, thereby enhancing classroom learning.

Suggested Citation

  • William Stout & Robert Henson & Lou DiBello, 2023. "Three Psychometric-Model-Based Option-Scored Multiple Choice Item Design Principles that Enhance Instruction by Improving Quiz Diagnostic Classification of Knowledge Attributes," Psychometrika, Springer;The Psychometric Society, vol. 88(4), pages 1299-1333, December.
    • Handle: RePEc:spr:psycho:v:88:y:2023:i:4:d:10.1007_s11336-022-09885-3
    DOI: 10.1007/s11336-022-09885-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11336-022-09885-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11336-022-09885-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. R. Darrell Bock, 1972. "Estimating item parameters and latent ability when responses are scored in two or more nominal categories," Psychometrika, Springer;The Psychometric Society, vol. 37(1), pages 29-51, March.
    2. Laine Bradshaw & Jonathan Templin, 2014. "Combining Item Response Theory and Diagnostic Classification Models: A Psychometric Model for Scaling Ability and Diagnosing Misconceptions," Psychometrika, Springer;The Psychometric Society, vol. 79(3), pages 403-425, July.
    3. Hans-Friedrich Köhn & Chia-Yi Chiu, 2016. "A Proof of the Duality of the DINA Model and the DINO Model," Journal of Classification, Springer;The Classification Society, vol. 33(2), pages 171-184, July.
    4. Chia-Yi Chiu & Jeff Douglas, 2013. "A Nonparametric Approach to Cognitive Diagnosis by Proximity to Ideal Response Patterns," Journal of Classification, Springer;The Classification Society, vol. 30(2), pages 225-250, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michelle M. LaMar, 2018. "Markov Decision Process Measurement Model," Psychometrika, Springer;The Psychometric Society, vol. 83(1), pages 67-88, March.
    2. Chia-Yi Chiu & Hans-Friedrich Köhn, 2019. "Consistency Theory for the General Nonparametric Classification Method," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 830-845, September.
    3. Pablo Nájera & Francisco J. Abad & Chia-Yi Chiu & Miguel A. Sorrel, 2023. "The Restricted DINA Model: A Comprehensive Cognitive Diagnostic Model for Classroom-Level Assessments," Journal of Educational and Behavioral Statistics, , vol. 48(6), pages 719-749, December.
    4. Lei Guo & Wenjie Zhou & Xiao Li, 2024. "Cognitive Diagnosis Testlet Model for Multiple-Choice Items," Journal of Educational and Behavioral Statistics, , vol. 49(1), pages 32-60, February.
    5. Kazuhiro Yamaguchi & Kensuke Okada, 2020. "Variational Bayes Inference for the DINA Model," Journal of Educational and Behavioral Statistics, , vol. 45(5), pages 569-597, October.
    6. Cheng-Hsuan Li & Yi-Jin Ju & Pei-Jyun Hsieh, 2022. "A Nonparametric Weighted Cognitive Diagnosis Model and Its Application on Remedial Instruction in a Small-Class Situation," Sustainability, MDPI, vol. 14(10), pages 1-17, May.
    7. Peida Zhan & Hong Jiao & Kaiwen Man & Lijun Wang, 2019. "Using JAGS for Bayesian Cognitive Diagnosis Modeling: A Tutorial," Journal of Educational and Behavioral Statistics, , vol. 44(4), pages 473-503, August.
    8. Luo, Nanyu & Ji, Feng & Han, Yuting & He, Jinbo & Zhang, Xiaoya, 2024. "Fitting item response theory models using deep learning computational frameworks," OSF Preprints tjxab, Center for Open Science.
    9. Bas Hemker & Klaas Sijtsma & Ivo Molenaar & Brian Junker, 1996. "Polytomous IRT models and monotone likelihood ratio of the total score," Psychometrika, Springer;The Psychometric Society, vol. 61(4), pages 679-693, December.
    10. Hans-Friedrich Köhn & Chia-Yi Chiu, 2018. "How to Build a Complete Q-Matrix for a Cognitively Diagnostic Test," Journal of Classification, Springer;The Classification Society, vol. 35(2), pages 273-299, July.
    11. Sijia Huang & Li Cai, 2024. "Cross-Classified Item Response Theory Modeling With an Application to Student Evaluation of Teaching," Journal of Educational and Behavioral Statistics, , vol. 49(3), pages 311-341, June.
    12. Hans-Friedrich Köhn & Chia-Yi Chiu, 2017. "A Procedure for Assessing the Completeness of the Q-Matrices of Cognitively Diagnostic Tests," Psychometrika, Springer;The Psychometric Society, vol. 82(1), pages 112-132, March.
    13. Björn Andersson & Tao Xin, 2021. "Estimation of Latent Regression Item Response Theory Models Using a Second-Order Laplace Approximation," Journal of Educational and Behavioral Statistics, , vol. 46(2), pages 244-265, April.
    14. Xin Xu & Guanhua Fang & Jinxin Guo & Zhiliang Ying & Susu Zhang, 2024. "Diagnostic Classification Models for Testlets: Methods and Theory," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 851-876, September.
    15. Jouni Kuha & Myrsini Katsikatsou & Irini Moustaki, 2018. "Latent variable modelling with non‐ignorable item non‐response: multigroup response propensity models for cross‐national analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(4), pages 1169-1192, October.
    16. Laine Bradshaw & Jonathan Templin, 2014. "Combining Item Response Theory and Diagnostic Classification Models: A Psychometric Model for Scaling Ability and Diagnosing Misconceptions," Psychometrika, Springer;The Psychometric Society, vol. 79(3), pages 403-425, July.
    17. Javier Revuelta, 2004. "Analysis of distractor difficulty in multiple-choice items," Psychometrika, Springer;The Psychometric Society, vol. 69(2), pages 217-234, June.
    18. Ulf Böckenholt, 2012. "The Cognitive-Miser Response Model: Testing for Intuitive and Deliberate Reasoning," Psychometrika, Springer;The Psychometric Society, vol. 77(2), pages 388-399, April.
    19. Albert Yu & Jeffrey A. Douglas, 2023. "IRT Models for Learning With Item-Specific Learning Parameters," Journal of Educational and Behavioral Statistics, , vol. 48(6), pages 866-888, December.
    20. John Hsu & Tom Leonard & Kam-Wah Tsui, 1991. "Statistical inference for multiple choice tests," Psychometrika, Springer;The Psychometric Society, vol. 56(2), pages 327-348, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:88:y:2023:i:4:d:10.1007_s11336-022-09885-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.