IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v80y2015i1p1-20.html
   My bibliography  Save this article

Psychometrics Behind Computerized Adaptive Testing

Author

Listed:
  • Hua-Hua Chang

Abstract

The paper provides a survey of 18 years’ progress that my colleagues, students (both former and current) and I made in a prominent research area in Psychometrics—Computerized Adaptive Testing (CAT). We start with a historical review of the establishment of a large sample foundation for CAT. It is worth noting that the asymptotic results were derived under the framework of Martingale Theory, a very theoretical perspective of Probability Theory, which may seem unrelated to educational and psychological testing. In addition, we address a number of issues that emerged from large scale implementation and show that how theoretical works can be helpful to solve the problems. Finally, we propose that CAT technology can be very useful to support individualized instruction on a mass scale. We show that even paper and pencil based tests can be made adaptive to support classroom teaching. Copyright The Psychometric Society 2015

Suggested Citation

  • Hua-Hua Chang, 2015. "Psychometrics Behind Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 1-20, March.
    • Handle: RePEc:spr:psycho:v:80:y:2015:i:1:p:1-20
    DOI: 10.1007/s11336-014-9401-5
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s11336-014-9401-5
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11336-014-9401-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Wim van der Linden, 2007. "A Hierarchical Framework for Modeling Speed and Accuracy on Test Items," Psychometrika, Springer;The Psychometric Society, vol. 72(3), pages 287-308, September.
    2. Hua-Hua Chang & Zhiliang Ying, 2008. "To Weight or Not to Weight? Balancing Influence of Initial Items in Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 73(3), pages 441-450, September.
    3. Chun Wang & Zhewen Fan & Hua-Hua Chang & Jeffrey A. Douglas, 2013. "A Semiparametric Model for Jointly Analyzing Response Times and Accuracy in Computerized Testing," Journal of Educational and Behavioral Statistics, , vol. 38(4), pages 381-417, August.
    4. Eric Maris, 1993. "Additive and multiplicative models for gamma distributed random variables, and their application as psychometric models for response times," Psychometrika, Springer;The Psychometric Society, vol. 58(3), pages 445-469, September.
    5. Hong-Yun Liu & Xiao-Feng You & Wen-Yi Wang & Shu-Liang Ding & Hua-Hua Chang, 2013. "The Development of Computerized Adaptive Testing with Cognitive Diagnosis for an English Achievement Test in China," Journal of Classification, Springer;The Classification Society, vol. 30(2), pages 152-172, July.
    6. Joris Mulder & Wim Linden, 2009. "Multidimensional Adaptive Testing with Optimal Design Criteria for Item Selection," Psychometrika, Springer;The Psychometric Society, vol. 74(2), pages 273-296, June.
    7. Robert Mislevy & Hua-Hua Chang, 2000. "Does adaptive testing violate local independence?," Psychometrika, Springer;The Psychometric Society, vol. 65(2), pages 149-156, June.
    8. Paul Holland, 1990. "The Dutch Identity: A new tool for the study of item response models," Psychometrika, Springer;The Psychometric Society, vol. 55(1), pages 5-18, March.
    9. Chun Wang & Hua-Hua Chang, 2011. "Item Selection in Multidimensional Computerized Adaptive Testing—Gaining Information from Different Angles," Psychometrika, Springer;The Psychometric Society, vol. 76(3), pages 363-384, July.
    10. Jeffrey Rouder & Dongchu Sun & Paul Speckman & Jun Lu & Duo Zhou, 2003. "A hierarchical bayesian statistical framework for response time distributions," Psychometrika, Springer;The Psychometric Society, vol. 68(4), pages 589-606, December.
    11. Bernard Veldkamp & Wim Linden, 2002. "Multidimensional adaptive testing with constraints on test content," Psychometrika, Springer;The Psychometric Society, vol. 67(4), pages 575-588, December.
    12. Daniel Segall, 1996. "Multidimensional adaptive testing," Psychometrika, Springer;The Psychometric Society, vol. 61(2), pages 331-354, June.
    13. Hua-Hua Chang & William Stout, 1993. "The asymptotic posterior normality of the latent trait in an IRT model," Psychometrika, Springer;The Psychometric Society, vol. 58(1), pages 37-52, March.
    14. Chun Wang & Hua-Hua Chang & Keith Boughton, 2011. "Kullback–Leibler Information and Its Applications in Multi-Dimensional Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 76(1), pages 13-39, January.
    15. Daniel Segall, 2001. "General ability measurement: An application of multidimensional item response theory," Psychometrika, Springer;The Psychometric Society, vol. 66(1), pages 79-97, March.
    16. Ying Cheng, 2009. "When Cognitive Diagnosis Meets Computerized Adaptive Testing: CD-CAT," Psychometrika, Springer;The Psychometric Society, vol. 74(4), pages 619-632, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yan Li & Chao Huang & Jia Liu, 2023. "Diagnosing Primary Students’ Reading Progression: Is Cognitive Diagnostic Computerized Adaptive Testing the Way Forward?," Journal of Educational and Behavioral Statistics, , vol. 48(6), pages 842-865, December.
    2. Xiao Li & Hanchen Xu & Jinming Zhang & Hua-hua Chang, 2023. "Deep Reinforcement Learning for Adaptive Learning Systems," Journal of Educational and Behavioral Statistics, , vol. 48(2), pages 220-243, April.
    3. Edison M. Choe & Hua-Hua Chang, 2019. "The Asymptotic Distribution of Average Test Overlap Rate in Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 84(4), pages 1129-1151, December.
    4. Edison M. Choe & Jinming Zhang & Hua-Hua Chang, 2018. "Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 83(3), pages 650-673, September.
    5. Wenyi Wang & Lihong Song & Teng Wang & Peng Gao & Jian Xiong, 2020. "A Note on the Relationship of the Shannon Entropy Procedure and the Jensen–Shannon Divergence in Cognitive Diagnostic Computerized Adaptive Testing," SAGE Open, , vol. 10(1), pages 21582440198, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chun Wang & Hua-Hua Chang, 2011. "Item Selection in Multidimensional Computerized Adaptive Testing—Gaining Information from Different Angles," Psychometrika, Springer;The Psychometric Society, vol. 76(3), pages 363-384, July.
    2. Ping Chen & Chun Wang, 2016. "A New Online Calibration Method for Multidimensional Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 674-701, September.
    3. Chun Wang, 2015. "On Latent Trait Estimation in Multidimensional Compensatory Item Response Models," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 428-449, June.
    4. Chun Wang & Hua-Hua Chang & Keith Boughton, 2011. "Kullback–Leibler Information and Its Applications in Multi-Dimensional Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 76(1), pages 13-39, January.
    5. Chun Wang, 2014. "Improving Measurement Precision of Hierarchical Latent Traits Using Adaptive Testing," Journal of Educational and Behavioral Statistics, , vol. 39(6), pages 452-477, December.
    6. Chun Wang & David J. Weiss & Zhuoran Shang, 2019. "Variable-Length Stopping Rules for Multidimensional Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 749-771, September.
    7. Lihua Yao, 2012. "Multidimensional CAT Item Selection Methods for Domain Scores and Composite Scores: Theory and Applications," Psychometrika, Springer;The Psychometric Society, vol. 77(3), pages 495-523, July.
    8. Ping Chen, 2017. "A Comparative Study of Online Item Calibration Methods in Multidimensional Computerized Adaptive Testing," Journal of Educational and Behavioral Statistics, , vol. 42(5), pages 559-590, October.
    9. Zhewen Fan & Chun Wang & Hua-Hua Chang & Jeffrey Douglas, 2012. "Utilizing Response Time Distributions for Item Selection in CAT," Journal of Educational and Behavioral Statistics, , vol. 37(5), pages 655-670, October.
    10. Edison M. Choe & Jinming Zhang & Hua-Hua Chang, 2018. "Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 83(3), pages 650-673, September.
    11. Chun Wang & Gongjun Xu & Zhuoran Shang, 2018. "A Two-Stage Approach to Differentiating Normal and Aberrant Behavior in Computer Based Testing," Psychometrika, Springer;The Psychometric Society, vol. 83(1), pages 223-254, March.
    12. M. Marsman & H. Sigurdardóttir & M. Bolsinova & G. Maris, 2019. "Characterizing the Manifest Probability Distributions of Three Latent Trait Models for Accuracy and Response Time," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 870-891, September.
    13. Maria Bolsinova & Jesper Tijmstra, 2019. "Modeling Differences Between Response Times of Correct and Incorrect Responses," Psychometrika, Springer;The Psychometric Society, vol. 84(4), pages 1018-1046, December.
    14. Matthias von Davier & Lale Khorramdel & Qiwei He & Hyo Jeong Shin & Haiwen Chen, 2019. "Developments in Psychometric Population Models for Technology-Based Large-Scale Assessments: An Overview of Challenges and Opportunities," Journal of Educational and Behavioral Statistics, , vol. 44(6), pages 671-705, December.
    15. Qingrong Tan & Yan Cai & Fen Luo & Dongbo Tu, 2023. "Development of a High-Accuracy and Effective Online Calibration Method in CD-CAT Based on Gini Index," Journal of Educational and Behavioral Statistics, , vol. 48(1), pages 103-141, February.
    16. Yan Li & Chao Huang & Jia Liu, 2023. "Diagnosing Primary Students’ Reading Progression: Is Cognitive Diagnostic Computerized Adaptive Testing the Way Forward?," Journal of Educational and Behavioral Statistics, , vol. 48(6), pages 842-865, December.
    17. Sandip Sinharay, 2015. "The Asymptotic Distribution of Ability Estimates," Journal of Educational and Behavioral Statistics, , vol. 40(5), pages 511-528, October.
    18. Sandip Sinharay & Peter W. van Rijn, 2020. "Assessing Fit of the Lognormal Model for Response Times," Journal of Educational and Behavioral Statistics, , vol. 45(5), pages 534-568, October.
    19. Crabbe, Marjolein & Akinc, Deniz & Vandebroek, Martina, 2014. "Fast algorithms to generate individualized designs for the mixed logit choice model," Transportation Research Part B: Methodological, Elsevier, vol. 60(C), pages 1-15.
    20. Pascal Jordan, 2023. "On Reverse Shrinkage Effects and Shrinkage Overshoot," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 274-301, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:80:y:2015:i:1:p:1-20. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.