IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v48y2023i4p490-520.html

Bayesian Change-Point Analysis Approach to Detecting Aberrant Test-Taking Behavior Using Response Times

Author

Listed:
  • Hongyue Zhu

    (Northeast Normal University)

  • Hong Jiao

    (University of Maryland)

  • Wei Gao
  • Xiangbin Meng

    (Northeast Normal University)

Abstract

Change-point analysis (CPA) is a method for detecting abrupt changes in parameter(s) underlying a sequence of random variables. It has been applied to detect examinees’ aberrant test-taking behavior by identifying abrupt test performance change. Previous studies utilized maximum likelihood estimations of ability parameters, focusing on detecting one change point for each examinee. This article proposes a Bayesian CPA procedure using response times (RTs) to detect abrupt changes in examinee speed, which may be related to aberrant responding behaviors. The lognormal RT model is used to derive a procedure for detecting aberrant RT patterns. The method takes the numbers and locations of the change points as parameters in the model to detect multiple change points or multiple aberrant behaviors. Given the change points, the corresponding speed of each segment in the test can be estimated, which enables more accurate inferences about aberrant behaviors. Simulation study results indicate that the proposed procedure can effectively detect simulated aberrant behaviors and estimate change points accurately. The method is applied to data from a high-stakes computerized adaptive test, where its applicability is demonstrated.

Suggested Citation

  • Hongyue Zhu & Hong Jiao & Wei Gao & Xiangbin Meng, 2023. "Bayesian Change-Point Analysis Approach to Detecting Aberrant Test-Taking Behavior Using Response Times," Journal of Educational and Behavioral Statistics, , vol. 48(4), pages 490-520, August.
    • Handle: RePEc:sae:jedbes:v:48:y:2023:i:4:p:490-520
    DOI: 10.3102/10769986231151961
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/10769986231151961
    Download Restriction: no
    File URL: https://libkey.io/10.3102/10769986231151961?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Wim van der Linden, 2007. "A Hierarchical Framework for Modeling Speed and Accuracy on Test Items," Psychometrika, Springer;The Psychometric Society, vol. 72(3), pages 287-308, September.
    2. Wim Linden & Fanmin Guo, 2008. "Bayesian Procedures for Identifying Aberrant Response-Time Patterns in Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 73(3), pages 365-384, September.
    3. Nicolas Chopin, 2007. "Dynamic Detection of Change Points in Long Time Series," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 59(2), pages 349-366, June.
    4. Edison M. Choe & Jinming Zhang & Hua-Hua Chang, 2018. "Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 83(3), pages 650-673, September.
    5. Ruggieri, Eric & Antonellis, Marcus, 2016. "An exact approach to Bayesian sequential change point detection," Computational Statistics & Data Analysis, Elsevier, vol. 97(C), pages 71-86.
    6. Sandip Sinharay, 2017. "Some Remarks on Applications of Tests for Detecting A Change Point to Psychometric Problems," Psychometrika, Springer;The Psychometric Society, vol. 82(4), pages 1149-1161, December.
    7. Can Shao & Jun Li & Ying Cheng, 2016. "Detection of Test Speededness Using Change-Point Analysis," Psychometrika, Springer;The Psychometric Society, vol. 81(4), pages 1118-1141, December.
    8. Wim Linden & Edith Krimpen-Stoop, 2003. "Using response times to detect aberrant responses in computerized adaptive testing," Psychometrika, Springer;The Psychometric Society, vol. 68(2), pages 251-265, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yang Du & Susu Zhang, 2025. "Detecting Compromised Items With Response Times Using a Bayesian Change-Point Approach," Journal of Educational and Behavioral Statistics, , vol. 50(2), pages 296-330, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Edison M. Choe & Jinming Zhang & Hua-Hua Chang, 2018. "Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 83(3), pages 650-673, September.
    2. Dylan Molenaar & Paul Boeck, 2018. "Response Mixture Modeling: Accounting for Heterogeneity in Item Characteristics across Response Times," Psychometrika, Springer;The Psychometric Society, vol. 83(2), pages 279-297, June.
    3. Hyeon-Ah Kang, 2023. "Sequential Generalized Likelihood Ratio Tests for Online Item Monitoring," Psychometrika, Springer;The Psychometric Society, vol. 88(2), pages 672-696, June.
    4. Chun Wang & Gongjun Xu & Zhuoran Shang, 2018. "A Two-Stage Approach to Differentiating Normal and Aberrant Behavior in Computer Based Testing," Psychometrika, Springer;The Psychometric Society, vol. 83(1), pages 223-254, March.
    5. Ebru Balta & Celal Deha Dogan, 2024. "Investigation of Preknowledge Cheating via Joint Hierarchical Modeling Patterns of Response Accuracy and Response Time," SAGE Open, , vol. 14(4), pages 21582440241, November.
    6. Onur Demirkaya & Ummugul Bezirhan & Jinming Zhang, 2023. "Detecting Item Preknowledge Using Revisits With Speed and Accuracy," Journal of Educational and Behavioral Statistics, , vol. 48(4), pages 521-542, August.
    7. Wim J. van der Linden, 2022. "Two Statistical Tests for the Detection of Item Compromise," Journal of Educational and Behavioral Statistics, , vol. 47(4), pages 485-504, August.
    8. Yang Du & Susu Zhang, 2025. "Detecting Compromised Items With Response Times Using a Bayesian Change-Point Approach," Journal of Educational and Behavioral Statistics, , vol. 50(2), pages 296-330, April.
    9. Hyeon-Ah Kang & Yi Zheng & Hua-Hua Chang, 2020. "Online Calibration of a Joint Model of Item Responses and Response Times in Computerized Adaptive Testing," Journal of Educational and Behavioral Statistics, , vol. 45(2), pages 175-208, April.
    10. Jochen Ranger, 2013. "A Note on the Hierarchical Model for Responses and Response Times in Tests of van der Linden (2007)," Psychometrika, Springer;The Psychometric Society, vol. 78(3), pages 538-544, July.
    11. Steffi Pohl & Esther Ulitzsch & Matthias Davier, 2019. "Using Response Times to Model Not-Reached Items due to Time Limits," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 892-920, September.
    12. Maria Bolsinova & Paul Boeck & Jesper Tijmstra, 2017. "Modelling Conditional Dependence Between Response Time and Accuracy," Psychometrika, Springer;The Psychometric Society, vol. 82(4), pages 1126-1148, December.
    13. Sandip Sinharay & Peter W. van Rijn, 2020. "Assessing Fit of the Lognormal Model for Response Times," Journal of Educational and Behavioral Statistics, , vol. 45(5), pages 534-568, October.
    14. Renske E. Kuijpers & Ingmar Visser & Dylan Molenaar, 2021. "Testing the Within-State Distribution in Mixture Models for Responses and Response Times," Journal of Educational and Behavioral Statistics, , vol. 46(3), pages 348-373, June.
    15. Cardenas Hurtado, Camilo & Moustaki, Irini & Chen, Yunxiao & Marra, Giampiero, 2025. "Generalized latent variable models for location, scale, and shape parameters," LSE Research Online Documents on Economics 127387, London School of Economics and Political Science, LSE Library.
    16. Eric Ruggieri, 2018. "A pruned recursive solution to the multiple change point problem," Computational Statistics, Springer, vol. 33(2), pages 1017-1045, June.
    17. Wallin, Gabriel & Chen, Yunxiao & Lee, Yi-Hsuan & Li, Xiaoou, 2025. "A latent variable model with change-points and its application to time pressure effects in educational assessment," LSE Research Online Documents on Economics 128070, London School of Economics and Political Science, LSE Library.
    18. Chongqin Xi & Wenjing Guo & Qingrong Tan & Dongbo Tu & Yan Cai, 2025. "A Mixture Response Model for Identifying Item Preknowledge," Journal of Educational and Behavioral Statistics, , vol. 50(5), pages 863-895, October.
    19. Wim J. van der Linden, 2009. "A Bivariate Lognormal Response-Time Model for the Detection of Collusion Between Test Takers," Journal of Educational and Behavioral Statistics, , vol. 34(3), pages 378-394, September.
    20. Rui Qiang & Eric Ruggieri, 2023. "Autocorrelation and Parameter Estimation in a Bayesian Change Point Model," Mathematics, MDPI, vol. 11(5), pages 1-22, February.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:48:y:2023:i:4:p:490-520. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.