IDEAS home Printed from https://ideas.repec.org/p/hig/wpaper/57edu2020.html
   My bibliography  Save this paper

Investigating The Cross-National Comparability Of Testing Using Response Times

Author

Listed:
  • Denis Federiakin

    (National Research University Higher School of Economics)

Abstract

The cross-national comparability of testing results is one of the main concerns associated with international assessment. One way to approach this concern is the theoretical framework of cross-national comparability. One such framework, proposed by Van de Vijver and Tanzer [1997], describes the bias in item meaning across countries and the bias in terms of test administration procedures and respondent behavior across countries. To study how items function in terms of respondent behavior and in terms of their psychological meaning, we introduce the concept of time-related differential item functioning (DIF). This concept is similar to traditional DIF but describes the incomparable time parameters of items across countries rather than item difficulty parameters. We discuss time-related DIF, referencing dual processing cognitive theory and illustrate how it limits the interpretability of a certain type of time-related DIF. We also demonstrate the analysis with real data, and discuss the difference introduced by response time information in parameter interpretation and modelling results.

Suggested Citation

  • Denis Federiakin, 2020. "Investigating The Cross-National Comparability Of Testing Using Response Times," HSE Working papers WP BRP 57/EDU/2020, National Research University Higher School of Economics.
    • Handle: RePEc:hig:wpaper:57edu2020
    as

    Download full text from publisher

    File URL: https://wp.hse.ru/data/2020/02/28/1560424464/57EDU2020.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Robin Shealy & William Stout, 1993. "A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF," Psychometrika, Springer;The Psychometric Society, vol. 58(2), pages 159-194, June.
    2. Stegmueller, Daniel, 2011. "Apples and Oranges? The Problem of Equivalence in Comparative Research," Political Analysis, Cambridge University Press, vol. 19(4), pages 471-487.
    3. Erling Andersen, 1977. "Sufficient statistics and latent trait models," Psychometrika, Springer;The Psychometric Society, vol. 42(1), pages 69-81, March.
    4. Gerhard Fischer, 1983. "Logistic latent trait models with linear constraints," Psychometrika, Springer;The Psychometric Society, vol. 48(1), pages 3-26, March.
    5. Florin Vaida & Suzette Blanchard, 2005. "Conditional Akaike information for mixed-effects models," Biometrika, Biometrika Trust, vol. 92(2), pages 351-370, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. G. Fischer & P. Parzer, 1991. "An extension of the rating scale model with an application to the measurement of change," Psychometrika, Springer;The Psychometric Society, vol. 56(4), pages 637-651, December.
    2. Bijlsma Ineke & van den Brakel Jan & van der Velden Rolf & Allen Jim, 2020. "Estimating Literacy Levels at a Detailed Regional Level: an Application Using Dutch Data," Journal of Official Statistics, Sciendo, vol. 36(2), pages 251-274, June.
    3. Jie Huang & Haiming Zhou & Nader Ebrahimi, 2022. "Bayesian Bivariate Cure Rate Models Using Copula Functions," International Journal of Statistics and Probability, Canadian Center of Science and Education, vol. 11(3), pages 1-9, May.
    4. Kubokawa, Tatsuya & Nagashima, Bui, 2012. "Parametric bootstrap methods for bias correction in linear mixed models," Journal of Multivariate Analysis, Elsevier, vol. 106(C), pages 1-16.
    5. Myung-Jae Hwang & Jong-Hun Kim & Hae-Kwan Cheong, 2020. "Short-Term Impacts of Ambient Air Pollution on Health-Related Quality of Life: A Korea Health Panel Survey Study," IJERPH, MDPI, vol. 17(23), pages 1-11, December.
    6. Roberto Burro & Riccardo Sartori & Giulio Vidotto, 2011. "The method of constant stimuli with three rating categories and the use of Rasch models," Quality & Quantity: International Journal of Methodology, Springer, vol. 45(1), pages 43-58, January.
    7. J. N. K. Rao, 2015. "Inferential issues in model-based small area estimation: some new developments," Statistics in Transition new series, Główny Urząd Statystyczny (Polska), vol. 16(4), pages 491-510, December.
    8. David Andrich, 1995. "Further remarks on nondichotomization of graded responses," Psychometrika, Springer;The Psychometric Society, vol. 60(1), pages 37-46, March.
    9. Tsonaka, R. & Moustaki, I., 2007. "Parameter constraints in generalized linear latent variable models," Computational Statistics & Data Analysis, Elsevier, vol. 51(9), pages 4164-4177, May.
    10. T. W. G. Meer & E. Ouattara, 2019. "Putting ‘political’ back in political trust: an IRT test of the unidimensionality and cross-national equivalence of political trust measures," Quality & Quantity: International Journal of Methodology, Springer, vol. 53(6), pages 2983-3002, November.
    11. Simona Buscemi & Antonella Plaia, 2020. "Model selection in linear mixed-effect models," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 104(4), pages 529-575, December.
    12. Joseph Paul Lavallee & Bruno Di Giusto & Tai-Yi Yu & Su-Pin Hung, 2022. "Reliability and Validity of Widely Used International Surveys on the Environment," Sustainability, MDPI, vol. 14(18), pages 1-22, September.
    13. Masahiro Kojima & Tatsuya Kubokawa, 2013. "Bartlett Adjustments for Hypothesis Testing in Linear Models with General Error Covariance Matrices," CIRJE F-Series CIRJE-F-884, CIRJE, Faculty of Economics, University of Tokyo.
    14. Sun-Joo Cho & Paul Boeck & Susan Embretson & Sophia Rabe-Hesketh, 2014. "Additive Multilevel Item Structure Models with Random Residuals: Item Modeling for Explanation and Item Generation," Psychometrika, Springer;The Psychometric Society, vol. 79(1), pages 84-104, January.
    15. Ward, Eric J., 2008. "A review and comparison of four commonly used Bayesian and maximum likelihood model selection tools," Ecological Modelling, Elsevier, vol. 211(1), pages 1-10.
    16. María José Lombardía & Esther López‐Vizcaíno & Cristina Rueda, 2017. "Mixed generalized Akaike information criterion for small area models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(4), pages 1229-1252, October.
    17. Mojtaba Ganjali & Taban Baghfalaki, 2018. "Application of Penalized Mixed Model in Identification of Genes in Yeast Cell-Cycle Gene Expression Data," Biostatistics and Biometrics Open Access Journal, Juniper Publishers Inc., vol. 6(2), pages 38-41, April.
    18. Kauermann, Goran & Xu, Ronghui & Vaida, Florin, 2008. "Stacked Laplace-EM algorithm for duration models with time-varying and random effects," Computational Statistics & Data Analysis, Elsevier, vol. 52(5), pages 2514-2528, January.
    19. Philipp F. M. Baumann & Enzo Rossi & Alexander Volkmann, 2020. "What Drives Inflation and How: Evidence from Additive Mixed Models Selected by cAIC," Papers 2006.06274, arXiv.org, revised Aug 2022.
    20. Jiang, Jiming & Nguyen, Thuan & Rao, J. Sunil, 2009. "A simplified adaptive fence procedure," Statistics & Probability Letters, Elsevier, vol. 79(5), pages 625-629, March.

    More about this item

    Keywords

    Cross-national comparability; method bias; item response time; Rasch model; differential item functioning;
    All these keywords.

    JEL classification:

    • Z0 - Other Special Topics - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hig:wpaper:57edu2020. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Shamil Abdulaev or Shamil Abdulaev (email available below). General contact details of provider: https://edirc.repec.org/data/hsecoru.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.