IDEAS home Printed from https://ideas.repec.org/p/inn/wpaper/2016-05.html
   My bibliography  Save this paper

Score-Based Tests of Differential Item Functioning in the Two-Parameter Model

Author

Listed:
  • Ting Wang
  • Carolin Strobl
  • Achim Zeileis
  • Edgar C. Merkle

Abstract

Measurement invariance is a fundamental assumption in item response theory models, where the relationship between a latent construct (ability) and observed item responses is of interest. Violation of this assumption would render the scale misinterpreted or cause systematic bias against certain groups of people. While a number of methods have been proposed to detect measurement invariance violations, they typically require advance definition of problematic item parameters and respondent grouping information. However, these pieces of information are typically unknown in practice. As an alternative, this paper focuses on a family of recently-proposed tests based on stochastic processes of casewise derivatives of the likelihood function (i.e., scores). These score-based tests only require estimation of the null model (when measurement invariance is assumed to hold), and they have been previously applied in factor-analytic, continuous data contexts as well as in models of the Rasch family. In this paper, we aim to extend these tests to two parameter item response models estimated via maximum likelihood. The tests' theoretical background and implementation are detailed, and the tests' abilities to identify problematic item parameters are studied via simulation. An empirical example illustrating the tests' use in practice is also provided.

Suggested Citation

  • Ting Wang & Carolin Strobl & Achim Zeileis & Edgar C. Merkle, 2016. "Score-Based Tests of Differential Item Functioning in the Two-Parameter Model," Working Papers 2016-05, Faculty of Economics and Statistics, Universität Innsbruck.
  • Handle: RePEc:inn:wpaper:2016-05
    as

    Download full text from publisher

    File URL: https://www2.uibk.ac.at/downloads/c4041030/wpaper/2016-05.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Karl Jöreskog, 1990. "New developments in LISREL: analysis of ordinal variables using polychoric correlations and weighted least squares," Quality & Quantity: International Journal of Methodology, Springer, vol. 24(4), pages 387-404, November.
    2. Katsikatsou, Myrsini & Moustaki, Irini & Yang-Wallentin, Fan & Jöreskog, Karl G., 2012. "Pairwise likelihood estimation for factor analysis models with ordinal data," Computational Statistics & Data Analysis, Elsevier, vol. 56(12), pages 4243-4258.
    3. Chalmers, R. Philip, 2012. "mirt: A Multidimensional Item Response Theory Package for the R Environment," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i06).
    4. Zeileis, Achim & Leisch, Friedrich & Hornik, Kurt & Kleiber, Christian, 2002. "strucchange: An R Package for Testing for Structural Change in Linear Regression Models," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 7(i02).
    5. David Thissen, 1982. "Marginal maximum likelihood estimation for the one-parameter logistic model," Psychometrika, Springer;The Psychometric Society, vol. 47(2), pages 175-186, June.
    6. Achim Zeileis & Kurt Hornik, 2007. "Generalized M‐fluctuation tests for parameter instability," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 61(4), pages 488-508, November.
    7. Carolin Strobl & Julia Kopf & Achim Zeileis, 2015. "Rasch Trees: A New Method for Detecting Differential Item Functioning in the Rasch Model," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 289-316, June.
    8. Andrews, Donald W K, 1993. "Tests for Parameter Instability and Structural Change with Unknown Change Point," Econometrica, Econometric Society, vol. 61(4), pages 821-856, July.
    9. Rosseel, Yves, 2012. "lavaan: An R Package for Structural Equation Modeling," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i02).
    10. Katsikatsou, Myrsini & Moustaki, Irini & Yang-Wallentin, Fan & Jöreskog, Karl G., 2012. "Pairwise likelihood estimation for factor analysis models with ordinal data," LSE Research Online Documents on Economics 43182, London School of Economics and Political Science, LSE Library.
    11. Albert Satorra, 1989. "Alternative test criteria in covariance structure analysis: A unified approach," Psychometrika, Springer;The Psychometric Society, vol. 54(1), pages 131-151, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ting Wang & Carolin Strobl & Achim Zeileis & Edgar C. Merkle, 2018. "Score-Based Tests of Differential Item Functioning via Pairwise Maximum Likelihood Estimation," Psychometrika, Springer;The Psychometric Society, vol. 83(1), pages 132-155, March.
    2. Edgar Merkle & Achim Zeileis, 2013. "Tests of Measurement Invariance Without Subgroups: A Generalization of Classical Methods," Psychometrika, Springer;The Psychometric Society, vol. 78(1), pages 59-82, January.
    3. Edgar Merkle & Jinyan Fan & Achim Zeileis, 2014. "Testing for Measurement Invariance with Respect to an Ordinal Variable," Psychometrika, Springer;The Psychometric Society, vol. 79(4), pages 569-584, October.
    4. Ting Wang & Edgar C. Merkle & Achim Zeileis, 2013. "Score-Based Tests of Measurement Invariance: Use in Practice," Working Papers 2013-33, Faculty of Economics and Statistics, Universität Innsbruck.
    5. K. B. S. Huth & L. J. Waldorp & J. Luigjes & A. E. Goudriaan & R. J. Holst & M. Marsman, 2022. "A Note on the Structural Change Test in Highly Parameterized Psychometric Models," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 1064-1080, September.
    6. Ting Wang & Benjamin Graves & Yves Rosseel & Edgar C. Merkle, 2022. "Computation and application of generalized linear mixed model derivatives using lme4," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 1173-1193, September.
    7. Grün, Bettina & Kosmidis, Ioannis & Zeileis, Achim, 2012. "Extended Beta Regression in R: Shaken, Stirred, Mixed, and Partitioned," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i11).
    8. Payton J. Jones & Patrick Mair & Thorsten Simon & Achim Zeileis, 2020. "Network Trees: A Method for Recursively Partitioning Covariance Structures," Psychometrika, Springer;The Psychometric Society, vol. 85(4), pages 926-945, December.
    9. Alexander Robitzsch, 2021. "A Comprehensive Simulation Study of Estimation Methods for the Rasch Model," Stats, MDPI, vol. 4(4), pages 1-23, October.
    10. Zachary F. Fisher & Kenneth A. Bollen, 2020. "An Instrumental Variable Estimator for Mixed Indicators: Analytic Derivatives and Alternative Parameterizations," Psychometrika, Springer;The Psychometric Society, vol. 85(3), pages 660-683, September.
    11. Thomas Windberger & Achim Zeileis, 2011. "Structural Breaks in Inflation Dynamics within the European Monetary Union," Working Papers 2011-12, Faculty of Economics and Statistics, Universität Innsbruck.
    12. Myrsini Katsikatsou & Irini Moustaki, 2016. "Pairwise Likelihood Ratio Tests and Model Selection Criteria for Structural Equation Models with Ordinal Variables," Psychometrika, Springer;The Psychometric Society, vol. 81(4), pages 1046-1068, December.
    13. Jones, Payton J. & Mair, Patrick & Simon, Thorsten & Zeileis, Achim, 2019. "Network Model Trees," OSF Preprints ha4cw, Center for Open Science.
    14. Papageorgiou, Ioulia & Moustaki, Irini, 2019. "Sampling of pairs in pairwise likelihood estimation for latent variable models with categorical observed variables," LSE Research Online Documents on Economics 87592, London School of Economics and Political Science, LSE Library.
    15. Edgar C. Merkle & Achim Zeileis, 2011. "Generalized Measurement Invariance Tests with Application to Factor Analysis," Working Papers 2011-09, Faculty of Economics and Statistics, Universität Innsbruck.
    16. Zeileis, Achim, 2006. "Implementing a class of structural change tests: An econometric computing approach," Computational Statistics & Data Analysis, Elsevier, vol. 50(11), pages 2987-3008, July.
    17. Cervantes, Víctor H., 2017. "DFIT: An R Package for Raju's Differential Functioning of Items and Tests Framework," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 76(i05).
    18. Yang, Yang & Liu, Qing & Chang, Chia-Hsun, 2023. "China-Europe freight transportation under the first wave of COVID-19 pandemic and government restriction measures," Research in Transportation Economics, Elsevier, vol. 97(C).
    19. Abhijit Sharma & Kelvin G Balcombe & Iain M Fraser, 2009. "Non-renewable resource prices: Structural breaks and long term trends," Economics Bulletin, AccessEcon, vol. 29(2), pages 805-819.
    20. Fabio Clementi & Marco Gallegati & Mauro Gallegati, 2015. "Growth and Cycles of the Italian Economy Since 1861: The New Evidence," Italian Economic Journal: A Continuation of Rivista Italiana degli Economisti and Giornale degli Economisti, Springer;Società Italiana degli Economisti (Italian Economic Association), vol. 1(1), pages 25-59, March.

    More about this item

    Keywords

    measurement invariance; item response theory; factor analysis; 2PL model; differential item functioning;
    All these keywords.

    JEL classification:

    • C30 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C87 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Econometric Software

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inn:wpaper:2016-05. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Janette Walde (email available below). General contact details of provider: https://edirc.repec.org/data/fuibkat.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.