IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v47y2022i1p36-68.html

Mean Comparisons of Many Groups in the Presence of DIF: An Evaluation of Linking and Concurrent Scaling Approaches

Author

Listed:
  • Alexander Robitzsch
  • Oliver Lüdtke

    (IPN—Leibniz Institute for Science and Mathematics Education, Centre for International Student Assessment)

Abstract

One of the primary goals of international large-scale assessments in education is the comparison of country means in student achievement. This article introduces a framework for discussing differential item functioning (DIF) for such mean comparisons. We compare three different linking methods: concurrent scaling based on full invariance, concurrent scaling based on partial invariance using the RMSD statistic, and robust and nonrobust linking approaches based on separate scaling. Furthermore, we analytically derive the bias in the country means of different linking methods in the presence of DIF. In a simulation study, we show that the partial invariance and robust linking approaches provide less biased country means than the full invariance approach in the case of biased items.

Suggested Citation

  • Alexander Robitzsch & Oliver Lüdtke, 2022. "Mean Comparisons of Many Groups in the Presence of DIF: An Evaluation of Linking and Concurrent Scaling Approaches," Journal of Educational and Behavioral Statistics, , vol. 47(1), pages 36-68, February.
    • Handle: RePEc:sae:jedbes:v:47:y:2022:i:1:p:36-68
    DOI: 10.3102/10769986211017479
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/10769986211017479
    Download Restriction: no
    File URL: https://libkey.io/10.3102/10769986211017479?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Roger Koenker, 2017. "Quantile Regression: 40 Years On," Annual Review of Economics, Annual Reviews, vol. 9(1), pages 155-176, September.
    2. Robert J. Zwitser & S. Sjoerd F. Glaser & Gunter Maris, 2017. "Monitoring Countries in a Changing World: A New Look at DIF in International Surveys," Psychometrika, Springer;The Psychometric Society, vol. 82(1), pages 210-232, March.
    3. William Meredith, 1993. "Measurement invariance, factor analysis and factorial invariance," Psychometrika, Springer;The Psychometric Society, vol. 58(4), pages 525-543, December.
    4. Michela Battauz, 2019. "On Wald tests for differential item functioning detection," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 28(1), pages 103-118, March.
    5. Jean-Paul Fox & Joris Mulder & Sandip Sinharay, 2017. "Bayes Factor Covariance Testing in Item Response Models," Psychometrika, Springer;The Psychometric Society, vol. 82(4), pages 979-1006, December.
    6. Michela Battauz, 2017. "Multiple Equating of Separate IRT Calibrations," Psychometrika, Springer;The Psychometric Society, vol. 82(3), pages 610-636, September.
    7. Stanislav Kolenikov, 2010. "Resampling variance estimation for complex survey data," Stata Journal, StataCorp LLC, vol. 10(2), pages 165-199, June.
    8. Roger Koenker, 2017. "Quantile regression 40 years on," CeMMAP working papers CWP36/17, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    9. Timo Bechger & Gunter Maris, 2015. "A Statistical Test for Differential Item Pair Functioning," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 317-340, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alexander Robitzsch, 2020. "L p Loss Functions in Invariance Alignment and Haberman Linking with Few or Many Groups," Stats, MDPI, vol. 3(3), pages 1-38, August.
    2. Palaios, Panagiotis & Triantafillou, Anna, 2025. "Exploring spillover effects in the four shipping markets: Theory and empirical evidence from bulk shipping," Transport Policy, Elsevier, vol. 170(C), pages 75-91.
    3. Kaushik Gala & Andreas Schwab, 2026. "A Methodological Guide for Quantitative Analysis of Star Performance in Entrepreneurship," Entrepreneurship Theory and Practice, , vol. 50(3), pages 931-973, May.
    4. Ahmed, Walid M.A., 2021. "Stock market reactions to upside and downside volatility of Bitcoin: A quantile analysis," The North American Journal of Economics and Finance, Elsevier, vol. 57(C).
    5. Anne M. Lausier & Shaleen Jain, 2018. "Diversity in global patterns of observed precipitation variability and change on river basin scales," Climatic Change, Springer, vol. 149(2), pages 261-275, July.
    6. Guillen, Montserrat & Bermúdez, Lluís & Pitarque, Albert, 2021. "Joint generalized quantile and conditional tail expectation regression for insurance risk analysis," Insurance: Mathematics and Economics, Elsevier, vol. 99(C), pages 1-8.
    7. Alexander Robitzsch, 2023. "Linking Error in the 2PL Model," J, MDPI, vol. 6(1), pages 1-27, January.
    8. Chen, Zhao & Cheng, Vivian Xinyi & Liu, Xu, 2024. "Reprint: Hypothesis testing on high dimensional quantile regression," Journal of Econometrics, Elsevier, vol. 239(2).
    9. repec:rim:rimwps:23-03 is not listed on IDEAS
    10. Chong-Chuo Chang & Oshamah Lin Lin & Oshamah Yu-Cheng Chang & Oshamah Kun-Zhan Hsu, 2023. "Impact of Financial Liberalization on Firm Risk," Advances in Decision Sciences, Asia University, Taiwan, vol. 27(3), pages 14-45, September.
    11. Rita Pimentel & Morten Risstad & Sjur Westgaard, 2022. "Predicting interest rate distributions using PCA & quantile regression," Digital Finance, Springer, vol. 4(4), pages 291-311, December.
    12. Christian L. E. Franzke & Herminia Torelló i Sentelles, 2020. "Risk of extreme high fatalities due to weather and climate hazards and its connection to large-scale climate variability," Climatic Change, Springer, vol. 162(2), pages 507-525, September.
    13. Priyanshu Chavda & Dhyani Mehta, 2026. "Impact of fossil fuel subsidies, natural resource rent and corruption on renewable energy in Middle Eastern and North African countries," Asia-Pacific Journal of Regional Science, Springer, vol. 10(1), pages 1-23, March.
    14. Damian Clarke & Manuel Llorca Jaña & Daniel Pailañir, 2023. "The use of quantile methods in economic history," Historical Methods: A Journal of Quantitative and Interdisciplinary History, Taylor & Francis Journals, vol. 56(2), pages 115-132, April.
    15. Wang, Xuqin & Li, Muyi, 2023. "Bootstrapping the transformed goodness-of-fit test on heavy-tailed GARCH models," Computational Statistics & Data Analysis, Elsevier, vol. 184(C).
    16. Abdellah Atanane & Abdallah Mkhadri & Karim Oualkacha, 2025. "An efficient hybrid approach of quantile and expectile regression," Statistical Papers, Springer, vol. 66(6), pages 1-45, October.
    17. Fabio Bellini & Ilaria Peri, 2021. "An axiomatization of $\Lambda$-quantiles," Papers 2109.02360, arXiv.org, revised Jan 2022.
    18. Sulkhan Chavleishvili & Simone Manganelli, 2024. "Forecasting and stress testing with quantile vector autoregression," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 39(1), pages 66-85, January.
    19. Evangelista, Rui & Silva, João Andrade e & Ramalho, Esmeralda A., 2022. "How heterogeneous is the impact of energy efficiency on dwelling prices? Evidence from the application of the unconditional quantile hedonic model to the Portuguese residential market," Energy Economics, Elsevier, vol. 109(C).
    20. Sun, Zhaoyang & Liu, Ling & Pan, Runquan & Wang, Yiwei & Zhang, Bingbing, 2025. "Tourism and economic growth: The role of institutional quality," International Review of Economics & Finance, Elsevier, vol. 98(C).
    21. Chen, Xiaohong & Pouzo, Demian & Powell, James L., 2019. "Penalized sieve GEL for weighted average derivatives of nonparametric quantile IV regressions," Journal of Econometrics, Elsevier, vol. 213(1), pages 30-53.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:47:y:2022:i:1:p:36-68. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.