IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v82y2017i1d10.1007_s11336-016-9543-8.html
   My bibliography  Save this article

Monitoring Countries in a Changing World: A New Look at DIF in International Surveys

Author

Listed:
  • Robert J. Zwitser

    (University of Amsterdam)

  • S. Sjoerd F. Glaser

    (University of Amsterdam)

  • Gunter Maris

    (University of Amsterdam
    Cito Institute for Educational Measurement)

Abstract

This paper discusses the issue of differential item functioning (DIF) in international surveys. DIF is likely to occur in international surveys. What is needed is a statistical approach that takes DIF into account, while at the same time allowing for meaningful comparisons between countries. Some existing approaches are discussed and an alternative is provided. The core of this alternative approach is to define the construct as a large set of items, and to report in terms of summary statistics. Since the data are incomplete, measurement models are used to complete the incomplete data. For that purpose, different models can be used across countries. The method is illustrated with PISA’s reading literacy data. The results indicate that this approach fits the data better than the current PISA methodology; however, the league tables are nearly identical. The implications for monitoring changes over time are discussed.

Suggested Citation

  • Robert J. Zwitser & S. Sjoerd F. Glaser & Gunter Maris, 2017. "Monitoring Countries in a Changing World: A New Look at DIF in International Surveys," Psychometrika, Springer;The Psychometric Society, vol. 82(1), pages 210-232, March.
    • Handle: RePEc:spr:psycho:v:82:y:2017:i:1:d:10.1007_s11336-016-9543-8
    DOI: 10.1007/s11336-016-9543-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11336-016-9543-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11336-016-9543-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Geoff Masters, 1982. "A rasch model for partial credit scoring," Psychometrika, Springer;The Psychometric Society, vol. 47(2), pages 149-174, June.
    2. Timo Bechger & Gunter Maris, 2015. "A Statistical Test for Differential Item Pair Functioning," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 317-340, June.
    3. Maarten Marsman & Gunter Maris & Timo Bechger & Cees Glas, 2016. "What can we learn from Plausible Values?," Psychometrika, Springer;The Psychometric Society, vol. 81(2), pages 274-289, June.
    4. Svend Kreiner & Karl Christensen, 2014. "Analyses of Model Fit and Robustness. A New Look at the PISA Scaling Model Underlying Ranking of Countries According to Reading Literacy," Psychometrika, Springer;The Psychometric Society, vol. 79(2), pages 210-231, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jeanne A. Teresi & Chun Wang & Marjorie Kleinman & Richard N. Jones & David J. Weiss, 2021. "Differential Item Functioning Analyses of the Patient-Reported Outcomes Measurement Information System (PROMIS®) Measures: Methods, Challenges, Advances, and Future Directions," Psychometrika, Springer;The Psychometric Society, vol. 86(3), pages 674-711, September.
    2. Daniele, Vittorio, 2021. "Socioeconomic inequality and regional disparities in educational achievement: The role of relative poverty," Intelligence, Elsevier, vol. 84(C).
    3. Alexander Robitzsch, 2020. "L p Loss Functions in Invariance Alignment and Haberman Linking with Few or Many Groups," Stats, MDPI, vol. 3(3), pages 1-38, August.
    4. Alexander Robitzsch & Oliver Lüdtke, 2022. "Mean Comparisons of Many Groups in the Presence of DIF: An Evaluation of Linking and Concurrent Scaling Approaches," Journal of Educational and Behavioral Statistics, , vol. 47(1), pages 36-68, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Curt Hagquist & Raili Välimaa & Nina Simonsen & Sakari Suominen, 2017. "Differential Item Functioning in Trend Analyses of Adolescent Mental Health – Illustrative Examples Using HBSC-Data from Finland," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 10(3), pages 673-691, September.
    2. César Merino-Soto & Gina Chávez-Ventura & Verónica López-Fernández & Guillermo M. Chans & Filiberto Toledano-Toledano, 2022. "Learning Self-Regulation Questionnaire (SRQ-L): Psychometric and Measurement Invariance Evidence in Peruvian Undergraduate Students," Sustainability, MDPI, vol. 14(18), pages 1-17, September.
    3. Roberto Burro & Riccardo Sartori & Giulio Vidotto, 2011. "The method of constant stimuli with three rating categories and the use of Rasch models," Quality & Quantity: International Journal of Methodology, Springer, vol. 45(1), pages 43-58, January.
    4. Chang, Hsin-Li & Yang, Cheng-Hua, 2008. "Explore airlines’ brand niches through measuring passengers’ repurchase motivation—an application of Rasch measurement," Journal of Air Transport Management, Elsevier, vol. 14(3), pages 105-112.
    5. Ivana Bassi & Matteo Carzedda & Enrico Gori & Luca Iseppi, 2022. "Rasch analysis of consumer attitudes towards the mountain product label," Agricultural and Food Economics, Springer;Italian Society of Agricultural Economics (SIDEA), vol. 10(1), pages 1-25, December.
    6. Hua-Hua Chang, 1996. "The asymptotic posterior normality of the latent trait for polytomous IRT models," Psychometrika, Springer;The Psychometric Society, vol. 61(3), pages 445-463, September.
    7. Thilo J. Ketschau & Janne Kleinhans, 2019. "Concept and Implementation of a Two-Stage Coding Scheme for the Development of Computer-Based Testing (CBT)-Items in Traditional Test Software," J, MDPI, vol. 2(1), pages 1-9, January.
    8. David J. Hessen, 2023. "Fitting and Testing Log-Linear Subpopulation Models with Known Support," Psychometrika, Springer;The Psychometric Society, vol. 88(3), pages 917-939, September.
    9. Chang, Hsin-Li & Wu, Shun-Cheng, 2008. "Exploring the vehicle dependence behind mode choice: Evidence of motorcycle dependence in Taipei," Transportation Research Part A: Policy and Practice, Elsevier, vol. 42(2), pages 307-320, February.
    10. C. Glas & Anna Dagohoy, 2007. "A Person Fit Test For Irt Models For Polytomous Items," Psychometrika, Springer;The Psychometric Society, vol. 72(2), pages 159-180, June.
    11. Giulia Grisolia & Umberto Lucia & Marco Filippo Torchio, 2022. "Sustainable Development and Workers Ability: Considerations on the Education Index in the Human Development Index," Sustainability, MDPI, vol. 14(14), pages 1-18, July.
    12. Jesper Tijmstra & Maria Bolsinova, 2019. "Bayes Factors for Evaluating Latent Monotonicity in Polytomous Item Response Theory Models," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 846-869, September.
    13. Salzberger, Thomas & Koller, Monika, 2013. "Towards a new paradigm of measurement in marketing," Journal of Business Research, Elsevier, vol. 66(9), pages 1307-1317.
    14. Fumiko Samejima, 1995. "Acceleration model in the heterogeneous case of the general graded response model," Psychometrika, Springer;The Psychometric Society, vol. 60(4), pages 549-572, December.
    15. Laura Zieger & John Jerrim & Jake Anders & Nikki Shure, 2020. "Conditioning: How background variables can influence PISA scores," CEPEO Working Paper Series 20-09, UCL Centre for Education Policy and Equalising Opportunities, revised Apr 2020.
    16. Richard N McNeely & Salissou Moutari & Samuel Arba-Mosquera & Shwetabh Verma & Jonathan E Moore, 2018. "An alternative application of Rasch analysis to assess data from ophthalmic patient-reported outcome instruments," PLOS ONE, Public Library of Science, vol. 13(6), pages 1-32, June.
    17. Francesca DE BATTISTI & Giovanna NICOLINI & Silvia SALINI, 2008. "Methodological overview of Rasch model and application in customer satisfaction survey data," Departmental Working Papers 2008-04, Department of Economics, Management and Quantitative Methods at Università degli Studi di Milano.
    18. Kuan-Yu Jin & Yi-Jhen Wu & Hui-Fang Chen, 2022. "A New Multiprocess IRT Model With Ideal Points for Likert-Type Items," Journal of Educational and Behavioral Statistics, , vol. 47(3), pages 297-321, June.
    19. Daniele, Vittorio, 2021. "Socioeconomic inequality and regional disparities in educational achievement: The role of relative poverty," Intelligence, Elsevier, vol. 84(C).
    20. van der Ark, L. Andries, 2012. "New Developments in Mokken Scale Analysis in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i05).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:82:y:2017:i:1:d:10.1007_s11336-016-9543-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.