IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v88y2023i4d10.1007_s11336-023-09930-9.html
   My bibliography  Save this article

DIF Statistical Inference Without Knowing Anchoring Items

Author

Listed:
  • Yunxiao Chen

    (London School of Economics and Political Science)

  • Chengcheng Li

    (University of Michigan)

  • Jing Ouyang

    (University of Michigan)

  • Gongjun Xu

    (University of Michigan)

Abstract

Establishing the invariance property of an instrument (e.g., a questionnaire or test) is a key step for establishing its measurement validity. Measurement invariance is typically assessed by differential item functioning (DIF) analysis, i.e., detecting DIF items whose response distribution depends not only on the latent trait measured by the instrument but also on the group membership. DIF analysis is confounded by the group difference in the latent trait distributions. Many DIF analyses require knowing several anchor items that are DIF-free in order to draw inferences on whether each of the rest is a DIF item, where the anchor items are used to identify the latent trait distributions. When no prior information on anchor items is available, or some anchor items are misspecified, item purification methods and regularized estimation methods can be used. The former iteratively purifies the anchor set by a stepwise model selection procedure, and the latter selects the DIF-free items by a LASSO-type regularization approach. Unfortunately, unlike the methods based on a correctly specified anchor set, these methods are not guaranteed to provide valid statistical inference (e.g., confidence intervals and p-values). In this paper, we propose a new method for DIF analysis under a multiple indicators and multiple causes (MIMIC) model for DIF. This method adopts a minimal $$L_1$$ L 1 norm condition for identifying the latent trait distributions. Without requiring prior knowledge about an anchor set, it can accurately estimate the DIF effects of individual items and further draw valid statistical inferences for quantifying the uncertainty. Specifically, the inference results allow us to control the type-I error for DIF detection, which may not be possible with item purification and regularized estimation methods. We conduct simulation studies to evaluate the performance of the proposed method and compare it with the anchor-set-based likelihood ratio test approach and the LASSO approach. The proposed method is applied to analysing the three personality scales of the Eysenck personality questionnaire-revised (EPQ-R).

Suggested Citation

  • Yunxiao Chen & Chengcheng Li & Jing Ouyang & Gongjun Xu, 2023. "DIF Statistical Inference Without Knowing Anchoring Items," Psychometrika, Springer;The Psychometric Society, vol. 88(4), pages 1097-1122, December.
    • Handle: RePEc:spr:psycho:v:88:y:2023:i:4:d:10.1007_s11336-023-09930-9
    DOI: 10.1007/s11336-023-09930-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11336-023-09930-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11336-023-09930-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Robin Shealy & William Stout, 1993. "A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF," Psychometrika, Springer;The Psychometric Society, vol. 58(2), pages 159-194, June.
    2. Alexander Robitzsch, 2020. "L p Loss Functions in Invariance Alignment and Haberman Linking with Few or Many Groups," Stats, MDPI, vol. 3(3), pages 1-38, August.
    3. Gerhard Tutz & Gunther Schauberger, 2015. "A Penalty Approach to Differential Item Functioning in Rasch Models," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 21-43, March.
    4. Nambury Raju, 1988. "The area between two item characteristic curves," Psychometrika, Springer;The Psychometric Society, vol. 53(4), pages 495-502, December.
    5. Carolin Strobl & Julia Kopf & Achim Zeileis, 2015. "Rasch Trees: A New Method for Detecting Differential Item Functioning in the Rasch Model," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 289-316, June.
    6. Timo Bechger & Gunter Maris, 2015. "A Statistical Test for Differential Item Pair Functioning," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 317-340, June.
    7. Gerhard Tutz & Moritz Berger, 2016. "Item-focussed Trees for the Identification of Items in Differential Item Functioning," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 727-750, September.
    8. Goldberger, Arthur S, 1972. "Structural Equation Methods in the Social Sciences," Econometrica, Econometric Society, vol. 40(6), pages 979-1001, November.
    9. Steenkamp, Jan-Benedict E M & Baumgartner, Hans, 1998. "Assessing Measurement Invariance in Cross-National Consumer Research," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 25(1), pages 78-90, June.
    10. Ke-Hai Yuan & Hongyun Liu & Yuting Han, 2021. "Differential Item Functioning Analysis Without A Priori Information on Anchor Items: QQ Plots and Graphical Test," Psychometrika, Springer;The Psychometric Society, vol. 86(2), pages 345-377, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gabriel Wallin & Yunxiao Chen & Irini Moustaki, 2024. "DIF Analysis with Unknown Groups and Anchor Items," Psychometrika, Springer;The Psychometric Society, vol. 89(1), pages 267-295, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chen, Yunxiao & Li, Chengcheng & Ouyang, Jing & Xu, Gongjun, 2023. "DIF statistical inference without knowing anchoring items," LSE Research Online Documents on Economics 119923, London School of Economics and Political Science, LSE Library.
    2. Ke-Hai Yuan & Hongyun Liu & Yuting Han, 2021. "Differential Item Functioning Analysis Without A Priori Information on Anchor Items: QQ Plots and Graphical Test," Psychometrika, Springer;The Psychometric Society, vol. 86(2), pages 345-377, June.
    3. Gabriel Wallin & Yunxiao Chen & Irini Moustaki, 2024. "DIF Analysis with Unknown Groups and Anchor Items," Psychometrika, Springer;The Psychometric Society, vol. 89(1), pages 267-295, March.
    4. Wallin, Gabriel & Chen, Yunxiao & Moustaki, Irini, 2024. "DIF analysis with unknown groups and anchor items," LSE Research Online Documents on Economics 121991, London School of Economics and Political Science, LSE Library.
    5. Ting Wang & Carolin Strobl & Achim Zeileis & Edgar C. Merkle, 2018. "Score-Based Tests of Differential Item Functioning via Pairwise Maximum Likelihood Estimation," Psychometrika, Springer;The Psychometric Society, vol. 83(1), pages 132-155, March.
    6. Alexander Robitzsch, 2020. "L p Loss Functions in Invariance Alignment and Haberman Linking with Few or Many Groups," Stats, MDPI, vol. 3(3), pages 1-38, August.
    7. Gerhard Tutz & Moritz Berger, 2016. "Item-focussed Trees for the Identification of Items in Differential Item Functioning," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 727-750, September.
    8. Paul Boeck & Sun-Joo Cho, 2021. "Not all DIF is shaped similarly," Psychometrika, Springer;The Psychometric Society, vol. 86(3), pages 712-716, September.
    9. Jeanne A. Teresi & Chun Wang & Marjorie Kleinman & Richard N. Jones & David J. Weiss, 2021. "Differential Item Functioning Analyses of the Patient-Reported Outcomes Measurement Information System (PROMIS®) Measures: Methods, Challenges, Advances, and Future Directions," Psychometrika, Springer;The Psychometric Society, vol. 86(3), pages 674-711, September.
    10. Youmi Suk & Kyung T. Han, 2024. "A Psychometric Framework for Evaluating Fairness in Algorithmic Decision Making: Differential Algorithmic Functioning," Journal of Educational and Behavioral Statistics, , vol. 49(2), pages 151-172, April.
    11. Minjeong Jeon & Frank Rijmen & Sophia Rabe-Hesketh, 2013. "Modeling Differential Item Functioning Using a Generalization of the Multiple-Group Bifactor Model," Journal of Educational and Behavioral Statistics, , vol. 38(1), pages 32-60, February.
    12. Jorgensen, Bradley S. & Syme, Geoffrey J., 2000. "Protest responses and willingness to pay: attitude toward paying for stormwater pollution abatement," Ecological Economics, Elsevier, vol. 33(2), pages 251-265, May.
    13. Alexander Robitzsch, 2025. "Comparing Robust Haberman Linking and Invariance Alignment," Stats, MDPI, vol. 8(1), pages 1-15, January.
    14. David Magis & Francis Tuerlinckx & Paul De Boeck, 2015. "Detection of Differential Item Functioning Using the Lasso Approach," Journal of Educational and Behavioral Statistics, , vol. 40(2), pages 111-135, April.
    15. Terry A. Ackerman & Ye Ma, 2024. "Examining Differential Item Functioning from a Multidimensional IRT Perspective," Psychometrika, Springer;The Psychometric Society, vol. 89(1), pages 4-41, March.
    16. K. B. S. Huth & L. J. Waldorp & J. Luigjes & A. E. Goudriaan & R. J. Holst & M. Marsman, 2022. "A Note on the Structural Change Test in Highly Parameterized Psychometric Models," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 1064-1080, September.
    17. Peter F. Halpin, 2024. "Differential Item Functioning via Robust Scaling," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 796-821, September.
    18. Tsukasa Kato, 2021. "Measurement Invariance in the Center for Epidemiologic Studies-Depression (CES-D) Scale among English-Speaking Whites and Asians," IJERPH, MDPI, vol. 18(10), pages 1-10, May.
    19. Mao, Lu, 2022. "Identification of the outcome distribution and sensitivity analysis under weak confounder–instrument interaction," Statistics & Probability Letters, Elsevier, vol. 189(C).
    20. Janina Isabel Steinert & Lucie Dale Cluver & G. J. Melendez-Torres & Sebastian Vollmer, 2018. "One Size Fits All? The Validity of a Composite Poverty Index Across Urban and Rural Households in South Africa," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 136(1), pages 51-72, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:88:y:2023:i:4:d:10.1007_s11336-023-09930-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.