IDEAS home Printed from https://ideas.repec.org/a/sae/somere/v52y2023i3p1401-1419.html

Why Measurement Invariance is Important in Comparative Research. A Response to Welzel et al. (2021)

Author

Listed:
  • Bart Meuleman
  • Tomasz Żółtak
  • Artur Pokropek
  • Eldad Davidov
  • Bengt Muthén
  • Daniel L. Oberski
  • Jaak Billiet
  • Peter Schmidt

Abstract

Welzel et al. (2021) claim that non-invariance of instruments is inconclusive and inconsequential in the field for cross-cultural value measurement. In this response, we contend that several key arguments on which Welzel et al. (2021) base their critique of invariance testing are conceptually and statistically incorrect. First, Welzel et al. (2021) claim that value measurement follows a formative rather than reflective logic. Yet they do not provide sufficient theoretical arguments for this conceptualization, nor do they discuss the disadvantages of this approach for validation of instruments. Second, their claim that strong inter-item correlations cannot be retrieved when means are close to the endpoint of scales ignores the existence of factor-analytic approaches for ordered-categorical indicators. Third, Welzel et al. (2021) propose that rather than of relying on invariance tests, comparability can be assessed by studying the connection with theoretically related constructs. However, their proposal ignores that external validation through nomological linkages hinges on the assumption of comparability. By means of two examples, we illustrate that violating the assumptions of measurement invariance can distort conclusions substantially. Following the advice of Welzel et al. (2021) implies discarding a tool that has proven to be very useful for comparativists.

Suggested Citation

  • Bart Meuleman & Tomasz Żółtak & Artur Pokropek & Eldad Davidov & Bengt Muthén & Daniel L. Oberski & Jaak Billiet & Peter Schmidt, 2023. "Why Measurement Invariance is Important in Comparative Research. A Response to Welzel et al. (2021)," Sociological Methods & Research, , vol. 52(3), pages 1401-1419, August.
    • Handle: RePEc:sae:somere:v:52:y:2023:i:3:p:1401-1419
    DOI: 10.1177/00491241221091755
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/00491241221091755
    Download Restriction: no
    File URL: https://libkey.io/10.1177/00491241221091755?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Anna Ruelens & Bart Meuleman & Ides Nicaise, 2018. "Examining Measurement Isomorphism of Multilevel Constructs: The Case of Political Trust," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 140(3), pages 907-927, December.
    2. Yoshio Takane & Jan Leeuw, 1987. "On the relationship between item response theory and factor analysis of discretized variables," Psychometrika, Springer;The Psychometric Society, vol. 52(3), pages 393-408, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Alexander Robitzsch, 2025. "Comparing Robust Haberman Linking and Invariance Alignment," Stats, MDPI, vol. 8(1), pages 1-15, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. de Leeuw, Jan, 2006. "Principal component analysis of binary data by iterated singular value decomposition," Computational Statistics & Data Analysis, Elsevier, vol. 50(1), pages 21-39, January.
    2. Christoph T. Weiss, 2012. "Persistent Attitudes and Behaviors," "Marco Fanno" Working Papers 0143, Dipartimento di Scienze Economiche "Marco Fanno".
    3. Francesco Bartolucci & Ivonne Solis-Trapala, 2010. "Multidimensional Latent Markov Models in a Developmental Study of Inhibitory Control and Attentional Flexibility in Early Childhood," Psychometrika, Springer;The Psychometric Society, vol. 75(4), pages 725-743, December.
    4. Molenaar, Dylan & Kő, Natasa & Rózsa, Sandor & Mészáros, Andrea, 2017. "Differentiation of cognitive abilities in the WAIS-IV at the item level," Intelligence, Elsevier, vol. 65(C), pages 48-59.
    5. Yang Yixin & Lü Xin & Ma Jian & Qiao Han, 2014. "A Robust Factor Analysis Model for Dichotomous Data," Journal of Systems Science and Information, De Gruyter, vol. 2(5), pages 437-450, October.
    6. T. W. G. Meer & E. Ouattara, 2019. "Putting ‘political’ back in political trust: an IRT test of the unidimensionality and cross-national equivalence of political trust measures," Quality & Quantity: International Journal of Methodology, Springer, vol. 53(6), pages 2983-3002, November.
    7. Chun Wang & Steven W. Nydick, 2020. "On Longitudinal Item Response Theory Models: A Didactic," Journal of Educational and Behavioral Statistics, , vol. 45(3), pages 339-368, June.
    8. Yong Luo & Khaleel Al-Harbi, 2016. "The Utility of the Bifactor Method for Unidimensionality Assessment When Other Methods Disagree," SAGE Open, , vol. 6(4), pages 21582440166, October.
    9. Thum, Anna-Elisabeth, 2013. "Psychology in econometric models: conceptual and methodological foundations," MPRA Paper 52293, University Library of Munich, Germany.
    10. Michael Edwards, 2010. "A Markov Chain Monte Carlo Approach to Confirmatory Item Factor Analysis," Psychometrika, Springer;The Psychometric Society, vol. 75(3), pages 474-497, September.
    11. Edward Haertel, 1990. "Continuous and discrete latent structure models for item response data," Psychometrika, Springer;The Psychometric Society, vol. 55(3), pages 477-494, September.
    12. Royce Anders & William Batchelder, 2015. "Cultural Consensus Theory for the Ordinal Data Case," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 151-181, March.
    13. Yikun Luo & Qipeng Chen & Jianyong Chen & Peida Zhan, 2024. "Development and validation of two shortened anxiety sensitive index-3 scales based on item response theory," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 11(1), pages 1-7, December.
    14. Yang Liu, 2020. "A Riemannian Optimization Algorithm for Joint Maximum Likelihood Estimation of High-Dimensional Exploratory Item Factor Analysis," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 439-468, June.
    15. Edward R. Maguire & Todd Armstrong & Devon Johnson, 2017. "The Structure of Citizen Perceptions of Crime and Disorder: New Insights from a Caribbean Community," Journal of Quantitative Criminology, Springer, vol. 33(4), pages 675-699, December.
    16. Anna Brown, 2016. "Item Response Models for Forced-Choice Questionnaires: A Common Framework," Psychometrika, Springer;The Psychometric Society, vol. 81(1), pages 135-160, March.
    17. Alexander Robitzsch, 2024. "A Comparison of Limited Information Estimation Methods for the Two-Parameter Normal-Ogive Model with Locally Dependent Items," Stats, MDPI, vol. 7(3), pages 1-16, June.
    18. Mayssun El-Attar, 2013. "Trust, child care technology choice and female labor force participation," Review of Economics of the Household, Springer, vol. 11(4), pages 507-544, December.
    19. Jochen Ranger & Kay Brauer, 2022. "On the Generalized S − X 2 –Test of Item Fit: Some Variants, Residuals, and a Graphical Visualization," Journal of Educational and Behavioral Statistics, , vol. 47(2), pages 202-230, April.
    20. Yu-Wei Chang & Nan-Jung Hsu & Rung-Ching Tsai, 2017. "Unifying Differential Item Functioning in Factor Analysis for Categorical Data Under a Discretization of a Normal Variant," Psychometrika, Springer;The Psychometric Society, vol. 82(2), pages 382-406, June.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:somere:v:52:y:2023:i:3:p:1401-1419. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.