IDEAS home Printed from https://ideas.repec.org/p/osf/socarx/mbgxs_v1.html

The Illusion of Precision: Interpreting International Rankings Within Their Measurement Limits

Author

Listed:
  • Kankaras, Milos

Abstract

International rankings publish precise country positions. We ask which comparisons the evidence behind five of them supports, by reproducing published tables, optimising weights, applying multiplicity-adjusted rank inference and two source checks, within a framework separating invalid measurement, insufficient validation, normative trade-offs, underrepresentation of uncertainty and misleading communication. In every programme the adjacent ordering is unsupported, each for a different reason: Report Card 19 trades off dimensions in all 35 adjacent pairs; World Happiness Report countries move a median of 30 places when affect replaces life evaluation; 23 of 38 countries can lead the Better Life Index under unrestricted weights; retrospective rescoring moves Global Health Security Index countries a median of seven places within one year; only 4 of 80 adjacent PISA pairs are distinguishable. Broad contrasts survive throughout. Two unrepresented uncertainties are documented: rotated administration behind Report Card 19's social-skills indicator, and unlinked assessment vintages behind the Human Capital Index.

Suggested Citation

  • Kankaras, Milos, 2026. "The Illusion of Precision: Interpreting International Rankings Within Their Measurement Limits," SocArXiv mbgxs_v1, Center for Open Science.
    • Handle: RePEc:osf:socarx:mbgxs_v1
    DOI: 10.31235/osf.io/mbgxs_v1
    as

    Download full text from publisher

    File URL: https://osf.io/download/6aa442d4a35cb1fb121c4a23/
    Download Restriction: no
    File URL: https://libkey.io/10.31235/osf.io/mbgxs_v1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Svend Kreiner & Karl Christensen, 2014. "Analyses of Model Fit and Robustness. A New Look at the PISA Scaling Model Underlying Ranking of Countries According to Reading Literacy," Psychometrika, Springer;The Psychometric Society, vol. 79(2), pages 210-231, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Curt Hagquist & Raili Välimaa & Nina Simonsen & Sakari Suominen, 2017. "Differential Item Functioning in Trend Analyses of Adolescent Mental Health – Illustrative Examples Using HBSC-Data from Finland," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 10(3), pages 673-691, September.
    2. Engzell, Per, 2017. "What Do Books in the Home Proxy For? A Cautionary Tale," Working Paper Series 1/2016, Stockholm University, Swedish Institute for Social Research.
    3. Giulia Grisolia & Umberto Lucia & Marco Filippo Torchio, 2022. "Sustainable Development and Workers Ability: Considerations on the Education Index in the Human Development Index," Sustainability, MDPI, vol. 14(14), pages 1-18, July.
    4. Robert J. Zwitser & S. Sjoerd F. Glaser & Gunter Maris, 2017. "Monitoring Countries in a Changing World: A New Look at DIF in International Surveys," Psychometrika, Springer;The Psychometric Society, vol. 82(1), pages 210-232, March.
    5. Laura Zieger & John Jerrim & Jake Anders & Nikki Shure, 2020. "Conditioning: How background variables can influence PISA scores," CEPEO Working Paper Series 20-09, UCL Centre for Education Policy and Equalising Opportunities, revised Apr 2020.
    6. Daniele, Vittorio, 2021. "Socioeconomic inequality and regional disparities in educational achievement: The role of relative poverty," Intelligence, Elsevier, vol. 84(C).
    7. Maarten Marsman & Gunter Maris & Timo Bechger & Cees Glas, 2016. "What can we learn from Plausible Values?," Psychometrika, Springer;The Psychometric Society, vol. 81(2), pages 274-289, June.
    8. repec:osf:osfxxx:gu8sq_v1 is not listed on IDEAS
    9. Maria Bolsinova & Jesper Tijmstra & Leslie Rutkowski & David Rutkowski, 2024. "Generalizing Beyond the Test: Permutation-Based Profile Analysis for Explaining DIF Using Item Features," Journal of Educational and Behavioral Statistics, , vol. 49(2), pages 207-240, April.
    10. Krammer, Georg, 2019. "The Andersen likelihood ratio test with a random split criterion lacks power," OSF Preprints gu8sq, Center for Open Science.
    11. Sum, Paul E. & Bădescu, Gabriel, 2023. "Collaboration and socio-economic inequality: Estimating the effects of intra-school and inter-school inequality on collaborative problem-solving skills," International Journal of Educational Development, Elsevier, vol. 102(C).
    12. repec:eur:ejserj:560 is not listed on IDEAS
    13. Schnepf, Sylke, 2018. "Insights into survey errors of large scale educational achievement surveys," JRC Working Papers in Economics and Finance 2018-05, Joint Research Centre, European Commission.
    14. Liu, Ji & Steiner-Khamsi, Gita, 2020. "Human Capital Index and the hidden penalty for non-participation in ILSAs," International Journal of Educational Development, Elsevier, vol. 73(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:socarx:mbgxs_v1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://socarxiv.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.