IDEAS home Printed from https://ideas.repec.org/a/eee/soceps/v102y2025ics0038012125001636.html

On the coherence of composite indexes of well-being: Multiverse analysis for formative models of measurement

Author

Listed:
  • Cantone, Giulio Giacomo
  • Tomaselli, Venera

Abstract

Composite indexes are widely used in socio-economic research, yet methodological choices often affect their reliability, leading to variability in results and uncertainty in policy applications. This study applies Multiverse Analysis to systematically assess the robustness of the Italian Equitable and Sustainable Well-being system. 68 alternative specifications for a formative index are fitted. Results show that highly performing provinces exhibit significantly greater variability across the specifications, indicating that index-based assessments may be highly sensitive to methodological assumptions. The choice of aggregation function (e.g., arithmetic vs non-linear means) does not substantially impact uncertainty. However, techniques based on Principal Component Analysis (PCA) inflate the uncertainty in the results, as predicted by authors who question the suitability of PCA for formative measurement.

Suggested Citation

  • Cantone, Giulio Giacomo & Tomaselli, Venera, 2025. "On the coherence of composite indexes of well-being: Multiverse analysis for formative models of measurement," Socio-Economic Planning Sciences, Elsevier, vol. 102(C).
    • Handle: RePEc:eee:soceps:v:102:y:2025:i:c:s0038012125001636
    DOI: 10.1016/j.seps.2025.102314
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0038012125001636
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.seps.2025.102314?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Paolo Paruolo & Michaela Saisana & Andrea Saltelli, 2013. "Ratings and rankings: voodoo or science?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(3), pages 609-634, June.
    2. Marta Kuc-Czarnecka & Samuele Lo Piano & Andrea Saltelli, 2020. "Quantitative Storytelling in the Making of a Composite Indicator," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 149(3), pages 775-802, June.
    3. Venera Tomaselli & Mario Fordellone & Maurizio Vichi, 2021. "Building Well-Being Composite Indicator for Micro-Territorial Areas Through PLS-SEM and K-Means Approach," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 153(2), pages 407-429, January.
    4. Karagiannis, Giannis & Ravanos, Panagiotis, 2023. "A composite indicator of social inclusion for EU based on the inverted BoD model," Socio-Economic Planning Sciences, Elsevier, vol. 88(C).
    5. Cantone, Giulio Giacomo & Tomaselli, Venera, 2023. "A Multiversal Model of Vibration of Effects of the Equitable and Sustainable Well-Being (BES) on Fertility," MetaArXiv z5msx, Center for Open Science.
    6. Jesús Peiró-Palomino & Andrés J. Picazo-Tadeo, 2018. "OECD: One or Many? Ranking Countries with a Composite Well-Being Indicator," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 139(3), pages 847-869, October.
    7. Giannis Karagiannis, 2017. "On aggregate composite indicators," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(7), pages 741-746, July.
    8. William Becker & Paolo Paruolo & Michaela Saisana & Andrea Saltelli, 2017. "Weights and Importance in Composite Indicators: Mind the Gap," Springer Books, in: Roger Ghanem & David Higdon & Houman Owhadi (ed.), Handbook of Uncertainty Quantification, chapter 34, pages 1187-1216, Springer.
    9. Federica Onori & Giovanna Jona Lasinio, 2022. "Modeling “Equitable and Sustainable Well-being” (BES) Using Bayesian Networks: A Case Study of the Italian Regions," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 161(2), pages 1003-1037, June.
    10. Matteo Mazziotta & Adriano Pareto, 2019. "Use and Misuse of PCA for Measuring Well-Being," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 142(2), pages 451-476, April.
    11. repec:osf:metaar:z5msx_v1 is not listed on IDEAS
    12. Miguel de Carvalho, 2016. "Mean, What do You Mean?," The American Statistician, Taylor & Francis Journals, vol. 70(3), pages 270-274, July.
    13. Rogge, Nicky, 2012. "Undesirable specialization in the construction of composite policy indicators: The Environmental Performance Index," Working Papers 2012/08, Hogeschool-Universiteit Brussel, Faculteit Economie en Management.
    14. Enrico Casadio Tarabusi & Giulio Guarini, 2013. "An Unbalance Adjustment Method for Development Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 112(1), pages 19-45, May.
    15. Leonardo Salvatore Alaimo & Alberto Arcagni & Marco Fattore & Filomena Maggino, 2021. "Synthesis of Multi-indicator System Over Time: A Poset-based Approach," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 157(1), pages 77-99, August.
    16. G. Lun & D. Holzer & G. Tappeiner & U. Tappeiner, 2006. "The Stability of Rankings Derived From Composite Indicators: Analysis of the “IL Sole 24 Ore” Quality of Life Report," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 77(2), pages 307-331, June.
    17. James E. Foster & Mark McGillivray & Suman Seth, 2013. "Composite Indices: Rank Robustness, Statistical Association, and Redundancy," Econometric Reviews, Taylor & Francis Journals, vol. 32(1), pages 35-56, January.
    18. Grazini, Chiara, 2024. "Energy poverty as capacity deprivation: A study of social housing using the partially ordered set," Socio-Economic Planning Sciences, Elsevier, vol. 92(C).
    19. Mark McGillivray, 2005. "Measuring Non‐Economic Well‐Being Achievement," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 51(2), pages 337-364, June.
    20. Zhou, P. & Ang, B.W. & Poh, K.L., 2007. "A mathematical programming approach to constructing composite indicators," Ecological Economics, Elsevier, vol. 62(2), pages 291-297, April.
    21. Axel Dreher, 2006. "Does globalization affect growth? Evidence from a new index of globalization," Applied Economics, Taylor & Francis Journals, vol. 38(10), pages 1091-1110.
    22. Tianjiao Wang & Yelin Fu, 2020. "Constructing Composite Indicators with Individual Judgements and Best–Worst Method: An Illustration of Value Measure," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 149(1), pages 1-14, May.
    23. Leo Egghe & Loet Leydesdorff, 2009. "The relation between Pearson's correlation coefficient r and Salton's cosine measure," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(5), pages 1027-1036, May.
    24. Alberto Arcagni & Marco Fattore & Filomena Maggino & Giorgio Vittadini, 2021. "Some Critical Reflections on the Measurement of Social Sustainability and Well-Being in Complex Societies," Sustainability, MDPI, vol. 13(22), pages 1-16, November.
    25. Frederik Booysen, 2002. "An Overview and Evaluation of Composite Indices of Development," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 59(2), pages 115-151, August.
    26. Matteo Mazziotta & Adriano Pareto, 2018. "Measuring Well-Being Over Time: The Adjusted Mazziotta–Pareto Index Versus Other Non-compensatory Indices," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 136(3), pages 967-976, April.
    27. Sojung Lim, 2021. "Socioeconomic differentials in fertility in South Korea," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 44(39), pages 941-978.
    28. Talita Greyling & Fiona Tregenna, 2017. "Construction and Analysis of a Composite Quality of Life Index for a Region of South Africa," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 131(3), pages 887-930, April.
    29. Matteo Mazziotta & Adriano Pareto, 2016. "On a Generalized Non-compensatory Composite Index for Measuring Socio-economic Phenomena," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 127(3), pages 983-1003, July.
    30. Giuseppe Nicoletti & Stefano Scarpetta & Olivier Boylaud, 2000. "Summary Indicators of Product Market Regulation with an Extension to Employment Protection Legislation," OECD Economics Department Working Papers 226, OECD Publishing.
    31. Leonardo Salvatore Alaimo & Alberto Arcagni & Marco Fattore & Filomena Maggino & Valeria Quondamstefano, 2022. "Measuring Equitable and Sustainable Well-Being in Italian Regions: The Non-aggregative Approach," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 161(2), pages 711-733, June.
    32. Leonardo S. Alaimo & Filomena Maggino, 2020. "Sustainable Development Goals Indicators at Territorial Level: Conceptual and Methodological Issues—The Italian Perspective," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 147(2), pages 383-419, January.
    33. Francesco Vidoli & Elisa Fusco & Claudio Mazziotta, 2015. "Non-compensability in Composite Indicators: A Robust Directional Frontier Method," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 122(3), pages 635-652, July.
    34. Mangaraj, B.K. & Aparajita, Upali, 2020. "Constructing a generalized model of the human development index," Socio-Economic Planning Sciences, Elsevier, vol. 70(C).
    35. Comim, Flavio & Hirai, Tadashi, 2022. "Sustainability and Human Development Indicators: A Poset Analysis," Ecological Economics, Elsevier, vol. 198(C).
    36. Martinez-Alier, Joan & Munda, Giuseppe & O'Neill, John, 1998. "Weak comparability of values as a foundation for ecological economics," Ecological Economics, Elsevier, vol. 26(3), pages 277-286, September.
    37. Dejian Lai, 2000. "Temporal Analysis of Human Development Indicators: Principal Component Approach," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 51(3), pages 331-366, September.
    38. Enrico Ivaldi & Guido Bonatti & Riccardo Soliani, 2016. "The Construction of a Synthetic Index Comparing Multidimensional Well-Being in the European Union," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 125(2), pages 397-430, January.
    39. Ram, Rati, 1982. "Composite indices of physical quality of life, basic needs fulfilment, and income : A principal component representation," Journal of Development Economics, Elsevier, vol. 11(2), pages 227-247, October.
    40. Christian Leibel & Lutz Bornmann, 2024. "Specification uncertainty: what the disruption index tells us about the (hidden) multiverse of bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(12), pages 7971-7979, December.
    41. Lauri Peterson, 2014. "The Measurement of Non-economic Inequality in Well-Being Indices," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 119(2), pages 581-598, November.
    42. Silvia Facchinetti & Elena Siletti, 2022. "Well-being Indicators: a Review and Comparison in the Context of Italy," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 159(2), pages 523-547, January.
    43. Rogge, Nicky, 2018. "Composite indicators as generalized benefit-of-the-doubt weighted averages," European Journal of Operational Research, Elsevier, vol. 267(1), pages 381-392.
    44. Zhou, P. & Ang, B.W. & Poh, K.L., 2006. "Comparing aggregating methods for constructing the composite environmental index: An objective measure," Ecological Economics, Elsevier, vol. 59(3), pages 305-311, September.
    45. Mariateresa Ciommi & Andrea Gentili & Barbara Ermini & Chiara Gigliarano & Francesco M. Chelli & Mauro Gallegati, 2017. "Have Your Cake and Eat it Too: The Well-Being of the Italians (1861–2011)," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 134(2), pages 473-509, November.
    46. Frances Stewart, 2014. "Sustainability and Inequality," Development, Palgrave Macmillan;Society for International Deveopment, vol. 57(3-4), pages 344-361, December.
    47. Iñaki Permanyer, 2012. "Uncertainty and robustness in composite indices rankings," Oxford Economic Papers, Oxford University Press, vol. 64(1), pages 57-79, January.
    48. M. Saisana & A. Saltelli & S. Tarantola, 2005. "Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(2), pages 307-323, March.
    49. Matteo Mazziotta & Adriano Pareto, 2016. "Methods for Constructing Non-Compensatory Composite Indices: A Comparative Study," Forum for Social Economics, Taylor & Francis Journals, vol. 45(2-3), pages 213-229, August.
    50. Sayel Basel & K. U. Gopakumar & R. Prabhakara Rao, 2020. "Broad-based index for measurement of development," Journal of Social and Economic Development, Springer;Institute for Social and Economic Change, vol. 22(1), pages 182-206, June.
    51. Karagiannis, Roxani & Karagiannis, Giannis, 2020. "Constructing composite indicators with Shannon entropy: The case of Human Development Index," Socio-Economic Planning Sciences, Elsevier, vol. 70(C).
    52. Giulio Giacomo Cantone, 2025. "Multiversal model of measurement of a composite index of well-being," RIEDS - Rivista Italiana di Economia, Demografia e Statistica - The Italian Journal of Economic, Demographic and Statistical Studies, SIEDS Societa' Italiana di Economia Demografia e Statistica, vol. 79(3), pages 119-130, July-Sept.
    53. Cristina Davino & Pasquale Dolce & Stefania Taralli & Vincenzo Esposito Vinzi, 2018. "A Quantile Composite-Indicator Approach for the Measurement of Equitable and Sustainable Well-Being: A Case Study of the Italian Provinces," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 136(3), pages 999-1029, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:osf:metaar:d5y26_v1 is not listed on IDEAS
    2. Cantone, Giulio Giacomo & Tomaselli, Venera, 2024. "On the Coherence of Composite Indexes: Multiversal Model and Specification Analysis for an Index of Well-Being," MetaArXiv d5y26, Center for Open Science.
    3. Salvatore Greco & Alessio Ishizaka & Menelaos Tasiou & Gianpiero Torrisi, 2019. "On the Methodological Framework of Composite Indices: A Review of the Issues of Weighting, Aggregation, and Robustness," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 141(1), pages 61-94, January.
    4. Rajko Tomaš, 2022. "Measurement of the Concentration of Potential Quality of Life in Local Communities," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 163(1), pages 79-109, August.
    5. Milica Maricic & Jose A. Egea & Veljko Jeremic, 2019. "A Hybrid Enhanced Scatter Search—Composite I-Distance Indicator (eSS-CIDI) Optimization Approach for Determining Weights Within Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 144(2), pages 497-537, July.
    6. Giulio Giacomo Cantone, 2024. "How to measure interdisciplinary research? A systemic design for the model of measurement," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(8), pages 4937-4982, August.
    7. P. Zhou & B. Ang, 2009. "Comparing MCDA Aggregation Methods in Constructing Composite Indicators Using the Shannon-Spearman Measure," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 94(1), pages 83-96, October.
    8. Matheus Pereira Libório & Elisa Fusco & Alexandre Magno Alves Diniz & Oséias da Silva Martinuci & Petr Iakovlevitch Ekel, 2024. "A Novel Approach for Multispatial and Multitemporal Analysis of Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 173(3), pages 783-800, July.
    9. P. Zhou & B. Ang & D. Zhou, 2010. "Weighting and Aggregation in Composite Indicator Construction: a Multiplicative Optimization Approach," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 96(1), pages 169-181, March.
    10. Ilyes Boumahdi & Nouzha Zaoujal, 2023. "Regional Well-Being Disparities in Morocco and its OECD Partners," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 167(1), pages 183-211, June.
    11. Su, Weihua & Chen, Sibo & Zhang, Chonghui & Li, Kevin W., 2023. "A subgroup dominance-based benefit of the doubt method for addressing rank reversals: A case study of the human development index in Europe," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1299-1317.
    12. Matteo Mazziotta & Adriano Pareto, 2022. "Normalization methods for spatio‐temporal analysis of environmental performance: Revisiting the Min–Max method," Environmetrics, John Wiley & Sons, Ltd., vol. 33(5), August.
    13. Rosanna Cataldo & Corrado Crocetta & Maria Gabriella Grassia & Paolo Mazzocchi & Antonella Rocca & Claudio Quintano, 2024. "Sustainable Innovation: The Italian Scenario Studied Through Higher-Order Partial Least Squares-Path Modeling," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 173(1), pages 201-225, May.
    14. Ariane Carla Barbosa da Silva & Diogo Rodrigo dos Reis & Matheus Pereira Libório & Hasheem Mannan & Cristiane Neri Nobre, 2026. "A Composite Indicator of Anxiety and Depression Signs in Children and Adolescents of Low- and Middle-Income Countries: a Subjective-Objective Multidimensional Approach," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 19(2), pages 917-956, April.
    15. Giulio Giacomo Cantone & Venera Tomaselli, 2024. "A Multiversal Model of Vibration of Effects of the Equitable and Sustainable Well-Being (BES) on Fertility," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 175(3), pages 941-964, December.
    16. Petróczy, Dóra Gréta, 2021. "An alternative quality of life ranking on the basis of remittances," Socio-Economic Planning Sciences, Elsevier, vol. 78(C).
    17. Rogge, Nicky, 2018. "Composite indicators as generalized benefit-of-the-doubt weighted averages," European Journal of Operational Research, Elsevier, vol. 267(1), pages 381-392.
    18. Greco, Salvatore & Ishizaka, Alessio & Tasiou, Menelaos & Torrisi, Gianpiero, 2018. "σ-µ efficiency analysis: A new methodology for evaluating units through composite indices," MPRA Paper 83569, University Library of Munich, Germany.
    19. Giuliano Resce & Fritz Schiltz, 2021. "Sustainable Development in Europe: A Multicriteria Decision Analysis," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 67(2), pages 509-529, June.
    20. Greco, Salvatore & Ishizaka, Alessio & Tasiou, Menelaos & Torrisi, Gianpiero, 2019. "Sigma-Mu efficiency analysis: A methodology for evaluating units through composite indicators," European Journal of Operational Research, Elsevier, vol. 278(3), pages 942-960.
    21. Rajesh Gupta & Atulan Guha, 2024. "Construction of Sub-country Level Environment Policy Stringency Index: A Study on Indian States," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 172(2), pages 449-479, March.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:soceps:v:102:y:2025:i:c:s0038012125001636. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/seps .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.