IDEAS home Printed from https://ideas.repec.org/a/pal/jorsoc/v59y2008i2d10.1057_palgrave.jors.2602445.html
   My bibliography  Save this article

Creating composite indicators with DEA and robustness analysis: the case of the Technology Achievement Index

Author

Listed:
  • L Cherchye

    (Catholic University of Leuven)

  • W Moesen

    (Catholic University of Leuven)

  • N Rogge

    (Catholic University of Leuven)

  • T Van Puyenbroeck

    (Catholic University of Leuven)

  • M Saisana

    (Joint Research Centre, European Commission)

  • A Saltelli

    (Joint Research Centre, European Commission)

  • R Liska

    (Joint Research Centre, European Commission)

  • S Tarantola

    (Joint Research Centre, European Commission)

Abstract

Composite indicators (CIs) are often used for benchmarking countries' performance, but they frequently stir controversies about the unavoidable subjectivity in their construction. Data Envelopment Analysis helps to overcome some key limitations, as it does not need any prior information on either the normalization of sub-indicators or on an agreed unique set of weights. Still, subjective decisions remain, and such modelling uncertainty propagates onto countries' CI scores and rankings. Uncertainty and sensitivity analysis are therefore needed to assess the robustness of the final outcome and to analyse how much each source of uncertainty contributes to the output variance. The current paper reports on these issues, using the Technology Achievement Index as an illustration.

Suggested Citation

  • L Cherchye & W Moesen & N Rogge & T Van Puyenbroeck & M Saisana & A Saltelli & R Liska & S Tarantola, 2008. "Creating composite indicators with DEA and robustness analysis: the case of the Technology Achievement Index," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(2), pages 239-251, February.
    • Handle: RePEc:pal:jorsoc:v:59:y:2008:i:2:d:10.1057_palgrave.jors.2602445
    DOI: 10.1057/palgrave.jors.2602445
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1057/palgrave.jors.2602445
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1057/palgrave.jors.2602445?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    2. Valdmanis, Vivian, 1992. "Sensitivity analysis for DEA models : An empirical example using public vs. NFP hospitals," Journal of Public Economics, Elsevier, vol. 48(2), pages 185-205, July.
    3. Léopold Simar & Paul W. Wilson, 1998. "Sensitivity Analysis of Efficiency Scores: How to Bootstrap in Nonparametric Frontier Models," Management Science, INFORMS, vol. 44(1), pages 49-61, January.
    4. Michael Freudenberg, 2003. "Composite Indicators of Country Performance: A Critical Assessment," OECD Science, Technology and Industry Working Papers 2003/16, OECD Publishing.
    5. Cazals, Catherine & Florens, Jean-Pierre & Simar, Leopold, 2002. "Nonparametric frontier estimation: a robust approach," Journal of Econometrics, Elsevier, vol. 106(1), pages 1-25, January.
    6. D K Despotis, 2005. "A reassessment of the human development index via data envelopment analysis," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 56(8), pages 969-980, August.
    7. Michela Nardo & Michaela Saisana & Andrea Saltelli & Stefano Tarantola & Anders Hoffman & Enrico Giovannini, 2005. "Handbook on Constructing Composite Indicators: Methodology and User Guide," OECD Statistics Working Papers 2005/3, OECD Publishing.
    8. Atkinson, Tony & Cantillon, Bea & Marlier, Eric & Nolan, Brian, 2002. "Social Indicators: The EU and Social Inclusion," OUP Catalogue, Oxford University Press, number 9780199253494.
    9. Cherchye, Laurens & Knox Lovell, C.A. & Moesen, Wim & Van Puyenbroeck, Tom, 2007. "One market, one number? A composite indicator assessment of EU internal market dynamics," European Economic Review, Elsevier, vol. 51(3), pages 749-779, April.
    10. Laurens Cherchye & Wim Moesen & Tom Van Puyenbroeck, 2004. "Legitimately Diverse, yet Comparable: On Synthesizing Social Inclusion Performance in the EU," Journal of Common Market Studies, Wiley Blackwell, vol. 42(5), pages 919-955, December.
    11. Meghnad Desai & Sakiko Fukuda-Parr & Claes Johansson & Fransisco Sagasti, 2002. "Measuring the Technology Achievement of Nations and the Capacity to Participate in the Network Age," Journal of Human Development and Capabilities, Taylor & Francis Journals, vol. 3(1), pages 95-122.
    12. Knox Lovell, C. A. & Pastor, Jesus T. & Turner, Judi A., 1995. "Measuring macroeconomic performance in the OECD: A comparison of European and non-European countries," European Journal of Operational Research, Elsevier, vol. 87(3), pages 507-518, December.
    13. Laurens Cherchye & & Timo Kuosmanen, 2002. "Benchmarking Sustainable Development: A Synthetic Meta-index Approach," Others 0210001, University Library of Munich, Germany, revised 04 Dec 2002.
    14. Sen, Amartya, 1997. "On Economic Inequality," OUP Catalogue, Oxford University Press, number 9780198292975.
    15. R. Allen & A. Athanassopoulos & R.G. Dyson & E. Thanassoulis, 1997. "Weights restrictions and value judgements in Data Envelopment Analysis: Evolution, development and future directions," Annals of Operations Research, Springer, vol. 73(0), pages 13-34, October.
    16. Tony Atkinson & Bea Cantillon & Eric Marlier & Brian Nolan, 2002. "Indicators for Social Inclusion," Politica economica, Società editrice il Mulino, issue 1, pages 7-28.
    17. Léopold Simar, 2003. "Detecting Outliers in Frontier Models: A Simple Approach," Journal of Productivity Analysis, Springer, vol. 20(3), pages 391-424, November.
    18. Sarrico, C. S. & Dyson, R. G., 2004. "Restricting virtual weights in data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 159(1), pages 17-34, November.
    19. Cook, Wade D. & Kress, Moshe, 1994. "A multiple-criteria composite index model for quantitative and qualitative data," European Journal of Operational Research, Elsevier, vol. 78(3), pages 367-379, November.
    20. M. Saisana & A. Saltelli & S. Tarantola, 2005. "Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(2), pages 307-323, March.
    21. C Kao & H-T Hung, 2005. "Data envelopment analysis with common weights: the compromise solution approach," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 56(10), pages 1196-1203, October.
    22. Francisco Pedraja-Chaparro & Javier Salinas-Jimenez & Peter Smith, 1997. "On the Role of Weight Restrictions in Data Envelopment Analysis," Journal of Productivity Analysis, Springer, vol. 8(2), pages 215-230, May.
    23. Storrie, Donald & Bjurek, Hans, 2000. "Benchmarking European labour market performance with efficiency frontier techniques," Discussion Papers, Research Unit: Labor Market Policy and Employment FS I 00-211, WZB Berlin Social Science Center.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Laurens Cherchye & Willem Moesen & Nicky Rogge & Tom Puyenbroeck, 2007. "An Introduction to ‘Benefit of the Doubt’ Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 82(1), pages 111-145, May.
    2. Laurens CHERCHYE & Willem MOESEN & Nicky ROGGE & Tom VAN PUYENBROECK, 2009. "Constructing a knowledge economy composite indicator with imprecise data," Working Papers of Department of Economics, Leuven ces09.15, KU Leuven, Faculty of Economics and Business (FEB), Department of Economics, Leuven.
    3. Fusco, Elisa, 2015. "Enhancing non-compensatory composite indicators: A directional proposal," European Journal of Operational Research, Elsevier, vol. 242(2), pages 620-630.
    4. Amado, Carla A.F. & São José, José M.S. & Santos, Sérgio P., 2016. "Measuring active ageing: A Data Envelopment Analysis approach," European Journal of Operational Research, Elsevier, vol. 255(1), pages 207-223.
    5. Francesco Vidoli & Elisa Fusco & Claudio Mazziotta, 2015. "Non-compensability in Composite Indicators: A Robust Directional Frontier Method," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 122(3), pages 635-652, July.
    6. Cherchye, Laurens & Knox Lovell, C.A. & Moesen, Wim & Van Puyenbroeck, Tom, 2007. "One market, one number? A composite indicator assessment of EU internal market dynamics," European Economic Review, Elsevier, vol. 51(3), pages 749-779, April.
    7. Kristof Witte & Nicky Rogge, 2010. "To publish or not to publish? On the aggregation and drivers of research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(3), pages 657-680, December.
    8. Eduardo González & Ana Cárcaba & Juan Ventura, 2018. "Weight Constrained DEA Measurement of the Quality of Life in Spanish Municipalities in 2011," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 136(3), pages 1157-1182, April.
    9. Salvatore Greco & Alessio Ishizaka & Menelaos Tasiou & Gianpiero Torrisi, 2019. "On the Methodological Framework of Composite Indices: A Review of the Issues of Weighting, Aggregation, and Robustness," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 141(1), pages 61-94, January.
    10. Pilar Murias & José Miguel & David Rodríguez, 2008. "A Composite Indicator for University Quality Assesment: The Case of Spanish Higher Education System," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 89(1), pages 129-146, October.
    11. Fabio Pammolli & Francesco Porcelli & Francesco Vidoli & Guido Borà, 2014. "La spesa sanitaria delle Regioni in Italia - Saniregio 3," Working Papers CERM 02-2014, Competitività, Regole, Mercati (CERM).
    12. Fabio Pammolli & Francesco Porcelli & Francesco Vidoli & Monica Auteri & Guido Borà, 2017. "La spesa sanitaria delle Regioni in Italia - Saniregio2017," Working Papers CERM 01-2017, Competitività, Regole, Mercati (CERM).
    13. Fusco, Elisa & Maggi, Bernardo & Rizzuto, Livia, 2022. "Alternative indicators for the evaluation of renewables in Europe: An efficiency approach," Renewable Energy, Elsevier, vol. 190(C), pages 48-65.
    14. Sabina Alkire, James Foster, 2010. "Designing the Inequality-Adjusted Human Development Index (HDI)," OPHI Working Papers 37, Queen Elizabeth House, University of Oxford.
    15. Ana Cárcaba & Eduardo González & Juan Ventura, 2017. "Social Progress in Spanish Municipalities (2001–2011)," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 12(4), pages 997-1019, December.
    16. Hatefi, S.M. & Torabi, S.A., 2010. "A common weight MCDA-DEA approach to construct composite indicators," Ecological Economics, Elsevier, vol. 70(1), pages 114-120, November.
    17. Dovile Stumbriene & Ana S. Camanho & Audrone Jakaitiene, 2020. "The performance of education systems in the light of Europe 2020 strategy," Annals of Operations Research, Springer, vol. 288(2), pages 577-608, May.
    18. Emrouznejad, Ali & De Witte, Kristof, 2010. "COOPER-framework: A unified process for non-parametric projects," European Journal of Operational Research, Elsevier, vol. 207(3), pages 1573-1586, December.
    19. Grupp, Hariolf & Schubert, Torben, 2010. "Review and new evidence on composite innovation indicators for evaluating national performance," Research Policy, Elsevier, vol. 39(1), pages 67-78, February.
    20. Riccardo Natoli & Segu Zuhair, 2011. "Measuring Progress: A Comparison of the GDP, HDI, GS and the RIE," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 103(1), pages 33-56, August.

    More about this item

    Keywords

    Data Envelopment Analysis; composite indicator construction; robustness analysis;
    All these keywords.

    JEL classification:

    • E13 - Macroeconomics and Monetary Economics - - General Aggregative Models - - - Neoclassical
    • E60 - Macroeconomics and Monetary Economics - - Macroeconomic Policy, Macroeconomic Aspects of Public Finance, and General Outlook - - - General
    • F02 - International Economics - - General - - - International Economic Order and Integration
    • O47 - Economic Development, Innovation, Technological Change, and Growth - - Economic Growth and Aggregate Productivity - - - Empirical Studies of Economic Growth; Aggregate Productivity; Cross-Country Output Convergence

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pal:jorsoc:v:59:y:2008:i:2:d:10.1057_palgrave.jors.2602445. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.palgrave-journals.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.