IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Login to save this article

Review and new evidence on composite innovation indicators for evaluating national performance

  • Grupp, Hariolf
  • Schubert, Torben

The purpose of this contribution is to present a survey of the recent developments in constructing composite science and technology (S&T) indicators on a national level as well as new evidence of the variability of such S&T indicators which opens the gateway to "country-tuning". It has become standard practice to combine several indicators for science, technology, and innovation to form composite numbers. Especially in the light of this variability, two questions arise. Firstly, are the results (especially rankings) stable with respect to weights? Secondly, is there hope to define "economically" reasonable weights? In order to provide answers to these questions, we use data from the European Innovation Scoreboard 2005 (EIS 2005) to exemplify our reasoning. Concerning the first question, we give genuine evidence on the existence of immense variability, possibly invalidating the results. Further, we also show that even existing and well-accepted methods, like equal weighting, Benefit of the Doubt weighting (BoD) and principal component analysis weighting (PCA) may lead to drastically differing results. Concerning the second question we will demonstrate that by each composite indicator weighting a set of shadow prices is implied expressing one indicator in terms of another. Whether the weights are sensible should be evaluated on the basis of these shadow prices. It turns out that those implied by EIS 2005 contain strange peculiarities. After that we plead for more care in constructing composite indicators. Especially weights should be chosen on the basis of shadow prices, rather than, say, by equal weighting or other automatic methods. Lastly, we discuss the merit of composite indicators and argue that they have a valuable communication and competition function, but they should be accompanied by multidimensional representations, which provide the basis for the construction of policy measures.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.sciencedirect.com/science/article/pii/S0048-7333(09)00199-1
Download Restriction: Full text for ScienceDirect subscribers only

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Article provided by Elsevier in its journal Research Policy.

Volume (Year): 39 (2010)
Issue (Month): 1 (February)
Pages: 67-78

as
in new window

Handle: RePEc:eee:respol:v:39:y:2010:i:1:p:67-78
Contact details of provider: Web page: http://www.elsevier.com/locate/respol

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Paul M Romer, 1999. "Endogenous Technological Change," Levine's Working Paper Archive 2135, David K. Levine.
  2. Philippe Aghion & Peter Howitt, 1990. "A Model of Growth Through Creative Destruction," NBER Working Papers 3223, National Bureau of Economic Research, Inc.
  3. Michela Nardo & Michaela Saisana & Andrea Saltelli & Stefano Tarantola & Anders Hoffman & Enrico Giovannini, 2005. "Handbook on Constructing Composite Indicators: Methodology and User Guide," OECD Statistics Working Papers 2005/3, OECD Publishing.
  4. Jon Zabala-Iturriagagoitia & Peter Voigt & Antonio Gutierrez-Gracia & Fernando Jimenez-Saez, 2007. "Regional Innovation Systems: How to Assess Performance," Regional Studies, Taylor & Francis Journals, vol. 41(5), pages 661-672.
  5. Laurens Cherchye & Wim Moesen & Nicky Rogge & Tom Van Puyenbroeck & Michaela Saisana & A. Saltelli & R. Liska & S. Tarantola, 2006. "Creating Composite Indicators with DEA and Robustness Analysis: the case of the Technology Achievement Index," Public Economics Working Paper Series ces0613, Katholieke Universiteit Leuven, Centrum voor Economische Studiën, Working Group Public Economics.
  6. Cherchye, Laurens & Knox Lovell, C.A. & Moesen, Wim & Van Puyenbroeck, Tom, 2007. "One market, one number? A composite indicator assessment of EU internal market dynamics," European Economic Review, Elsevier, vol. 51(3), pages 749-779, April.
  7. Archibugi, Daniele & Coco, Alberto, 2005. "Measuring technological capabilities at the country level: A survey and a menu for choice," Research Policy, Elsevier, vol. 34(2), pages 175-194, March.
  8. Laurens Cherchye & Wim Moesen & Tom Puyenbroeck, 2004. "Legitimately Diverse, yet Comparable: On Synthesizing Social Inclusion Performance in the EU," Journal of Common Market Studies, Wiley Blackwell, vol. 42(5), pages 919-955, December.
  9. Archibugi, Daniele & Coco, Alberto, 2004. "A New Indicator of Technological Capabilities for Developed and Developing Countries (ArCo)," World Development, Elsevier, vol. 32(4), pages 629-654, April.
  10. Grupp, Hariolf & Mogee, Mary Ellen, 2004. "Indicators for national science and technology policy: how robust are composite indicators?," Research Policy, Elsevier, vol. 33(9), pages 1373-1384, November.
  11. Michael Freudenberg, 2003. "Composite Indicators of Country Performance: A Critical Assessment," OECD Science, Technology and Industry Working Papers 2003/16, OECD Publishing.
  12. Cazals, Catherine & Florens, Jean-Pierre & Simar, Leopold, 2002. "Nonparametric frontier estimation: a robust approach," Journal of Econometrics, Elsevier, vol. 106(1), pages 1-25, January.
  13. Romer, Paul M, 1987. "Growth Based on Increasing Returns Due to Specialization," American Economic Review, American Economic Association, vol. 77(2), pages 56-62, May.
  14. Grupp, Hariolf, 1994. "The measurement of technical performance of innovations by technometrics and its impact on established technology indicators," Research Policy, Elsevier, vol. 23(2), pages 175-193, March.
  15. Hollenstein, Heinz, 1996. "A composite indicator of a firm's innovativeness. An empirical analysis based on survey data for Swiss manufacturing," Research Policy, Elsevier, vol. 25(4), pages 633-645, June.
  16. Frederik Booysen, 2002. "An Overview and Evaluation of Composite Indices of Development," Social Indicators Research- An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 59(2), pages 115-151, August.
  17. Marianne Paasi, 2005. "Collective benchmarking of policies: an instrument for policy learning in adaptive research and innovation policy," Science and Public Policy, Oxford University Press, vol. 32(1), pages 17-27, February.
  18. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
  19. Gene M. Grossman & Elhanan Helpman, 1993. "Innovation and Growth in the Global Economy," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262570971.
  20. Theodoros Papaioannou & Howard Rush & John Bessant, 2006. "Benchmarking as a policy-making tool: From the private to the public sector," Science and Public Policy, Oxford University Press, vol. 33(2), pages 91-102, March.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:39:y:2010:i:1:p:67-78. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.