Review and new evidence on composite innovation indicators for evaluating national performance
The purpose of this contribution is to present a survey of the recent developments in constructing composite science and technology (S&T) indicators on a national level as well as new evidence of the variability of such S&T indicators which opens the gateway to "country-tuning". It has become standard practice to combine several indicators for science, technology, and innovation to form composite numbers. Especially in the light of this variability, two questions arise. Firstly, are the results (especially rankings) stable with respect to weights? Secondly, is there hope to define "economically" reasonable weights? In order to provide answers to these questions, we use data from the European Innovation Scoreboard 2005 (EIS 2005) to exemplify our reasoning. Concerning the first question, we give genuine evidence on the existence of immense variability, possibly invalidating the results. Further, we also show that even existing and well-accepted methods, like equal weighting, Benefit of the Doubt weighting (BoD) and principal component analysis weighting (PCA) may lead to drastically differing results. Concerning the second question we will demonstrate that by each composite indicator weighting a set of shadow prices is implied expressing one indicator in terms of another. Whether the weights are sensible should be evaluated on the basis of these shadow prices. It turns out that those implied by EIS 2005 contain strange peculiarities. After that we plead for more care in constructing composite indicators. Especially weights should be chosen on the basis of shadow prices, rather than, say, by equal weighting or other automatic methods. Lastly, we discuss the merit of composite indicators and argue that they have a valuable communication and competition function, but they should be accompanied by multidimensional representations, which provide the basis for the construction of policy measures.
References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Theodoros Papaioannou & Howard Rush & John Bessant, 2006. "Benchmarking as a policy-making tool: From the private to the public sector," Science and Public Policy, Oxford University Press, vol. 33(2), pages 91-102, March.
- Grupp, Hariolf, 1994. "The measurement of technical performance of innovations by technometrics and its impact on established technology indicators," Research Policy, Elsevier, vol. 23(2), pages 175-193, March.
- Archibugi, Daniele & Coco, Alberto, 2005. "Measuring technological capabilities at the country level: A survey and a menu for choice," Research Policy, Elsevier, vol. 34(2), pages 175-194, March.
- Laurens Cherchye & Knox Lovell & Wim Moesen & Tom Van Puyenbroeck, 2005.
"One Market, One Number? A Composite Indicator Assessment of EU Internal Market Dynamics,"
Public Economics Working Paper Series
ces0513, Katholieke Universiteit Leuven, Centrum voor Economische Studiën, Working Group Public Economics.
- Cherchye, Laurens & Knox Lovell, C.A. & Moesen, Wim & Van Puyenbroeck, Tom, 2007. "One market, one number? A composite indicator assessment of EU internal market dynamics," European Economic Review, Elsevier, vol. 51(3), pages 749-779, April.
- Paul Romer, 1989.
"Endogenous Technological Change,"
NBER Working Papers
3210, National Bureau of Economic Research, Inc.
- Archibugi, Daniele & Coco, Alberto, 2004.
"A New Indicator of Technological Capabilities for Developed and Developing Countries (ArCo),"
Elsevier, vol. 32(4), pages 629-654, April.
- Daniele Archibugi & Alberto Coco, 2004. "A New Indicator of Technological Capabilities for Developed and Developing Countries (ArCo)," SPRU Working Paper Series 111, SPRU - Science and Technology Policy Research, University of Sussex.
- Daniele Archibugi & Alberto Coco, 2004. "A New Indicator of Technological Capabilities for Developed and Developing Countries (ArCo)," CEIS Research Paper 44, Tor Vergata University, CEIS.
- Philippe Aghion & Peter Howitt, 1990.
"A Model of Growth Through Creative Destruction,"
NBER Working Papers
3223, National Bureau of Economic Research, Inc.
- Aghion, P. & Howitt, P., 1989. "A Model Of Growth Through Creative Destruction," Working papers 527, Massachusetts Institute of Technology (MIT), Department of Economics.
- Aghion, P. & Howitt, P., 1989. "A Model Of Growth Through Creative Destruction," UWO Department of Economics Working Papers 8904, University of Western Ontario, Department of Economics.
- Aghion, P. & Howitt, P., 1990. "A Model Of Growth Through Creative Destruction," DELTA Working Papers 90-12, DELTA (Ecole normale supérieure).
- Laurens Cherchye & Wim Moesen & Tom Puyenbroeck, 2004.
"Legitimately Diverse, yet Comparable: On Synthesizing Social Inclusion Performance in the EU,"
Journal of Common Market Studies,
Wiley Blackwell, vol. 42(5), pages 919-955, December.
- Laurens Cherchye & Wim Moesen & Tom Van Puyenbroeck, 2003. "Legitimately Diverse, yet Comparable: On Synthesising Social Inclusion Performance in the EU," Public Economics Working Paper Series ces0301, Katholieke Universiteit Leuven, Centrum voor Economische Studiën, Working Group Public Economics.
- Aghion, Philippe & Howitt, Peter, 1992. "A Model of Growth Through Creative Destruction," Scholarly Articles 12490578, Harvard University Department of Economics.
- Marianne Paasi, 2005. "Collective benchmarking of policies: an instrument for policy learning in adaptive research and innovation policy," Science and Public Policy, Oxford University Press, vol. 32(1), pages 17-27, February.
- Jon Zabala-Iturriagagoitia & Peter Voigt & Antonio Gutierrez-Gracia & Fernando Jimenez-Saez, 2007. "Regional Innovation Systems: How to Assess Performance," Regional Studies, Taylor & Francis Journals, vol. 41(5), pages 661-672.
- Frederik Booysen, 2002. "An Overview and Evaluation of Composite Indices of Development," Social Indicators Research, Springer, vol. 59(2), pages 115-151, August.
- Grupp, Hariolf & Mogee, Mary Ellen, 2004. "Indicators for national science and technology policy: how robust are composite indicators?," Research Policy, Elsevier, vol. 33(9), pages 1373-1384, November.
- Michael Freudenberg, 2003. "Composite Indicators of Country Performance: A Critical Assessment," OECD Science, Technology and Industry Working Papers 2003/16, OECD Publishing.
- Michela Nardo & Michaela Saisana & Andrea Saltelli & Stefano Tarantola & Anders Hoffman & Enrico Giovannini, 2005. "Handbook on Constructing Composite Indicators: Methodology and User Guide," OECD Statistics Working Papers 2005/3, OECD Publishing.
- Romer, Paul M, 1987. "Growth Based on Increasing Returns Due to Specialization," American Economic Review, American Economic Association, vol. 77(2), pages 56-62, May.
- Gene M. Grossman & Elhanan Helpman, 1993. "Innovation and Growth in the Global Economy," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262570971, June.
- Laurens Cherchye & Wim Moesen & Nicky Rogge & Tom Van Puyenbroeck & Michaela Saisana & A. Saltelli & R. Liska & S. Tarantola, 2006. "Creating Composite Indicators with DEA and Robustness Analysis: the case of the Technology Achievement Index," Public Economics Working Paper Series ces0613, Katholieke Universiteit Leuven, Centrum voor Economische Studiën, Working Group Public Economics.
- Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
- Hollenstein, Heinz, 1996. "A composite indicator of a firm's innovativeness. An empirical analysis based on survey data for Swiss manufacturing," Research Policy, Elsevier, vol. 25(4), pages 633-645, June.
- Cazals, Catherine & Florens, Jean-Pierre & Simar, Leopold, 2002. "Nonparametric frontier estimation: a robust approach," Journal of Econometrics, Elsevier, vol. 106(1), pages 1-25, January.
When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:39:y:2010:i:1:p:67-78. See general information about how to correct material in RePEc.
If references are entirely missing, you can add them using this form.