Review and new evidence on composite innovation indicators for evaluating national performance
AbstractThe purpose of this contribution is to present a survey of the recent developments in constructing composite science and technology (S&T) indicators on a national level as well as new evidence of the variability of such S&T indicators which opens the gateway to "country-tuning". It has become standard practice to combine several indicators for science, technology, and innovation to form composite numbers. Especially in the light of this variability, two questions arise. Firstly, are the results (especially rankings) stable with respect to weights? Secondly, is there hope to define "economically" reasonable weights? In order to provide answers to these questions, we use data from the European Innovation Scoreboard 2005 (EIS 2005) to exemplify our reasoning. Concerning the first question, we give genuine evidence on the existence of immense variability, possibly invalidating the results. Further, we also show that even existing and well-accepted methods, like equal weighting, Benefit of the Doubt weighting (BoD) and principal component analysis weighting (PCA) may lead to drastically differing results. Concerning the second question we will demonstrate that by each composite indicator weighting a set of shadow prices is implied expressing one indicator in terms of another. Whether the weights are sensible should be evaluated on the basis of these shadow prices. It turns out that those implied by EIS 2005 contain strange peculiarities. After that we plead for more care in constructing composite indicators. Especially weights should be chosen on the basis of shadow prices, rather than, say, by equal weighting or other automatic methods. Lastly, we discuss the merit of composite indicators and argue that they have a valuable communication and competition function, but they should be accompanied by multidimensional representations, which provide the basis for the construction of policy measures.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.
Bibliographic InfoArticle provided by Elsevier in its journal Research Policy.
Volume (Year): 39 (2010)
Issue (Month): 1 (February)
Contact details of provider:
Web page: http://www.elsevier.com/locate/respol
Composite indicators National innovation systems Science and technology indicators Scoreboards;
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Aghion, Philippe & Howitt, Peter, 1992.
"A Model of Growth through Creative Destruction,"
Econometric Society, vol. 60(2), pages 323-51, March.
- Aghion, P. & Howitt, P., 1990. "A Model Of Growth Through Creative Destruction," DELTA Working Papers 90-12, DELTA (Ecole normale supérieure).
- Philippe Aghion & Peter Howitt, 1990. "A Model of Growth Through Creative Destruction," NBER Working Papers 3223, National Bureau of Economic Research, Inc.
- Aghion, P. & Howitt, P., 1989. "A Model Of Growth Through Creative Destruction," Working papers 527, Massachusetts Institute of Technology (MIT), Department of Economics.
- Aghion, P. & Howitt, P., 1989. "A Model Of Growth Through Creative Destruction," UWO Department of Economics Working Papers 8904, University of Western Ontario, Department of Economics.
- Cherchye, Laurens & Knox Lovell, C.A. & Moesen, Wim & Van Puyenbroeck, Tom, 2007.
"One market, one number? A composite indicator assessment of EU internal market dynamics,"
European Economic Review,
Elsevier, vol. 51(3), pages 749-779, April.
- Laurens Cherchye & Knox Lovell & Wim Moesen & Tom Van Puyenbroeck, 2005. "One Market, One Number? A Composite Indicator Assessment of EU Internal Market Dynamics," Public Economics Working Paper Series ces0513, Katholieke Universiteit Leuven, Centrum voor Economische Studiën, Working Group Public Economics.
- Laurens Cherchye & Wim Moesen & Tom Van Puyenbroeck, 2003.
"Legitimately Diverse, yet Comparable: On Synthesising Social Inclusion Performance in the EU,"
Public Economics Working Paper Series
ces0301, Katholieke Universiteit Leuven, Centrum voor Economische Studiën, Working Group Public Economics.
- Laurens Cherchye & Wim Moesen & Tom Puyenbroeck, 2004. "Legitimately Diverse, yet Comparable: On Synthesizing Social Inclusion Performance in the EU," Journal of Common Market Studies, Wiley Blackwell, vol. 42(5), pages 919-955, December.
- Romer, Paul M, 1990.
"Endogenous Technological Change,"
Journal of Political Economy,
University of Chicago Press, vol. 98(5), pages S71-102, October.
- Cazals, Catherine & Florens, Jean-Pierre & Simar, Leopold, 2002. "Nonparametric frontier estimation: a robust approach," Journal of Econometrics, Elsevier, vol. 106(1), pages 1-25, January.
- Michael Freudenberg, 2003. "Composite Indicators of Country Performance: A Critical Assessment," OECD Science, Technology and Industry Working Papers 2003/16, OECD Publishing.
- Grupp, Hariolf & Mogee, Mary Ellen, 2004. "Indicators for national science and technology policy: how robust are composite indicators?," Research Policy, Elsevier, vol. 33(9), pages 1373-1384, November.
- Theodoros Papaioannou & Howard Rush & John Bessant, 2006. "Benchmarking as a policy-making tool: From the private to the public sector," Science and Public Policy, Oxford University Press, vol. 33(2), pages 91-102, March.
- Gene M. Grossman & Elhanan Helpman, 1993. "Innovation and Growth in the Global Economy," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262570971, January.
- Jon Zabala-Iturriagagoitia & Peter Voigt & Antonio Gutierrez-Gracia & Fernando Jimenez-Saez, 2007. "Regional Innovation Systems: How to Assess Performance," Regional Studies, Taylor & Francis Journals, vol. 41(5), pages 661-672.
- Grupp, Hariolf, 1994. "The measurement of technical performance of innovations by technometrics and its impact on established technology indicators," Research Policy, Elsevier, vol. 23(2), pages 175-193, March.
- Daniele Archibugi & Alberto Coco, 2004.
"A New Indicator of Technological Capabilities for Developed and Developing Countries (ArCo),"
CEIS Research Paper
44, Tor Vergata University, CEIS.
- Archibugi, Daniele & Coco, Alberto, 2004. "A New Indicator of Technological Capabilities for Developed and Developing Countries (ArCo)," World Development, Elsevier, vol. 32(4), pages 629-654, April.
- Daniele Archibugi & Alberto Coco, 2004. "A New Indicator of Technological Capabilities for Developed and Developing Countries (ArCo)," SPRU Working Paper Series 111, SPRU - Science and Technology Policy Research, University of Sussex.
- Romer, Paul M, 1987. "Growth Based on Increasing Returns Due to Specialization," American Economic Review, American Economic Association, vol. 77(2), pages 56-62, May.
- Frederik Booysen, 2002. "An Overview and Evaluation of Composite Indices of Development," Social Indicators Research, Springer, vol. 59(2), pages 115-151, August.
- Hollenstein, Heinz, 1996. "A composite indicator of a firm's innovativeness. An empirical analysis based on survey data for Swiss manufacturing," Research Policy, Elsevier, vol. 25(4), pages 633-645, June.
- Archibugi, Daniele & Coco, Alberto, 2005. "Measuring technological capabilities at the country level: A survey and a menu for choice," Research Policy, Elsevier, vol. 34(2), pages 175-194, March.
- Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
- Torben Schubert & Léopold Simar, 2011. "Innovation and export activities in the German mechanical engineering sector: an application of testing restrictions in production analysis," Journal of Productivity Analysis, Springer, vol. 36(1), pages 55-69, August.
- Filippetti, Andrea & Peyrache, Antonio, 2010. "The Dynamic of Technological Capabilities of Countries: A Dual Approach Using Composite Indicators & Data Envelopment Analysis," MPRA Paper 21629, University Library of Munich, Germany.
- Filippetti, Andrea & Peyrache, Antonio, 2011. "The Patterns of Technological Capabilities of Countries: A Dual Approach using Composite Indicators and Data Envelopment Analysis," World Development, Elsevier, vol. 39(7), pages 1108-1121, July.
- Blind, Knut, 2012. "The influence of regulations on innovation: A quantitative assessment for OECD countries," Research Policy, Elsevier, vol. 41(2), pages 391-400.
- Wu, Wei-Wen & Lan, Lawrence W. & Lee, Yu-Ting, 2012. "Exploring the critical pillars and causal relations within the NRI: An innovative approach," European Journal of Operational Research, Elsevier, vol. 218(1), pages 230-238.
- Nuno Boavida, 2011. "How composite indicators of innovation can influence technology policy decision?," IET Working Papers Series 03/2011, Universidade Nova de Lisboa, IET/CESNOVA-Research on Enterprise and Work Innovation, Faculty of Science and Technology.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei).
If references are entirely missing, you can add them using this form.