Review and new evidence on composite innovation indicators for evaluating national performance
AbstractThe purpose of this contribution is to present a survey of the recent developments in constructing composite science and technology (S&T) indicators on a national level as well as new evidence of the variability of such S&T indicators which opens the gateway to "country-tuning". It has become standard practice to combine several indicators for science, technology, and innovation to form composite numbers. Especially in the light of this variability, two questions arise. Firstly, are the results (especially rankings) stable with respect to weights? Secondly, is there hope to define "economically" reasonable weights? In order to provide answers to these questions, we use data from the European Innovation Scoreboard 2005 (EIS 2005) to exemplify our reasoning. Concerning the first question, we give genuine evidence on the existence of immense variability, possibly invalidating the results. Further, we also show that even existing and well-accepted methods, like equal weighting, Benefit of the Doubt weighting (BoD) and principal component analysis weighting (PCA) may lead to drastically differing results. Concerning the second question we will demonstrate that by each composite indicator weighting a set of shadow prices is implied expressing one indicator in terms of another. Whether the weights are sensible should be evaluated on the basis of these shadow prices. It turns out that those implied by EIS 2005 contain strange peculiarities. After that we plead for more care in constructing composite indicators. Especially weights should be chosen on the basis of shadow prices, rather than, say, by equal weighting or other automatic methods. Lastly, we discuss the merit of composite indicators and argue that they have a valuable communication and competition function, but they should be accompanied by multidimensional representations, which provide the basis for the construction of policy measures.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.
Bibliographic InfoArticle provided by Elsevier in its journal Research Policy.
Volume (Year): 39 (2010)
Issue (Month): 1 (February)
Contact details of provider:
Web page: http://www.elsevier.com/locate/respol
Composite indicators National innovation systems Science and technology indicators Scoreboards;
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Archibugi, Daniele & Coco, Alberto, 2005. "Measuring technological capabilities at the country level: A survey and a menu for choice," Research Policy, Elsevier, vol. 34(2), pages 175-194, March.
- Cherchye, Laurens & Knox Lovell, C.A. & Moesen, Wim & Van Puyenbroeck, Tom, 2007.
"One market, one number? A composite indicator assessment of EU internal market dynamics,"
European Economic Review,
Elsevier, vol. 51(3), pages 749-779, April.
- Laurens Cherchye & Knox Lovell & Wim Moesen & Tom Van Puyenbroeck, 2005. "One Market, One Number? A Composite Indicator Assessment of EU Internal Market Dynamics," Public Economics Working Paper Series ces0513, Katholieke Universiteit Leuven, Centrum voor Economische Studiën, Working Group Public Economics.
- Cherchye, Laurens & Knox, ovell, CA & Moesen, Willem & Van Puyenbroeck, Tom, 2005. "One market, one number? A composite indicator assessment of EU internal market dynamics," Open Access publications from Katholieke Universiteit Leuven urn:hdl:123456789/119340, Katholieke Universiteit Leuven.
- Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
- Aghion, P. & Howitt, P., 1989.
"A Model Of Growth Through Creative Destruction,"
527, Massachusetts Institute of Technology (MIT), Department of Economics.
- Philippe Aghion & Peter Howitt, 1990. "A Model of Growth Through Creative Destruction," NBER Working Papers 3223, National Bureau of Economic Research, Inc.
- Aghion, P. & Howitt, P., 1990. "A Model Of Growth Through Creative Destruction," DELTA Working Papers 90-12, DELTA (Ecole normale supérieure).
- Aghion, P. & Howitt, P., 1989. "A Model Of Growth Through Creative Destruction," UWO Department of Economics Working Papers 8904, University of Western Ontario, Department of Economics.
- Gene M. Grossman & Elhanan Helpman, 1993. "Innovation and Growth in the Global Economy," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262570971, June.
- Romer, Paul M, 1990.
"Endogenous Technological Change,"
Journal of Political Economy,
University of Chicago Press, vol. 98(5), pages S71-102, October.
- Archibugi, Daniele & Coco, Alberto, 2004. "A New Indicator of Technological Capabilities for Developed and Developing Countries (ArCo)," World Development, Elsevier, vol. 32(4), pages 629-654, April.
- Hollenstein, Heinz, 1996. "A composite indicator of a firm's innovativeness. An empirical analysis based on survey data for Swiss manufacturing," Research Policy, Elsevier, vol. 25(4), pages 633-645, June.
- Grupp, Hariolf & Mogee, Mary Ellen, 2004. "Indicators for national science and technology policy: how robust are composite indicators?," Research Policy, Elsevier, vol. 33(9), pages 1373-1384, November.
- Frederik Booysen, 2002. "An Overview and Evaluation of Composite Indices of Development," Social Indicators Research, Springer, vol. 59(2), pages 115-151, August.
- Michael Freudenberg, 2003. "Composite Indicators of Country Performance: A Critical Assessment," OECD Science, Technology and Industry Working Papers 2003/16, OECD Publishing.
- Grupp, Hariolf, 1994. "The measurement of technical performance of innovations by technometrics and its impact on established technology indicators," Research Policy, Elsevier, vol. 23(2), pages 175-193, March.
- Theodoros Papaioannou & Howard Rush & John Bessant, 2006. "Benchmarking as a policy-making tool: From the private to the public sector," Science and Public Policy, Oxford University Press, vol. 33(2), pages 91-102, March.
- Daniele Archibugi & Alberto Coco, 2004. "A New Indicator of Technological Capabilities for Developed and Developing Countries (ArCo)," CEIS Research Paper 44, Tor Vergata University, CEIS.
- Romer, Paul M, 1987. "Growth Based on Increasing Returns Due to Specialization," American Economic Review, American Economic Association, vol. 77(2), pages 56-62, May.
- Laurens Cherchye & Wim Moesen & Tom Van Puyenbroeck, 2003.
"Legitimately Diverse, yet Comparable: On Synthesising Social Inclusion Performance in the EU,"
Public Economics Working Paper Series
ces0301, Katholieke Universiteit Leuven, Centrum voor Economische Studiën, Working Group Public Economics.
- Laurens Cherchye & Wim Moesen & Tom Puyenbroeck, 2004. "Legitimately Diverse, yet Comparable: On Synthesizing Social Inclusion Performance in the EU," Journal of Common Market Studies, Wiley Blackwell, vol. 42(5), pages 919-955, December.
- Cazals, Catherine & Florens, Jean-Pierre & Simar, Leopold, 2002.
"Nonparametric frontier estimation: a robust approach,"
Journal of Econometrics,
Elsevier, vol. 106(1), pages 1-25, January.
- Cazals, Catherine & Florens, Jean-Pierre & Simar, Léopold, 2002. "Nonparametric Frontier Estimation: A Robust Approach," Open Access publications from University of Toulouse 1 Capitole http://neeo.univ-tlse1.fr, University of Toulouse 1 Capitole.
- Daniele Archibugi & Alberto Coco, 2004. "A New Indicator of Technological Capabilities for Developed and Developing Countries (ArCo)," SPRU Working Paper Series 111, SPRU - Science and Technology Policy Research, University of Sussex.
- Jon Zabala-Iturriagagoitia & Peter Voigt & Antonio Gutierrez-Gracia & Fernando Jimenez-Saez, 2007. "Regional Innovation Systems: How to Assess Performance," Regional Studies, Taylor and Francis Journals, vol. 41(5), pages 661-672.
- Filippetti, Andrea & Peyrache, Antonio, 2010. "The Dynamic of Technological Capabilities of Countries: A Dual Approach Using Composite Indicators & Data Envelopment Analysis," MPRA Paper 21629, University Library of Munich, Germany.
- Nuno Boavida, 2011. "How composite indicators of innovation can influence technology policy decision?," IET Working Papers Series 03/2011, Universidade Nova de Lisboa, IET/CESNOVA-Research on Enterprise and Work Innovation, Faculty of Science and Technology.
- Torben Schubert & Léopold Simar, 2011. "Innovation and export activities in the German mechanical engineering sector: an application of testing restrictions in production analysis," Journal of Productivity Analysis, Springer, vol. 36(1), pages 55-69, August.
- Wu, Wei-Wen & Lan, Lawrence W. & Lee, Yu-Ting, 2012. "Exploring the critical pillars and causal relations within the NRI: An innovative approach," European Journal of Operational Research, Elsevier, vol. 218(1), pages 230-238.
- Blind, Knut, 2012. "The influence of regulations on innovation: A quantitative assessment for OECD countries," Research Policy, Elsevier, vol. 41(2), pages 391-400.
- Filippetti, Andrea & Peyrache, Antonio, 2011. "The Patterns of Technological Capabilities of Countries: A Dual Approach using Composite Indicators and Data Envelopment Analysis," World Development, Elsevier, vol. 39(7), pages 1108-1121, July.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Wendy Shamier).
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If references are entirely missing, you can add them using this form.
If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.
Please note that corrections may take a couple of weeks to filter through the various RePEc services.