Research, Development, and Engineering Metrics
AbstractWe seek to understand how the use of Research, Development, and Engineering (R,D&E) metrics can lead to more effective management of R,D&E. This paper combines qualitative and quantitative research to understand and improve the use of R,D&E metrics. Our research begins with interviews of 43 representative Chief Technical Officers, Chief Executive Offices, and researchers at 10 research-intensive international organizations. These interviews, and an extensive review of the literature, provide qualitative insights. Formal mathematical models attempt to explore these qualitative insights based on more general principles. Our research suggests that metrics-based evaluation and management vary according to the characteristics of the R,D&E activity. For applied projects, we find that project selection can be based on market-outcome metrics when firms use central subsidies to account for short-termism, risk aversion, and scope. With an efficient form of subsidies known as "tin-cupping," the business units have the incentives to choose the projects that are in the firm's best long-term interests. For core-technological development, longer time delays and more risky programs imply that popular R,D&E effectiveness metrics lead researchers to select programs that are not in the firm's long-term interest. Our analyses suggest that firms moderate such market-outcome metrics by placing a larger weight on metrics that attempt to measure research effort more directly. These metrics include standard measures such as publications, citations, patents, citations to patents, and peer review. For basic research, the issues shift to getting the right people and encouraging a breadth of ideas. Unfortunately, metrics that identify the "best people" based on research success lead directly to "not-invented-here" behaviors. Such behaviors result in research empires that are larger than necessary, but lead to fewer ideas. We suggest that firms use "research tourism" metrics, which encourage researchers to take advantage of research spillovers from universities, other industries, and, even, competitors.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoArticle provided by INFORMS in its journal Management Science.
Volume (Year): 44 (1998)
Issue (Month): 12-Part-1 (December)
Marketing; Research and Development; Product Development; Incentives;
You can help add them by filling out this form.
CitEc Project, subscribe to its RSS feed for this item.
- Cassiman, Bruno & di Guardo, Chiara & Valentini, Giovanni, 2010.
"Organizing links with science: Cooperate or contract? A project level analysis,"
Open Access publications from Katholieke Universiteit Leuven
urn:hdl:123456789/269656, Katholieke Universiteit Leuven.
- Cassiman, Bruno & Di Guardo, Maria Chiara & Valentini, Giovanni, 2010. "Organizing links with science: Cooperate or contract?: A project-level analysis," Research Policy, Elsevier, vol. 39(7), pages 882-892, September.
- Audia, Pino G. & Brion, Sebastien, 2007. "Reluctant to change: Self-enhancing responses to diverging performance measures," Organizational Behavior and Human Decision Processes, Elsevier, vol. 102(2), pages 255-269, March.
- Cherchye, L. & Abeele, P. Vanden, 2005.
"On research efficiency: A micro-analysis of Dutch university research in Economics and Business Management,"
Elsevier, vol. 34(4), pages 495-516, May.
- Laurens Cherchye & Piet Vanden Abeele, 2002. "On Research Efficiency: A Micro-Analysis of Dutch University Research in Economics and Business Management," Public Economics Working Paper Series ces0206, Katholieke Universiteit Leuven, Centrum voor Economische Studiën, Working Group Public Economics.
- Cherchye, Laurens & Vanden Abeele, Pierre, 2002. "On research efficiency: a micro-analysis of Dutch university research in economics and business management," Open Access publications from Katholieke Universiteit Leuven urn:hdl:123456789/138359, Katholieke Universiteit Leuven.
- Lazzarotti, Valentina & Manzini, Raffaella & Mari, Luca, 2011. "A model for R&D performance measurement," International Journal of Production Economics, Elsevier, vol. 134(1), pages 212-223, November.
- Hauser, John R. & Katz, Gerald M. & International Center for Research on the Management of Technology., 1998. "Metrics : you are what you measure!," Working papers 172-98, Massachusetts Institute of Technology (MIT), Sloan School of Management.
- Davila, Antonio, 2003. "Short-term economic incentives in new product development," Research Policy, Elsevier, vol. 32(8), pages 1397-1420, September.
- Subramaniam Ananthram & Cecil Pearson & Samir Chatterjee, 2010. "Do organisational reform measures impact on global mindset intensity of managers?: Empirical evidence from Indian and Chinese service industry managers," Journal of Chinese Economic and Foreign Trade Studies, Emerald Group Publishing, vol. 3(2), pages 146-168, June.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mirko Janc).
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If references are entirely missing, you can add them using this form.
If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.
Please note that corrections may take a couple of weeks to filter through the various RePEc services.