IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/5478.html
   My bibliography  Save this paper

Measuring Science: An Exploration

Author

Listed:
  • James Adams
  • Zvi Griliches

Abstract

This paper examines available U.S. data on academic R&D expenditures and the number of papers published and the number of citations to these papers as possible measures of `output' of this enterprise. We look at these numbers for science and engineering as a whole, for 5 selected major fields, and at the individual university-field level. The published data in Science and Engineering Indicators imply sharply diminishing returns to academic R&D using published papers as a 'output' measure. These data are problematic. Using a newer set of data on papers and citations, based on an `expanding' set changes the picture drastically, eliminating seemingly diminishing returns but raising the question of why input prices of academic R&D are rising so much faster than either the GDP deflator or the implicit R&D deflator in industry. A production function analysis of such data indicates significant diminishing returns to `own' R&D, with the R&D coefficients hovering around 0.5 for estimates with paper numbers as the dependent variable and around 0.6 if total citations are used. When we substitute scientists and engineers in place of R&D as the right hand side variables, the coefficient on papers rises from 0.5 to 0.8, and the coefficient on citations rises from 0.6 to 0.9, indicating systematic measurement problems with R&D as the sole input into the production of scientific output. But allowing for individual university-field effects drives these numbers down below unity. Since in the aggregate both paper numbers and citations are growing as fast or faster than R&D, this can be seen as leaving a major, yet unmeasured role, for the contribution of spill- overs from other fields, other universities, and other countries.

Suggested Citation

  • James Adams & Zvi Griliches, 1996. "Measuring Science: An Exploration," NBER Working Papers 5478, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:5478
    Note: PR
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w5478.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Richard Nelson, 1962. "The Link Between Science and Invention: The Case of the Transistor," NBER Chapters, in: The Rate and Direction of Inventive Activity: Economic and Social Factors, pages 549-584, National Bureau of Economic Research, Inc.
    2. Zvi Griliches, 1998. "Productivity, R&D, and the Data Constraint," NBER Chapters, in: R&D and Productivity: The Econometric Evidence, pages 347-374, National Bureau of Economic Research, Inc.
    3. Mansfield, Edwin, 1991. "Academic research and industrial innovation," Research Policy, Elsevier, vol. 20(1), pages 1-12, February.
    4. Dale W. Jorgenson & Barbara M. Fraumeni, 1992. "The Output of the Education Sector," NBER Chapters, in: Output Measurement in the Service Sectors, pages 303-341, National Bureau of Economic Research, Inc.
    5. Pardey, Philip G, 1989. "The Agricultural Knowledge Production Function: An Empirical Look," The Review of Economics and Statistics, MIT Press, vol. 71(3), pages 453-461, August.
    6. Stephan, Paula E., 2010. "The Economics of Science," Handbook of the Economics of Innovation, in: Bronwyn H. Hall & Nathan Rosenberg (ed.), Handbook of the Economics of Innovation, edition 1, volume 1, chapter 0, pages 217-273, Elsevier.
    7. Adams, James D, 1993. "Science, R&D, and Invention Potential Recharge: U.S. Evidence," American Economic Review, American Economic Association, vol. 83(2), pages 458-462, May.
    8. Rosenberg, Nathan & Nelson, Richard R., 1994. "American universities and technical advance in industry," Research Policy, Elsevier, vol. 23(3), pages 323-348, May.
    9. Zvi Griliches, 1958. "Research Costs and Social Returns: Hybrid Corn and Related Innovations," Journal of Political Economy, University of Chicago Press, vol. 66, pages 419-419.
    10. Adams, James D, 1990. "Fundamental Stocks of Knowledge and Productivity Growth," Journal of Political Economy, University of Chicago Press, vol. 98(4), pages 673-702, August.
    11. George J. Stigler & Claire Freidland, 1979. "The Pattern of Citation Practices in Economics," History of Political Economy, Duke University Press, vol. 11(1), pages 1-20, Spring.
    12. Weisbrod, Burton A, 1971. "Costs and Benefits of Medical Research: A Case Study of Poliomyelitis," Journal of Political Economy, University of Chicago Press, vol. 79(3), pages 527-544, May-June.
    13. Richard C. Levin & Alvin K. Klevorick & Richard R. Nelson & Sidney G. Winter, 1987. "Appropriating the Returns from Industrial Research and Development," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 18(3, Specia), pages 783-832.
    14. Richard C. Levin & Alvin K. Klevorick & Richard R. Nelson & Sidney G. Winter, 1988. "Appropriating the Returns from Industrial R&D," Cowles Foundation Discussion Papers 862, Cowles Foundation for Research in Economics, Yale University.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Saul Lach & Mark Schankerman, 2008. "Incentives and invention in universities," RAND Journal of Economics, RAND Corporation, vol. 39(2), pages 403-433, June.
    2. Hinloopen, Jeroen & Smrkolj, Grega & Wagener, Florian, 2013. "From mind to market: A global, dynamic analysis of R&D," Journal of Economic Dynamics and Control, Elsevier, vol. 37(12), pages 2729-2754.
    3. Carayol, Nicolas & Matt, Mireille, 2006. "Individual and collective determinants of academic scientists' productivity," Information Economics and Policy, Elsevier, vol. 18(1), pages 55-72, March.
    4. Nathan Berg, 2002. "Coping with journal-price inflation: leading policy proposals and the quality-spectrum," Economics Bulletin, AccessEcon, vol. 4(14), pages 1-7.
    5. Lee, Kyootai & Jung, Hyun Ju, 2021. "Does TTO capability matter in commercializing university technology? Evidence from longitudinal data in South Korea," Research Policy, Elsevier, vol. 50(1).
    6. Paul Heisey & Sarah Adelman, 2011. "Research expenditures, technology transfer activity, and university licensing revenue," The Journal of Technology Transfer, Springer, vol. 36(1), pages 38-60, February.
    7. Graddy-Reed, Alexandra & Lanahan, Lauren & Eyer, Jonathan, 2019. "Gender discrepancies in publication productivity of high-performing life science graduate students," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    8. Yann Kossi & Jean-Yves Lesueur & Mareva Sabatier, 2016. "Publish or teach? The role of the scientific environment on academics’ multitasking," Industrial and Corporate Change, Oxford University Press, vol. 25(3), pages 487-506.
    9. Zhang, Daqun & Banker, Rajiv D. & Li, Xiaoxuan & Liu, Wenbin, 2011. "Performance impact of research policy at the Chinese Academy of Sciences," Research Policy, Elsevier, vol. 40(6), pages 875-885, July.
    10. Maryann P. Feldman & Frank R. Lichtenberg, 1998. "The Impact and Organization of Publicly-Funded Research and development in the European Community," Annals of Economics and Statistics, GENES, issue 49-50, pages 199-222.
    11. Pierre Azoulay, 2002. "Do Pharmaceutical Sales Respond to Scientific Evidence?," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 11(4), pages 551-594, December.
    12. Ashish Arora & Alfonso Gambardella, 2010. "The Impact of NSF Support for Basic Research in Economics," NBER Chapters, in: Contributions in Memory of Zvi Griliches, pages 91-115, National Bureau of Economic Research, Inc.
    13. Sood, Ashish & Kappe, Eelco & Stremersch, Stefan, 2014. "The commercial contribution of clinical studies for pharmaceutical drugs," International Journal of Research in Marketing, Elsevier, vol. 31(1), pages 65-77.
    14. Carayol, Nicolas & Matt, Mireille, 2004. "Does research organization influence academic production?: Laboratory level evidence from a large European university," Research Policy, Elsevier, vol. 33(8), pages 1081-1102, October.
    15. repec:ebl:ecbull:v:4:y:2002:i:14:p:1-7 is not listed on IDEAS
    16. Oguz K. Baskurt, 2011. "Time series analysis of publication counts of a university: what are the implications?," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(3), pages 645-656, March.
    17. Robert D. Shelton, 2008. "Relations between national research investment and publication output: Application to an American Paradox," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 191-205, February.
    18. Arthur Diamond, 2004. "Zvi Griliches's contributions to the economics of technology and growth," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 13(4), pages 365-397.
    19. Stefano Breschi & Francesco Lissoni & Fabio Montobbio, 2006. "University patenting and scientific productivity. A quantitative study of Italian academic inventors," KITeS Working Papers 189, KITeS, Centre for Knowledge, Internationalization and Technology Studies, Universita' Bocconi, Milano, Italy, revised Nov 2006.
    20. Azagra-Caro, Joaquín M. & Tijssen, Robert J.W. & Tur, Elena M. & Yegros-Yegros, Alfredo, 2019. "University-industry scientific production and the Great Recession," Technological Forecasting and Social Change, Elsevier, vol. 139(C), pages 210-220.
    21. Gregory J Hather & Winston Haynes & Roger Higdon & Natali Kolker & Elizabeth A Stewart & Peter Arzberger & Patrick Chain & Dawn Field & B Robert Franza & Biaoyang Lin & Folker Meyer & Vural Ozdemir & , 2010. "The United States of America and Scientific Research," PLOS ONE, Public Library of Science, vol. 5(8), pages 1-9, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cohen, Wesley M., 2010. "Fifty Years of Empirical Studies of Innovative Activity and Performance," Handbook of the Economics of Innovation, in: Bronwyn H. Hall & Nathan Rosenberg (ed.), Handbook of the Economics of Innovation, edition 1, volume 1, chapter 0, pages 129-213, Elsevier.
    2. Becker Wolfgang & Peters Jürgen, 2005. "Innovation Effects of Science-Related Technological Opportunities / Innovationseffekte von technologischen Möglichkeiten aus dem Wissenschaftsbereich: Theoretical Considerations and Empirical Findings," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 225(2), pages 130-150, April.
    3. Hall, Bronwyn H. & Mairesse, Jacques & Mohnen, Pierre, 2010. "Measuring the Returns to R&D," Handbook of the Economics of Innovation, in: Bronwyn H. Hall & Nathan Rosenberg (ed.), Handbook of the Economics of Innovation, edition 1, volume 2, chapter 0, pages 1033-1082, Elsevier.
    4. Iain M. Cockburn & Rebecca M. Henderson, 2001. "Publicly Funded Science and the Productivity of the Pharmaceutical Industry," NBER Chapters, in: Innovation Policy and the Economy, Volume 1, pages 1-34, National Bureau of Economic Research, Inc.
    5. Albino, Vito & Ardito, Lorenzo & Dangelico, Rosa Maria & Messeni Petruzzelli, Antonio, 2014. "Understanding the development trends of low-carbon energy technologies: A patent analysis," Applied Energy, Elsevier, vol. 135(C), pages 836-854.
    6. Juergen Peters & Wolfgang Becker, 1998. "Technological Opportunities, Academic Research, and Innovation Activities in the German Automobile Supply Industry," Discussion Paper Series 175, Universitaet Augsburg, Institute for Economics.
    7. Scott Stern, 1999. "Do Scientists Pay to Be Scientists?," NBER Working Papers 7410, National Bureau of Economic Research, Inc.
    8. Wolfgang Becker, 2003. "Evaluation of the Role of Universities in the Innovation Process," Discussion Paper Series 241, Universitaet Augsburg, Institute for Economics.
    9. Stephan, Paula E., 2010. "The Economics of Science," Handbook of the Economics of Innovation, in: Bronwyn H. Hall & Nathan Rosenberg (ed.), Handbook of the Economics of Innovation, edition 1, volume 1, chapter 0, pages 217-273, Elsevier.
    10. Beise, Marian & Stahl, Harald, 1999. "Public research and industrial innovations in Germany," Research Policy, Elsevier, vol. 28(4), pages 397-422, April.
    11. Beck, Mathias & Junge, Martin & Kaiser, Ulrich, 2017. "Public Funding and Corporate Innovation," IZA Discussion Papers 11196, Institute of Labor Economics (IZA).
    12. Lee Branstetter & Kwon Hyeog Ug, 2004. "The Restructuring Of Japanese Research And Development: The Increasing Impact Of Science On Japanese R&D," Discussion papers 04021, Research Institute of Economy, Trade and Industry (RIETI).
    13. Maria Rosaria Carillo & Erasmo Papagni, 2004. "Incentive Structure In Basic Research And Economic Growth," Working Papers 9_2004, D.E.S. (Department of Economic Studies), University of Naples "Parthenope", Italy.
    14. Daniel J. Wilson, 2002. "Is Embodied Technology the Result of Upstream R&D? Industry-Level Evidence," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 5(2), pages 285-317, April.
    15. Bronwyn H. Hall & Albert N. Link & John T. Scott, 2003. "Universities as Research Partners," The Review of Economics and Statistics, MIT Press, vol. 85(2), pages 485-491, May.
    16. Michele Cincera, 2005. "Firms' productivity growth and R&D spillovers: An analysis of alternative technological proximity measures," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 14(8), pages 657-682.
    17. David, Paul A. & Hall, Bronwyn H. & Toole, Andrew A., 2000. "Is public R&D a complement or substitute for private R&D? A review of the econometric evidence," Research Policy, Elsevier, vol. 29(4-5), pages 497-529, April.
    18. Thursby, Marie & Thursby, Jerry & Gupta-Mukherjee, Swasti, 2007. "Are there real effects of licensing on academic research? A life cycle view," Journal of Economic Behavior & Organization, Elsevier, vol. 63(4), pages 577-598, August.
    19. Castelnovo, Paolo & Florio, Massimo & Forte, Stefano & Rossi, Lucio & Sirtori, Emanuela, 2018. "The economic impact of technological procurement for large-scale research infrastructures: Evidence from the Large Hadron Collider at CERN," Research Policy, Elsevier, vol. 47(9), pages 1853-1867.
    20. Wesley M. Cohen & Richard R. Nelson & John P. Walsh, 2003. "Links and Impacts: The Influence of Public Research on Industrial R&D," Chapters, in: Aldo Geuna & Ammon J. Salter & W. Edward Steinmueller (ed.), Science and Innovation, chapter 4, Edward Elgar Publishing.

    More about this item

    JEL classification:

    • O47 - Economic Development, Innovation, Technological Change, and Growth - - Economic Growth and Aggregate Productivity - - - Empirical Studies of Economic Growth; Aggregate Productivity; Cross-Country Output Convergence
    • H40 - Public Economics - - Publicly Provided Goods - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:5478. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.