IDEAS home Printed from https://ideas.repec.org/a/eee/tefoso/v197y2023ics0040162523005590.html
   My bibliography  Save this article

Using big data for generating firm-level innovation indicators - a literature review

Author

Listed:
  • Rammer, Christian
  • Es-Sadki, Nordine

Abstract

Obtaining indicators on the innovation activities of firms has been a challenge in economic research for a long time. The most frequently used indicators - R&D expenditures and patents - provide an incomplete picture as they represent inputs in the innovation process. Output measurement of innovation has strongly relied on survey data such as the Community Innovation Survey (CIS). However, this type of data suffers from several shortcomings typical of surveys, including incomplete coverage of the business sector, subjectivity concerns, low timeliness, and limited comparability across industries and firms. An alternative that has attracted growing interest is to use big data sources to collect innovation data at the firm level. This paper discusses recent attempts to use digital big data sources including websites and social media to generate firm-level innovation indicators. It summarises the main challenges of using big data and proposes practical guidelines for their use, including a research agenda that should be useful to practitioners as well as users of statistics derived from big data.

Suggested Citation

  • Rammer, Christian & Es-Sadki, Nordine, 2023. "Using big data for generating firm-level innovation indicators - a literature review," Technological Forecasting and Social Change, Elsevier, vol. 197(C).
    • Handle: RePEc:eee:tefoso:v:197:y:2023:i:c:s0040162523005590
    DOI: 10.1016/j.techfore.2023.122874
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0040162523005590
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.techfore.2023.122874?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Albert, Till & Moehrle, Martin G. & Meyer, Stefan, 2015. "Technology maturity assessment based on blog analysis," Technological Forecasting and Social Change, Elsevier, vol. 92(C), pages 196-209.
    2. Bastian Krieger & Maikel Pellens & Knut Blind & Sonia Gruber & Torben Schubert, 2021. "Are firms withdrawing from basic research? An analysis of firm-level publication behaviour in Germany," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9677-9698, December.
    3. Ulrich Schmoch, 2003. "Service marks as novel innovation indicator," Research Evaluation, Oxford University Press, vol. 12(2), pages 149-156, August.
    4. Schubert, Torben & Jäger, Angela & Türkeli, Serdar & Visentin, Fabiana, 2020. "Addressing the productivity paradox with big data: A literature review and adaptation of the CDM econometric model," MERIT Working Papers 2020-050, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    5. Cojoianu, Theodor F. & Clark, Gordon L. & Hoepner, Andreas G.F. & Veneri, Paolo & Wójcik, Dariusz, 2020. "Entrepreneurs for a low carbon world: How environmental knowledge and policy shape the creation and financing of green start-ups," Research Policy, Elsevier, vol. 49(6).
    6. Roger C. Brackin & Michael J. Jackson & Andrew Leyshon & Jeremy G. Morley & Sarah Jewitt, 2022. "Generating Indicators of Disruptive Innovation Using Big Data," Future Internet, MDPI, vol. 14(11), pages 1-24, November.
    7. G. M.P. Swann, 2009. "The Economics of Innovation," Books, Edward Elgar Publishing, number 13211.
    8. Martin Obschonka & David B. Audretsch, 0. "Artificial intelligence and big data in entrepreneurship: a new era has begun," Small Business Economics, Springer, vol. 0, pages 1-11.
    9. Tether, Bruce S., 2002. "Who co-operates for innovation, and why: An empirical analysis," Research Policy, Elsevier, vol. 31(6), pages 947-967, August.
    10. Anthony Arundel & Keith Smith, 2013. "History of the Community Innovation Survey," Chapters, in: Fred Gault (ed.), Handbook of Innovation Indicators and Measurement, chapter 3, pages 60-87, Edward Elgar Publishing.
    11. Christian Rammer & Dirk Czarnitzki & Alfred Spielkamp, 2009. "Innovation success of non-R&D-performers: substituting technology by management in SMEs," Small Business Economics, Springer, vol. 33(1), pages 35-58, June.
    12. Alfred Kleinknecht & Kees Van Montfort & Erik Brouwer, 2002. "The Non-Trivial Choice between Innovation Indicators," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 11(2), pages 109-121.
    13. Sanjay K. Arora & Jan Youtie & Philip Shapira & Lidan Gao & TingTing Ma, 2013. "Entry strategies in an emerging technology: a pilot web-based study of graphene firms," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 1189-1207, June.
    14. Geroski, P. A. & Van Reenen, J. & Walters, C. F., 1997. "How persistently do firms innovate?," Research Policy, Elsevier, vol. 26(1), pages 33-48, March.
    15. Oliver Som, 2012. "Innovation without R&D," Springer Books, Springer, number 978-3-8349-3492-5, January.
    16. Mendonca, Sandro & Pereira, Tiago Santos & Godinho, Manuel Mira, 2004. "Trademarks as an indicator of innovation and industrial change," Research Policy, Elsevier, vol. 33(9), pages 1385-1404, November.
    17. Ghasemaghaei, Maryam & Calic, Goran, 2020. "Assessing the impact of big data on firm innovation performance: Big data is not always better data," Journal of Business Research, Elsevier, vol. 108(C), pages 147-162.
    18. Chang, Victor, 2021. "An ethical framework for big data and smart cities," Technological Forecasting and Social Change, Elsevier, vol. 165(C).
    19. Misirlis, Nikolaos & Vlachopoulou, Maro, 2018. "Social media metrics and analytics in marketing – S3M: A mapping literature review," International Journal of Information Management, Elsevier, vol. 38(1), pages 270-276.
    20. Thomas Niebel & Fabienne Rasel & Steffen Viete, 2019. "BIG data – BIG gains? Understanding the link between big data analytics and innovation," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 28(3), pages 296-316, April.
    21. Henriette Ruhrmann & Michael Fritsch & Loet Leydesdorff, 2022. "Synergy and policy-making in German innovation systems: Smart Specialisation Strategies at national, regional, local levels?," Regional Studies, Taylor & Francis Journals, vol. 56(9), pages 1468-1479, September.
    22. Andersson, Martin & Johansson, Borje & Karlsson, Charlie & Loof, Hans (ed.), 2012. "Innovation and Growth: From R&D Strategies of Innovating Firms to Economy-wide Technological Change," OUP Catalogue, Oxford University Press, number 9780199646685.
    23. Johannes Bloh & Tom Broekel & Burcu Özgun & Rolf Sternberg, 2020. "New(s) data for entrepreneurship research? An innovative approach to use Big Data on media coverage," Small Business Economics, Springer, vol. 55(3), pages 673-694, October.
    24. Kinne, Jan & Axenbeck, Janna, 2018. "Web mining of firm websites: A framework for web scraping and a pilot study for Germany," ZEW Discussion Papers 18-033, ZEW - Leibniz Centre for European Economic Research.
    25. Arundel, Anthony & Kabla, Isabelle, 1998. "What percentage of innovations are patented? empirical estimates for European firms," Research Policy, Elsevier, vol. 27(2), pages 127-141, June.
    26. Jean-Michel Dalle & Matthijs den Besten & Carlo Menon, 2017. "Using Crunchbase for economic and managerial research," OECD Science, Technology and Industry Working Papers 2017/08, OECD Publishing.
    27. Ilaria Gandin & Claudio Cozza, 2019. "Can we predict firms’ innovativeness? The identification of innovation performers in an Italian region through a supervised learning approach," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-16, June.
    28. Kinne, Jan & Lenz, David, 2019. "Predicting innovative firms using web mining and deep learning," ZEW Discussion Papers 19-001, ZEW - Leibniz Centre for European Economic Research.
    29. Samuel Pinto Ribeiro & Stefano Menghinello & Koen De Backer, 2010. "The OECD ORBIS Database: Responding to the Need for Firm-Level Micro-Data in the OECD," OECD Statistics Working Papers 2010/1, OECD Publishing.
    30. Crass, Dirk, 2014. "Which firms use trademarks - and why? Representative firm-level evidence from Germany," ZEW Discussion Papers 14-118, ZEW - Leibniz Centre for European Economic Research.
    31. Gaizka Garechana & Rosa Río-Belver & Iñaki Bildosola & Marisela Rodríguez Salvador, 2017. "Effects of innovation management system standardization on firms: evidence from text mining annual reports," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1987-1999, June.
    32. Ritu Agarwal & Vasant Dhar, 2014. "Editorial —Big Data, Data Science, and Analytics: The Opportunity and Challenge for IS Research," Information Systems Research, INFORMS, vol. 25(3), pages 443-448, September.
    33. Bhimani, Hardik & Mention, Anne-Laure & Barlatier, Pierre-Jean, 2019. "Social media and innovation: A systematic literature review and future research directions," Technological Forecasting and Social Change, Elsevier, vol. 144(C), pages 251-269.
    34. Zvi Griliches, 1998. "Patent Statistics as Economic Indicators: A Survey," NBER Chapters, in: R&D and Productivity: The Econometric Evidence, pages 287-343, National Bureau of Economic Research, Inc.
    35. Sanjay K. Arora & Yin Li & Jan Youtie & Philip Shapira, 2016. "Using the wayback machine to mine websites in the social sciences: A methodological resource," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(8), pages 1904-1915, August.
    36. Alfred Kleinknecht & Jeroen O. N. Reijnen & Wendy Smits, 1993. "Collecting Literature-based Innovation Output Indicators. The Experience in the Netherlands," Palgrave Macmillan Books, in: Alfred Kleinknecht & Donald Bain (ed.), New Concepts in Innovation Output Measurement, chapter 3, pages 42-84, Palgrave Macmillan.
    37. Jan Kinne & Janna Axenbeck, 2020. "Web mining for innovation ecosystem mapping: a framework and a large-scale pilot study," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2011-2041, December.
    38. Breithaupt, Patrick & Kesler, Reinhold & Niebel, Thomas & Rammer, Christian, 2020. "Intangible capital indicators based on web scraping of social media," ZEW Discussion Papers 20-046, ZEW - Leibniz Centre for European Economic Research.
    39. Fred Gault (ed.), 2013. "Handbook of Innovation Indicators and Measurement," Books, Edward Elgar Publishing, number 14427.
    40. Cohen, Wesley M., 2010. "Fifty Years of Empirical Studies of Innovative Activity and Performance," Handbook of the Economics of Innovation, in: Bronwyn H. Hall & Nathan Rosenberg (ed.), Handbook of the Economics of Innovation, edition 1, volume 1, chapter 0, pages 129-213, Elsevier.
    41. Manfred Bruhn & Verena Schoenmueller & Daniela B. Schäfer, 2012. "Are social media replacing traditional media in terms of brand equity creation?," Management Research Review, Emerald Group Publishing Limited, vol. 35(9), pages 770-790, August.
    42. Cirera, Xavier & Muzi, Silvia, 2020. "Measuring innovation using firm-level surveys: Evidence from developing countries✰," Research Policy, Elsevier, vol. 49(3).
    43. Seshadri Tirunillai & Gerard J. Tellis, 2012. "Does Chatter Really Matter? Dynamics of User-Generated Content and Stock Performance," Marketing Science, INFORMS, vol. 31(2), pages 198-215, March.
    44. Meyer-Krahmer, Frieder, 1984. "Recent results in measuring innovation output," Research Policy, Elsevier, vol. 13(3), pages 175-182, June.
    45. Marco Guerzoni & Consuelo R. Nava & Massimiliano Nuccio, 2021. "Start-ups survival through a crisis. Combining machine learning with econometrics to measure innovation," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 30(5), pages 468-493, July.
    46. Abdullah Gök & Alec Waterworth & Philip Shapira, 2015. "Use of web mining in studying innovation," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 653-671, January.
    47. Anthony Arundel & Kieran O’Brien & Ann Torugsa, 2013. "How firm managers understand innovation: implications for the design of innovation surveys," Chapters, in: Fred Gault (ed.), Handbook of Innovation Indicators and Measurement, chapter 4, pages 88-108, Edward Elgar Publishing.
    48. Yang, Guancan & Lu, Guoxuan & Xu, Shuo & Chen, Liang & Wen, Yuxin, 2023. "Which type of dynamic indicators should be preferred to predict patent commercial potential?," Technological Forecasting and Social Change, Elsevier, vol. 193(C).
    49. Dziallas, Marisa & Blind, Knut, 2019. "Innovation indicators throughout the innovation process: An extensive literature analysis," Technovation, Elsevier, vol. 80, pages 3-29.
    50. Martin Obschonka & David B. Audretsch, 2020. "Artificial intelligence and big data in entrepreneurship: a new era has begun," Small Business Economics, Springer, vol. 55(3), pages 529-539, October.
    51. Crass, Dirk, 2014. "The impact of brand use on innovation performance: Empirical results for Germany," ZEW Discussion Papers 14-119, ZEW - Leibniz Centre for European Economic Research.
    52. Kahn, Kenneth B., 2018. "Understanding innovation," Business Horizons, Elsevier, vol. 61(3), pages 453-460.
    53. Li, Yin & Arora, Sanjay & Youtie, Jan & Shapira, Philip, 2018. "Using web mining to explore Triple Helix influences on growth in small and mid-size firms," Technovation, Elsevier, vol. 76, pages 3-14.
    54. Coombs, R. & Narandren, P. & Richards, A., 1996. "A literature-based innovation output indicator," Research Policy, Elsevier, vol. 25(3), pages 403-413, May.
    55. Acciarini, Chiara & Cappa, Francesco & Boccardelli, Paolo & Oriani, Raffaele, 2023. "How can organizations leverage big data to innovate their business models? A systematic literature review," Technovation, Elsevier, vol. 123(C).
    56. Manfred Bruhn & Verena Schoenmueller & Daniela B. Schäfer, 2012. "Are social media replacing traditional media in terms of brand equity creation?," Management Research Review, Emerald Group Publishing Limited, vol. 35(9), pages 770-790, August.
    57. Shangqin Hong & Les Oxley & Philip McCann, 2012. "A Survey Of The Innovation Surveys," Journal of Economic Surveys, Wiley Blackwell, vol. 26(3), pages 420-444, July.
    58. Ulrich Schmoch & Stephan Gauch, 2009. "Service marks as indicators for innovation in knowledge-based services," Research Evaluation, Oxford University Press, vol. 18(4), pages 323-335, October.
    59. Hamilton, R.H. & Davison, H. Kristl, 2018. "The search for skills: Knowledge stars and innovation in the hiring process," Business Horizons, Elsevier, vol. 61(3), pages 409-419.
    60. Jan Kinne & David Lenz, 2021. "Predicting innovative firms using web mining and deep learning," PLOS ONE, Public Library of Science, vol. 16(4), pages 1-18, April.
    61. Krüger, Miriam & Kinne, Jan & Lenz, David & Resch, Bernd, 2020. "The digital layer: How innovative firms relate on the web," ZEW Discussion Papers 20-003, ZEW - Leibniz Centre for European Economic Research.
    62. Kinne, Jan & Krüger, Miriam & Lenz, David & Licht, Georg & Winker, Peter, 2020. "Coronavirus pandemic affects companies differently: A high-frequency website analysis of companies' reactions to the coronavirus pandemic in Germany," ZEW Expert Briefs 20-05e, ZEW - Leibniz Centre for European Economic Research.
    63. Castellacci, Fulvio & Natera, Jose Miguel, 2012. "Innovation surveys in Latin America: a primer," MPRA Paper 37769, University Library of Munich, Germany.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Axenbeck, Janna & Breithaupt, Patrick, 2022. "Measuring the digitalisation of firms: A novel text mining approach," ZEW Discussion Papers 22-065, ZEW - Leibniz Centre for European Economic Research.
    2. Zhao, Guoqing & Xie, Xiaotian & Wang, Yi & Liu, Shaofeng & Jones, Paul & Lopez, Carmen, 2024. "Barrier analysis to improve big data analytics capability of the maritime industry: A mixed-method approach," Technological Forecasting and Social Change, Elsevier, vol. 203(C).
    3. Schubert, Torben & Ashouri, Sajad & Deschryvere, Matthias & Jäger, Angela & Visentin, Fabiana & Cunningham, Scott & Hajikhani, Arash & Pukelis, Lukas & Suominen, Arho, 2023. "The role of product digitization for productivity," MERIT Working Papers 2023-004, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Janna Axenbeck & Patrick Breithaupt, 2021. "Innovation indicators based on firm websites—Which website characteristics predict firm-level innovation activity?," PLOS ONE, Public Library of Science, vol. 16(4), pages 1-23, April.
    2. Christian Rammer & Gastón P Fernández & Dirk Czarnitzki, 2021. "Artificial Intelligence and Industrial Innovation: Evidence from Firm-Level Data," Working Papers of Department of Economics, Leuven 674605, KU Leuven, Faculty of Economics and Business (FEB), Department of Economics, Leuven.
    3. Dziallas, Marisa & Blind, Knut, 2019. "Innovation indicators throughout the innovation process: An extensive literature analysis," Technovation, Elsevier, vol. 80, pages 3-29.
    4. Breithaupt, Patrick & Kesler, Reinhold & Niebel, Thomas & Rammer, Christian, 2020. "Intangible capital indicators based on web scraping of social media," ZEW Discussion Papers 20-046, ZEW - Leibniz Centre for European Economic Research.
    5. Mohnen, Pierre, 2019. "R&D, innovation and productivity," MERIT Working Papers 2019-016, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    6. Stephane Lhuillery & Julio Raffo & Intan Hamdan-Livramento, 2016. "Measuring creativity: Learning from innovation measurement," WIPO Economic Research Working Papers 31, World Intellectual Property Organization - Economics and Statistics Division.
    7. Janger, Jürgen & Schubert, Torben & Andries, Petra & Rammer, Christian & Hoskens, Machteld, 2017. "The EU 2020 innovation indicator: A step forward in measuring innovation outputs and outcomes?," Research Policy, Elsevier, vol. 46(1), pages 30-42.
    8. Matthias Siller & Christoph Hauser & Janette Walde & Gottfried Tappeiner, 2014. "The Multiple Facets of Regional Innovation," Working Papers 2014-19, Faculty of Economics and Statistics, Universität Innsbruck.
    9. Gatchev, Vladimir A. & Pirinsky, Christo A. & Venugopal, Buvaneshwaran, 2022. "A language-based approach to measuring creative exploration," Research Policy, Elsevier, vol. 51(1).
    10. Jan Kinne & Janna Axenbeck, 2020. "Web mining for innovation ecosystem mapping: a framework and a large-scale pilot study," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2011-2041, December.
    11. Teemu Makkonen & Robert P. Have, 2013. "Benchmarking regional innovative performance: composite measures and direct innovation counts," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(1), pages 247-262, January.
    12. Bei, Xiaoshu, 2019. "Trademarks, specialized complementary assets, and the external sourcing of innovation," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    13. Köhler, Christian & Sofka, Wolfgang & Grimpe, Christoph, 2009. "Selectivity in search strategies for innovation: from incremental to radical, from manufacturing to services," ZEW Discussion Papers 09-066, ZEW - Leibniz Centre for European Economic Research.
    14. Max Nathan & Anna Rosso, 2017. "Innovative events," Development Working Papers 429, Centro Studi Luca d'Agliano, University of Milano, revised 08 Apr 2019.
    15. Stephen Petrie & Mitchell Adams & Ben Mitra‐Kahn & Matthew Johnson & Russell Thomson & Paul Jensen & Alfons Palangkaraya & Elizabeth Webster, 2020. "TM‐Link: An Internationally Linked Trademark Database," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 53(2), pages 254-269, June.
    16. Matthias Siller & Christoph Hauser & Janette Walde & Gottfried Tappeiner, 2015. "Measuring regional innovation in one dimension: More lost than gained?," Working Papers 2015-14, Faculty of Economics and Statistics, Universität Innsbruck.
    17. Hud, Martin & Rammer, Christian, 2014. "FuE- und Innovationsausgaben während der Krise: Strategien zur Sicherung des Innovationserfolgs," ZEW Dokumentationen 14-03, ZEW - Leibniz Centre for European Economic Research.
    18. Abbasiharofteh, Milad & Kinne, Jan & Krüger, Miriam, 2021. "The strength of weak and strong ties in bridging geographic and cognitive distances," ZEW Discussion Papers 21-049, ZEW - Leibniz Centre for European Economic Research.
    19. Messer, Julia & Martin, Alexander, 2019. "Open Innovation in KMU: Eine empirische Analyse ausgewählter Faktoren," Flensburger Hefte zu Unternehmertum und Mittelstand 18, Jackstädt-Zentrum Flensburg.
    20. Cirera, Xavier & Muzi, Silvia, 2020. "Measuring innovation using firm-level surveys: Evidence from developing countries✰," Research Policy, Elsevier, vol. 49(3).

    More about this item

    Keywords

    Big data; Innovation indicators; CIS;
    All these keywords.

    JEL classification:

    • O30 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - General
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:tefoso:v:197:y:2023:i:c:s0040162523005590. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.sciencedirect.com/science/journal/00401625 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.