IDEAS home Printed from https://ideas.repec.org/a/eee/tefoso/v162y2021ics0040162520311926.html
   My bibliography  Save this article

A topic models based framework for detecting and forecasting emerging technologies

Author

Listed:
  • Xu, Shuo
  • Hao, Liyuan
  • Yang, Guancan
  • Lu, Kun
  • An, Xin

Abstract

The identification of emerging technologies can bring valuable intelligence to enterprises and countries determining research and development (R&D) priorities. Emerging technologies are closely related to emerging topics in terms of several well-documented attributes: relatively fast growth, radical novelty and prominent impact. Our previous work on detecting and forecasting emerging topics is adapted to measure technology emergence, but the dynamic influence model (DIM) is replaced by the topical n-grams (TNG) model in this framework to nominate several emerging technologies in technical terms and to exploit the potential of topic models. Hence, technologies are viewed as term-based themes in this study. Three indicators are designed to reflect the above attributes: the fast growth indicator, the radical novelty indicator and the prominent impact indicator. The relatively fast growth indicator is calculated from the results of the TNG model and the radical novelty indicator comes from the citation influence model (CIM). As for the prominent impact indicator, the involving authors are used after name disambiguation and credit allocation. The following fields are utilized to develop the models: title, abstract, keywords-author, publication year, byline information, and cited references. We participated in the 2018–2019 Measuring Tech Emergence Contest with the proposed method, and 8 out of 10 submitted ones met the contest organizer’s criteria of technology emergence. Criteria included the percentage of high growth terms out of total terms provided, the degree of growth of the terms, and the frequency of those high growth terms across the dataset. Then, a qualitative assessment of overall methodology was conducted by three judges. In the end, we won Second Prize in the contest.

Suggested Citation

  • Xu, Shuo & Hao, Liyuan & Yang, Guancan & Lu, Kun & An, Xin, 2021. "A topic models based framework for detecting and forecasting emerging technologies," Technological Forecasting and Social Change, Elsevier, vol. 162(C).
  • Handle: RePEc:eee:tefoso:v:162:y:2021:i:c:s0040162520311926
    DOI: 10.1016/j.techfore.2020.120366
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0040162520311926
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.techfore.2020.120366?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kyebambe, Moses Ntanda & Cheng, Ge & Huang, Yunqing & He, Chunhui & Zhang, Zhenyu, 2017. "Forecasting emerging technologies: A supervised learning approach through patent analysis," Technological Forecasting and Social Change, Elsevier, vol. 125(C), pages 236-244.
    2. Zhou, Xiao & Huang, Lu & Porter, Alan & Vicente-Gomila, Jose M., 2019. "Tracing the system transformations and innovation pathways of an emerging technology: Solid lipid nanoparticles," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 785-794.
    3. Christian Weismayer & Ilona Pezenka, 2017. "Identifying emerging research fields: a longitudinal latent semantic keyword analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1757-1785, December.
    4. Hanning Guo & Scott Weingart & Katy Börner, 2011. "Mixed-indicators model for identifying emerging research areas," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(1), pages 421-435, October.
    5. Nils T Hagen, 2008. "Harmonic Allocation of Authorship Credit: Source-Level Correction of Bibliometric Bias Assures Accurate Publication and Citation Analysis," PLOS ONE, Public Library of Science, vol. 3(12), pages 1-7, December.
    6. Ludo Waltman & Nees Jan van Eck, 2012. "A new methodology for constructing a publication‐level classification system of science," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(12), pages 2378-2392, December.
    7. Bolger, Fergus & Wright, George, 2017. "Use of expert knowledge to anticipate the future: Issues, analysis and directions," International Journal of Forecasting, Elsevier, vol. 33(1), pages 230-243.
    8. Rotolo, Daniele & Hicks, Diana & Martin, Ben R., 2015. "What is an emerging technology?," Research Policy, Elsevier, vol. 44(10), pages 1827-1843.
    9. Ma, Ruimin, 2012. "Author bibliographic coupling analysis: A test based on a Chinese academic database," Journal of Informetrics, Elsevier, vol. 6(4), pages 532-542.
    10. Chaomei Chen, 2006. "CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(3), pages 359-377, February.
    11. Chyi-Kwei Yau & Alan Porter & Nils Newman & Arho Suominen, 2014. "Clustering scientific documents with topic modeling," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(3), pages 767-786, September.
    12. Ke, Qing, 2020. "Technological impact of biomedical research: The role of basicness and novelty," Research Policy, Elsevier, vol. 49(7).
    13. Henry Small, 1973. "Co‐citation in the scientific literature: A new measure of the relationship between two documents," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 24(4), pages 265-269, July.
    14. Joung, Junegak & Kim, Kwangsoo, 2017. "Monitoring emerging technologies for technology planning using technical keyword based analysis from patent data," Technological Forecasting and Social Change, Elsevier, vol. 114(C), pages 281-292.
    15. Xu, Shuo & Hao, Liyuan & An, Xin & Yang, Guancan & Wang, Feifei, 2019. "Emerging research topics detection with multiple machine learning models," Journal of Informetrics, Elsevier, vol. 13(4).
    16. Zhang, Yi & Robinson, Douglas K.R. & Porter, Alan L. & Zhu, Donghua & Zhang, Guangquan & Lu, Jie, 2016. "Technology roadmapping for competitive technical intelligence," Technological Forecasting and Social Change, Elsevier, vol. 110(C), pages 175-186.
    17. Kevin W. Boyack & Richard Klavans, 2010. "Co-citation analysis, bibliographic coupling, and direct citation: Which citation approach represents the research front most accurately?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(12), pages 2389-2404, December.
    18. Sommarberg, Matti & Mäkinen, Saku J., 2019. "A method for anticipating the disruptive nature of digitalization in the machine-building industry," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 808-819.
    19. Dangzhi Zhao & Andreas Strotmann, 2008. "Evolution of research activities and intellectual influences in information science 1996–2005: Introducing author bibliographic‐coupling analysis," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(13), pages 2070-2086, November.
    20. Apreda, Riccardo & Bonaccorsi, Andrea & dell'Orletta, Felice & Fantoni, Gualtiero, 2019. "Expert forecast and realized outcomes in technology foresight," Technological Forecasting and Social Change, Elsevier, vol. 141(C), pages 277-288.
    21. Ludo Waltman & Nees Jan Eck, 2012. "A new methodology for constructing a publication-level classification system of science," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2378-2392, December.
    22. Liang Chen & Shuo Xu & Lijun Zhu & Jing Zhang & Xiaoping Lei & Guancan Yang, 2020. "A deep learning based method for extracting semantic information from patent documents," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 289-312, October.
    23. Jarneving, Bo, 2007. "Bibliographic coupling and its application to research-front and other core documents," Journal of Informetrics, Elsevier, vol. 1(4), pages 287-307.
    24. Sanjay K. Arora & Jan Youtie & Philip Shapira & Lidan Gao & TingTing Ma, 2013. "Entry strategies in an emerging technology: a pilot web-based study of graphene firms," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 1189-1207, June.
    25. Porter, Alan L. & Garner, Jon & Carley, Stephen F. & Newman, Nils C., 2019. "Emergence scoring to identify frontier R&D topics and key players," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 628-643.
    26. Qi Wang, 2018. "A bibliometric model for identifying emerging research topics," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 69(2), pages 290-304, February.
    27. Jian Xu & Ying Ding & Min Song & Tamy Chambers, 2016. "Author credit-assignment schemas: A comparison and analysis," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(8), pages 1973-1989, August.
    28. Kong, Dejing & Zhou, Yuan & Liu, Yufei & Xue, Lan, 2017. "Using the data mining method to assess the innovation gap: A case of industrial robotics in a catching-up country," Technological Forecasting and Social Change, Elsevier, vol. 119(C), pages 80-97.
    29. Arho Suominen & Hannes Toivanen, 2016. "Map of science with topic modeling: Comparison of unsupervised learning and human-assigned subject classification," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(10), pages 2464-2476, October.
    30. Mu-Hsuan Huang & Chia-Pin Chang, 2014. "Detecting research fronts in OLED field using bibliographic coupling with sliding window," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(3), pages 1721-1744, March.
    31. Shuo Xu & Junwan Liu & Dongsheng Zhai & Xin An & Zheng Wang & Hongshen Pang, 2018. "Overlapping thematic structures extraction with mixed-membership stochastic blockmodel," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 61-84, October.
    32. Lee, Changyong & Kwon, Ohjin & Kim, Myeongjung & Kwon, Daeil, 2018. "Early identification of emerging technologies: A machine learning approach using multiple patent indicators," Technological Forecasting and Social Change, Elsevier, vol. 127(C), pages 291-303.
    33. Shuo Xu & Dongsheng Zhai & Feifei Wang & Xin An & Hongshen Pang & Yirong Sun, 2019. "A novel method for topic linkages between scientific publications and patents," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 70(9), pages 1026-1042, September.
    34. Shuo Xu & Liyuan Hao & Xin An & Hongshen Pang & Ting Li, 2020. "Review on emerging research topics with key-route main path analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 607-624, January.
    35. Ismael Rafols & Alan L. Porter & Loet Leydesdorff, 2010. "Science overlay maps: A new tool for research policy and library management," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(9), pages 1871-1887, September.
    36. Small, Henry & Boyack, Kevin W. & Klavans, Richard, 2014. "Identifying emerging topics in science and technology," Research Policy, Elsevier, vol. 43(8), pages 1450-1467.
    37. Kevin W. Boyack & Richard Klavans, 2010. "Co‐citation analysis, bibliographic coupling, and direct citation: Which citation approach represents the research front most accurately?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(12), pages 2389-2404, December.
    38. Veugelers, Reinhilde & Wang, Jian, 2019. "Scientific novelty and technological impact," Research Policy, Elsevier, vol. 48(6), pages 1362-1372.
    39. Serhat Burmaoglu & Olivier Sartenaer & Alan Porter & Munan Li, 2019. "Analysing the theoretical roots of technology emergence: an evolutionary perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 97-118, April.
    40. Hagen, Nils T., 2013. "Harmonic coauthor credit: A parsimonious quantification of the byline hierarchy," Journal of Informetrics, Elsevier, vol. 7(4), pages 784-791.
    41. Jinseok Kim, 2018. "Evaluating author name disambiguation for digital libraries: a case of DBLP," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 1867-1886, September.
    42. Marianne Gauffriau & Peder Olesen Larsen, 2005. "Counting methods are decisive for rankings based on publication and citation studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(1), pages 85-93, July.
    43. Zhang, Yi & Huang, Ying & Porter, Alan L. & Zhang, Guangquan & Lu, Jie, 2019. "Discovering and forecasting interactions in big data research: A learning-enhanced bibliometric study," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 795-807.
    44. Hongqi Han & Changqing Yao & Yuan Fu & Yongsheng Yu & Yunliang Zhang & Shuo Xu, 2017. "Semantic fingerprints-based author name disambiguation in Chinese documents," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1879-1896, June.
    45. Teja Tscharntke & Michael E Hochberg & Tatyana A Rand & Vincent H Resh & Jochen Krauss, 2007. "Author Sequence and Credit for Contributions in Multiauthored Publications," PLOS Biology, Public Library of Science, vol. 5(1), pages 1-2, January.
    46. Blaise Cronin, 2001. "Hyperauthorship: A postmodern perversion or evidence of a structural shift in scholarly communication practices?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 52(7), pages 558-569.
    47. Robinson, Douglas K.R. & Lagnau, Axel & Boon, Wouter P.C., 2019. "Innovation pathways in additive manufacturing: Methods for tracing emerging and branching paths from rapid prototyping to alternative applications," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 733-750.
    48. Michael Levin & Stefan Krawczyk & Steven Bethard & Dan Jurafsky, 2012. "Citation-based bootstrapping for large-scale author disambiguation," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(5), pages 1030-1047, May.
    49. Michael Levin & Stefan Krawczyk & Steven Bethard & Dan Jurafsky, 2012. "Citation‐based bootstrapping for large‐scale author disambiguation," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(5), pages 1030-1047, May.
    50. Yoshiyuki Takeda & Yuya Kajikawa, 2009. "Optics: a bibliometric approach to detect emerging research domains and intellectual bases," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(3), pages 543-558, March.
    51. Shuo Xu & Liyuan Hao & Xin An & Dongsheng Zhai & Hongshen Pang, 2019. "Types of DOI errors of cited references in Web of Science with a cleaning method," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(3), pages 1427-1437, September.
    52. Gregorio González-Alcaide & Pedro Llorente & José M. Ramos, 2016. "Bibliometric indicators to identify emerging research fields: publications on mass gatherings," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1283-1298, November.
    53. Stephen F. Carley & Nils C. Newman & Alan L. Porter & Jon G. Garner, 2018. "An indicator of technical emergence," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 35-49, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhang, Hao & Daim, Tugrul & Zhang, Yunqiu (Peggy), 2021. "Integrating patent analysis into technology roadmapping: A latent dirichlet allocation based technology assessment and roadmapping in the field of Blockchain," Technological Forecasting and Social Change, Elsevier, vol. 167(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xu, Haiyun & Winnink, Jos & Yue, Zenghui & Zhang, Huiling & Pang, Hongshen, 2021. "Multidimensional Scientometric indicators for the detection of emerging research topics," Technological Forecasting and Social Change, Elsevier, vol. 163(C).
    2. Shuo Xu & Liyuan Hao & Xin An & Hongshen Pang & Ting Li, 2020. "Review on emerging research topics with key-route main path analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 607-624, January.
    3. Xu, Shuo & Hao, Liyuan & An, Xin & Yang, Guancan & Wang, Feifei, 2019. "Emerging research topics detection with multiple machine learning models," Journal of Informetrics, Elsevier, vol. 13(4).
    4. Porter, Alan L. & Chiavetta, Denise & Newman, Nils C., 2020. "Measuring tech emergence: A contest," Technological Forecasting and Social Change, Elsevier, vol. 159(C).
    5. Kwon, Seokbeom & Liu, Xiaoyu & Porter, Alan L. & Youtie, Jan, 2019. "Research addressing emerging technological ideas has greater scientific impact," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    6. Suominen, Arho & Peng, Haoshu & Ranaei, Samira, 2019. "Examining the dynamics of an emerging research network using the case of triboelectric nanogenerators," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 820-830.
    7. Liu, Yunmei & Yang, Liu & Chen, Min, 2021. "A new citation concept: Triangular citation in the literature," Journal of Informetrics, Elsevier, vol. 15(2).
    8. Yun, Jinhyuk & Ahn, Sejung & Lee, June Young, 2020. "Return to basics: Clustering of scientific literature using structural information," Journal of Informetrics, Elsevier, vol. 14(4).
    9. Porter, Alan L. & Garner, Jon & Carley, Stephen F. & Newman, Nils C., 2019. "Emergence scoring to identify frontier R&D topics and key players," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 628-643.
    10. Sohrabi, Babak & Khalilijafarabad, Ahmad, 2018. "Systematic method for finding emergence research areas as data quality," Technological Forecasting and Social Change, Elsevier, vol. 137(C), pages 280-287.
    11. Wang, Feifei & Jia, Chenran & Wang, Xiaohan & Liu, Junwan & Xu, Shuo & Liu, Yang & Yang, Chenyuyan, 2019. "Exploring all-author tripartite citation networks: A case study of gene editing," Journal of Informetrics, Elsevier, vol. 13(3), pages 856-873.
    12. Yuan Zhou & Heng Lin & Yufei Liu & Wei Ding, 2019. "A novel method to identify emerging technologies using a semi-supervised topic clustering model: a case of 3D printing industry," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(1), pages 167-185, July.
    13. Ying Huang & Wolfgang Glänzel & Lin Zhang, 2021. "Tracing the development of mapping knowledge domains," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6201-6224, July.
    14. Serhat Burmaoglu & Olivier Sartenaer & Alan Porter & Munan Li, 2019. "Analysing the theoretical roots of technology emergence: an evolutionary perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 97-118, April.
    15. Takano, Yasutomo & Kajikawa, Yuya, 2019. "Extracting commercialization opportunities of the Internet of Things: Measuring text similarity between papers and patents," Technological Forecasting and Social Change, Elsevier, vol. 138(C), pages 45-68.
    16. R. Fileto Maciel & P. Saskia Bayerl & Marta Macedo Kerr Pinheiro, 2019. "Technical research innovations of the US national security system," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 539-565, August.
    17. Ugo Moschini & Elena Fenialdi & Cinzia Daraio & Giancarlo Ruocco & Elisa Molinari, 0. "A comparison of three multidisciplinarity indices based on the diversity of Scopus subject areas of authors’ documents, their bibliography and their citing papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 0, pages 1-14.
    18. Li, Munan & Porter, Alan L. & Suominen, Arho & Burmaoglu, Serhat & Carley, Stephen, 2021. "An exploratory perspective to measure the emergence degree for a specific technology based on the philosophy of swarm intelligence," Technological Forecasting and Social Change, Elsevier, vol. 166(C).
    19. Sitaram Devarakonda & Dmitriy Korobskiy & Tandy Warnow & George Chacko, 2020. "Viewing computer science through citation analysis: Salton and Bergmark Redux," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 271-287, October.
    20. Huang, Lu & Chen, Xiang & Ni, Xingxing & Liu, Jiarun & Cao, Xiaoli & Wang, Changtian, 2021. "Tracking the dynamics of co-word networks for emerging topic identification," Technological Forecasting and Social Change, Elsevier, vol. 170(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:tefoso:v:162:y:2021:i:c:s0040162520311926. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.sciencedirect.com/science/journal/00401625 .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.sciencedirect.com/science/journal/00401625 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.