IDEAS home Printed from https://ideas.repec.org/a/eee/tefoso/v222y2026ics0040162525004068.html

Evaluating the value of LLMs in patent-based technology intelligence: Toward increasing efficiency and reducing expert dependency

Author

Listed:
  • Park, Sanghyun
  • Kim, Giyun
  • Lee, Sungjoo

Abstract

Patent analysis offers a useful tool for monitoring technological changes by transforming large volumes of patent data into valuable technological intelligence. However, current approaches are constrained by the large scale of patent datasets and the complexity of documents, requiring skilled analysts and a heavy reliance on domain experts to ensure accuracy. To support more agile technology planning, it is essential to reduce analysts' workloads and mitigate dependency on domain experts. This study investigates the applicability of Large Language Models (LLMs) in patent analysis by evaluating their performance in tasks traditionally handled by experts or analysts. Specifically, the accuracy of LLM-based patent analysis is assessed through a quantitative evaluation using F1 and ARI scores, and qualitative evaluation by domain experts. As a result, this study explores the possible use of LLMs to core patent analysis tasks including data collection, preprocessing, analysis and representation. The findings indicate that although LLMs have significant potential to enhance efficiency in the patent analysis process, further validation and continuous performance improvements are required, particularly in incorporating emerging trends and filtering out irrelevant content from patent search strategies.

Suggested Citation

  • Park, Sanghyun & Kim, Giyun & Lee, Sungjoo, 2026. "Evaluating the value of LLMs in patent-based technology intelligence: Toward increasing efficiency and reducing expert dependency," Technological Forecasting and Social Change, Elsevier, vol. 222(C).
  • Handle: RePEc:eee:tefoso:v:222:y:2026:i:c:s0040162525004068
    DOI: 10.1016/j.techfore.2025.124375
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0040162525004068
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.techfore.2025.124375?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Yoon, Janghyeok & Park, Hyunseok & Seo, Wonchul & Lee, Jae-Min & Coh, Byoung-youl & Kim, Jonghwa, 2015. "Technology opportunity discovery (TOD) from existing technologies and products: A function-based TOD framework," Technological Forecasting and Social Change, Elsevier, vol. 100(C), pages 153-167.
    2. Yoon, Byungun & Magee, Christopher L., 2018. "Exploring technology opportunities by visualizing patent information based on generative topographic mapping and link prediction," Technological Forecasting and Social Change, Elsevier, vol. 132(C), pages 105-117.
    3. Karvonen, Matti & Kässi, Tuomo, 2013. "Patent citations as a tool for analysing the early stages of convergence," Technological Forecasting and Social Change, Elsevier, vol. 80(6), pages 1094-1107.
    4. Karan Singhal & Shekoofeh Azizi & Tao Tu & S. Sara Mahdavi & Jason Wei & Hyung Won Chung & Nathan Scales & Ajay Tanwani & Heather Cole-Lewis & Stephen Pfohl & Perry Payne & Martin Seneviratne & Paul G, 2023. "Publisher Correction: Large language models encode clinical knowledge," Nature, Nature, vol. 620(7973), pages 19-19, August.
    5. Campbell, Richard S., 1983. "Patent trends as a technological forecasting tool," World Patent Information, Elsevier, vol. 5(3), pages 137-143.
    6. Janghyeok Yoon & Kwangsoo Kim, 2012. "Detecting signals of new technological opportunities using semantic patent analysis and outlier detection," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 445-461, February.
    7. Juite Wang, 2025. "Exploring technological landscape to uncover technological opportunities for immersive technologies in the Metaverse using patent data," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(7), pages 3505-3536, July.
    8. Ernst, Holger, 2003. "Patent information for strategic technology management," World Patent Information, Elsevier, vol. 25(3), pages 233-242, September.
    9. Roberts, Deborah L. & Candi, Marina, 2024. "Artificial intelligence and innovation management: Charting the evolving landscape," Technovation, Elsevier, vol. 136(C).
    10. Yun, Siyeong & Song, Kisik & Kim, Chulhyun & Lee, Sungjoo, 2021. "From stones to jewellery: Investigating technology opportunities from expired patents," Technovation, Elsevier, vol. 103(C).
    11. Li, Xin & Xie, Qianqian & Jiang, Jiaojiao & Zhou, Yuan & Huang, Lucheng, 2019. "Identifying and monitoring the development trends of emerging technologies using patent analysis and Twitter data mining: The case of perovskite solar cell technology," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 687-705.
    12. Seo, Wonchul & Afifuddin, Mokh, 2024. "Developing a supervised learning model for anticipating potential technology convergence between technology topics," Technological Forecasting and Social Change, Elsevier, vol. 203(C).
    13. Noh, Heeyong & Mortara, Letizia & Lee, Sungjoo, 2023. "Development of a maturity model for technology intelligence," Technological Forecasting and Social Change, Elsevier, vol. 192(C).
    14. Noh, Heeyong & Lee, Sungjoo, 2020. "What constitutes a promising technology in the era of open innovation? An investigation of patent potential from multiple perspectives," Technological Forecasting and Social Change, Elsevier, vol. 157(C).
    15. Patel, Pari & Pavitt, Keith, 1997. "The technological competencies of the world's largest firms: Complex and path-dependent, but not much variety," Research Policy, Elsevier, vol. 26(2), pages 141-156, May.
    16. Oleg Ena & Nadezhda Mikova & Ozcan Saritas & Anna Sokolova, 2016. "A methodology for technology trend monitoring: the case of semantic technologies," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(3), pages 1013-1041, September.
    17. William H. DeLone & Ephraim R. McLean, 1992. "Information Systems Success: The Quest for the Dependent Variable," Information Systems Research, INFORMS, vol. 3(1), pages 60-95, March.
    18. Youngjae Choi & Sanghyun Park & Sungjoo Lee, 2021. "Identifying emerging technologies to envision a future innovation ecosystem: A machine learning approach to patent data," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5431-5476, July.
    19. Ghaffari, Mohsen & Aliahmadi, Alireza & Khalkhali, Abolfazl & Zakery, Amir & Daim, Tugrul U. & Yalcin, Haydar, 2023. "Topic-based technology mapping using patent data analysis: A case study of vehicle tires," Technological Forecasting and Social Change, Elsevier, vol. 193(C).
    20. Lee, Sang M. & Lee, DonHee, 2021. "Opportunities and challenges for contactless healthcare services in the post-COVID-19 Era," Technological Forecasting and Social Change, Elsevier, vol. 167(C).
    21. Zhang, Hao & Daim, Tugrul & Zhang, Yunqiu (Peggy), 2021. "Integrating patent analysis into technology roadmapping: A latent dirichlet allocation based technology assessment and roadmapping in the field of Blockchain," Technological Forecasting and Social Change, Elsevier, vol. 167(C).
    22. Kim, Junhan & Geum, Youngjung, 2021. "How to develop data-driven technology roadmaps:The integration of topic modeling and link prediction," Technological Forecasting and Social Change, Elsevier, vol. 171(C).
    23. Liu, Zhenfeng & Feng, Jian & Uden, Lorna, 2023. "From technology opportunities to ideas generation via cross-cutting patent analysis: Application of generative topographic mapping and link prediction," Technological Forecasting and Social Change, Elsevier, vol. 192(C).
    24. Kim, Gabjo & Bae, Jinwoo, 2017. "A novel approach to forecast promising technology through patent analysis," Technological Forecasting and Social Change, Elsevier, vol. 117(C), pages 228-237.
    25. Li, Xin & Xie, Qianqian & Daim, Tugrul & Huang, Lucheng, 2019. "Forecasting technology trends using text mining of the gaps between science and technology: The case of perovskite solar cell technology," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 432-449.
    26. Henry Lopez-Vega & Jerker Moodysson, 2023. "Digital Transformation of the Automotive Industry: An Integrating Framework to Analyse Technological Novelty and Breadth," Industry and Innovation, Taylor & Francis Journals, vol. 30(1), pages 67-102, January.
    27. Reitzig, Markus, 2004. "Improving patent valuations for management purposes--validating new indicators by analyzing application rationales," Research Policy, Elsevier, vol. 33(6-7), pages 939-957, September.
    28. Narin, Francis & Noma, Elliot & Perry, Ross, 1987. "Patents as indicators of corporate technological strength," Research Policy, Elsevier, vol. 16(2-4), pages 143-155, August.
    29. Li, Munan & Wang, Liang, 2025. "Leveraging patent classification based on deep learning: The case study on smart cities and industrial Internet of Things," Journal of Informetrics, Elsevier, vol. 19(1).
    30. Hoisl, Karin & Stelzer, Tobias & Biala, Stefanie, 2015. "Forecasting technological discontinuities in the ICT industry," Research Policy, Elsevier, vol. 44(2), pages 522-532.
    31. Yoon, Byungun & Park, Inchae & Coh, Byoung-youl, 2014. "Exploring technological opportunities by linking technology and products: Application of morphology analysis and text mining," Technological Forecasting and Social Change, Elsevier, vol. 86(C), pages 287-303.
    32. Park, Yongtae & Yoon, Byungun & Lee, Sungjoo, 2005. "The idiosyncrasy and dynamism of technological innovation across industries: patent citation analysis," Technology in Society, Elsevier, vol. 27(4), pages 471-485.
    33. Karan Singhal & Shekoofeh Azizi & Tao Tu & S. Sara Mahdavi & Jason Wei & Hyung Won Chung & Nathan Scales & Ajay Tanwani & Heather Cole-Lewis & Stephen Pfohl & Perry Payne & Martin Seneviratne & Paul G, 2023. "Large language models encode clinical knowledge," Nature, Nature, vol. 620(7972), pages 172-180, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lee, Changyong, 2021. "A review of data analytics in technological forecasting," Technological Forecasting and Social Change, Elsevier, vol. 166(C).
    2. Kyuwoong Kim & Kyeongmin Park & Sungjoo Lee, 2019. "Investigating technology opportunities: the use of SAOx analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 45-70, January.
    3. Choi, Hyunhong & Woo, JongRoul, 2022. "Investigating emerging hydrogen technology topics and comparing national level technological focus: Patent analysis using a structural topic model," Applied Energy, Elsevier, vol. 313(C).
    4. Liu, Zhenfeng & Feng, Jian & Uden, Lorna, 2023. "Technology opportunity analysis using hierarchical semantic networks and dual link prediction," Technovation, Elsevier, vol. 128(C).
    5. Yuan Zhou & Fang Dong & Yufei Liu & Liang Ran, 2021. "A deep learning framework to early identify emerging technologies in large-scale outlier patents: an empirical study of CNC machine tool," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 969-994, February.
    6. Su, Yu-Shan & Huang, Hsini & Daim, Tugrul & Chien, Pan-Wei & Peng, Ru-Ling & Karaman Akgul, Arzu, 2023. "Assessing the technological trajectory of 5G-V2X autonomous driving inventions: Use of patent analysis," Technological Forecasting and Social Change, Elsevier, vol. 196(C).
    7. Juite Wang, 2025. "Exploring technological landscape to uncover technological opportunities for immersive technologies in the Metaverse using patent data," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(7), pages 3505-3536, July.
    8. Ren, Haiying & Zhao, Yuhui, 2021. "Technology opportunity discovery based on constructing, evaluating, and searching knowledge networks," Technovation, Elsevier, vol. 101(C).
    9. Park, Mingyu & Geum, Youngjung, 2022. "Two-stage technology opportunity discovery for firm-level decision making: GCN-based link-prediction approach," Technological Forecasting and Social Change, Elsevier, vol. 183(C).
    10. Park, Jongyong & Lee, Hakyeon & Park, Yongtae, 2009. "Disembodied knowledge flows among industrial clusters: A patent analysis of the Korean manufacturing sector," Technology in Society, Elsevier, vol. 31(1), pages 73-84.
    11. Ghaffari, Mohsen & Aliahmadi, Alireza & Khalkhali, Abolfazl & Zakery, Amir & Daim, Tugrul U. & Yalcin, Haydar, 2023. "Topic-based technology mapping using patent data analysis: A case study of vehicle tires," Technological Forecasting and Social Change, Elsevier, vol. 193(C).
    12. Wang, Xiaoli & Daim, Tugrul & Huang, Lucheng & Li, Zhiqiang & Shaikh, Ruqia & Kassi, Diby Francois, 2022. "Monitoring the development trend and competition status of high technologies using patent analysis and bibliographic coupling: The case of electronic design automation technology," Technology in Society, Elsevier, vol. 71(C).
    13. Gozuacik, Necip & Sakar, C. Okan & Ozcan, Sercan, 2023. "Technological forecasting based on estimation of word embedding matrix using LSTM networks," Technological Forecasting and Social Change, Elsevier, vol. 191(C).
    14. Ghaffari, Mohsen & Aliahmadi, Alireza & Khalkhali, Abolfazl & Zakery, Amir & Daim, Tugrul U. & Zamani, Mehdi, 2024. "Exploring the technological leaders using tire industry patents: A topic modeling approach," Technology in Society, Elsevier, vol. 78(C).
    15. Erzurumlu, S. Sinan & Pachamanova, Dessislava, 2020. "Topic modeling and technology forecasting for assessing the commercial viability of healthcare innovations," Technological Forecasting and Social Change, Elsevier, vol. 156(C).
    16. Youngjae Choi & Sanghyun Park & Sungjoo Lee, 2021. "Identifying emerging technologies to envision a future innovation ecosystem: A machine learning approach to patent data," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5431-5476, July.
    17. Seol, Youngjin & Lee, Seunghyun & Kim, Cheolhan & Yoon, Janghyeok & Choi, Jaewoong, 2023. "Towards firm-specific technology opportunities: A rule-based machine learning approach to technology portfolio analysis," Journal of Informetrics, Elsevier, vol. 17(4).
    18. Seo, Wonchul & Afifuddin, Mokh, 2024. "Developing a supervised learning model for anticipating potential technology convergence between technology topics," Technological Forecasting and Social Change, Elsevier, vol. 203(C).
    19. Lijie Feng & Kehui Liu & Jinfeng Wang & Kuo-Yi Lin & Ke Zhang & Luyao Zhang, 2022. "Identifying Promising Technologies of Electric Vehicles from the Perspective of Market and Technical Attributes," Energies, MDPI, vol. 15(20), pages 1-22, October.
    20. Nicola Amoroso & Annamaria Demarinis Loiotile & Ester Pantaleo & Giuseppe Conti & Shiva Loccisano & Sabina Tangaro & Alfonso Monaco & Roberto Bellotti, 2025. "An Italian Patent Multi-Label Classification System to Support the Innovation Demand and Supply Matching," Sustainability, MDPI, vol. 17(14), pages 1-27, July.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:tefoso:v:222:y:2026:i:c:s0040162525004068. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.sciencedirect.com/science/journal/00401625 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.