IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v128y2023i9d10.1007_s11192-023-04736-z.html
   My bibliography  Save this article

Prediction of patent grant and interpreting the key determinants: an application of interpretable machine learning approach

Author

Listed:
  • Li Yao

    (Zhejiang Gongshang University)

  • He Ni

    (Zhejiang Gongshang University)

Abstract

Patents are valuable intellectual property only when granted by the governments, and failing to receive an official grant means disclosing valuable technologies and information, which otherwise would be kept as commercial secrets. Yet, a typical patent application process takes years to complete and the outcome is uncertain. This study implements machine learning models to predict patent examination outcomes based on early information disclosed at patent publication and interpret the mechanism of how these models make predictions, highlighting the key determinants to patent grant and delineating the relationships between the patent features and the examination outcome. The predictive models that integrate patent-level variables with textual information accomplish the best prediction performances with a 0.854 ROC-AUC score and 77% accuracy rate. A number of interpretable machine learning methods are applied. The permutation-based feature importance metric identifies key determinants such as applicants’ prior experience, page length, backward citation, claim counts, number of patent family, etc. SHAP (SHapley Additive exPlanations), a local interpretability method, describes the marginal contributions to the model prediction of key predictors using two actual patent examples. Our study provides several valuable findings with important theoretical insights and practical applications. Specifically, we show that patent-level information can serve as a predictor of examination outcomes and the relationships between the predictors and outcome variables are complex. Knowledge accumulation and technology complexity positively affect the likelihood of patent grants, albeit with a curvilinear relationship. At lower levels, both factors significantly increase the chance of a grant, but beyond a certain threshold, the marginal effect becomes less pronounced. Additionally, prior experience, patent family size, and engagement with the patent agency have a monotonic and positive relationship with the grant likelihood, whereas the impact of patent scope on patent grants remains uncertain. While a narrower and more specific patent claim is associated with a higher grant rate, the number of claims increases it. Moreover, technology range, inventor team size, and examination duration have little effect on the patent grant results. From a practical standpoint, the accurate prediction of patent grants has significant potential applications. For instance, it could help firms better prioritize resources on the patent applications of high grant potentials to secure the final grant, as failure means a waste of R &D effort and disclosure of technology without IP protection. Additionally, patent examiners could utilize our predictive results as prior knowledge to enhance their judgment and accelerate the examination process.

Suggested Citation

  • Li Yao & He Ni, 2023. "Prediction of patent grant and interpreting the key determinants: an application of interpretable machine learning approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(9), pages 4933-4969, September.
  • Handle: RePEc:spr:scient:v:128:y:2023:i:9:d:10.1007_s11192-023-04736-z
    DOI: 10.1007/s11192-023-04736-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-023-04736-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-023-04736-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Higham, Kyle & de Rassenfosse, Gaétan & Jaffe, Adam B., 2021. "Patent Quality: Towards a Systematic Framework for Analysis and Measurement," Research Policy, Elsevier, vol. 50(4).
    2. Gaétan de Rassenfosse & Emilio Raiteri, 2022. "Technology Protectionism and the Patent System: Evidence from China," Journal of Industrial Economics, Wiley Blackwell, vol. 70(1), pages 1-43, March.
    3. van Zeebroeck, Nicolas & van Pottelsberghe de la Potterie, Bruno & Guellec, Dominique, 2009. "Claiming more: the Increased Voluminosity of Patent Applications and its Determinants," Research Policy, Elsevier, vol. 38(6), pages 1006-1020, July.
    4. Uijun Kwon & Youngjung Geum, 2020. "Identification of promising inventions considering the quality of knowledge accumulation: a machine learning approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 1877-1897, December.
    5. Katchanov, Yurij L. & Markova, Yulia V. & Shmatko, Natalia A., 2019. "The distinction machine: Physics journals from the perspective of the Kolmogorov–Smirnov statistic," Journal of Informetrics, Elsevier, vol. 13(4).
    6. Ying Xie & David Giles, 2011. "A survival analysis of the approval of US patent applications," Applied Economics, Taylor & Francis Journals, vol. 43(11), pages 1375-1384.
    7. Zhang, Yaojie & Ma, Feng & Wang, Yudong, 2019. "Forecasting crude oil prices with a large set of predictors: Can LASSO select powerful predictors?," Journal of Empirical Finance, Elsevier, vol. 54(C), pages 97-117.
    8. Xue Wang & Xuemei Yang & Jian Du & Xuwen Wang & Jiao Li & Xiaoli Tang, 2021. "A deep learning approach for identifying biomedical breakthrough discoveries using context analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5531-5549, July.
    9. Useche, Diego, 2014. "Are patents signals for the IPO market? An EU–US comparison for the software industry," Research Policy, Elsevier, vol. 43(8), pages 1299-1311.
    10. Michael D. Frakes & Melissa F. Wasserman, 2017. "Is the Time Allocated to Review Patent Applications Inducing Examiners to Grant Invalid Patents? Evidence from Microlevel Application Data," The Review of Economics and Statistics, MIT Press, vol. 99(3), pages 550-563, July.
    11. Bronwyn H. Hall & Adam Jaffe & Manuel Trajtenberg, 2005. "Market Value and Patent Citations," RAND Journal of Economics, The RAND Corporation, vol. 36(1), pages 16-38, Spring.
    12. Adam B. Jaffe & Manuel Trajtenberg & Rebecca Henderson, 1993. "Geographic Localization of Knowledge Spillovers as Evidenced by Patent Citations," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 108(3), pages 577-598.
    13. Yuan Zhou & Fang Dong & Yufei Liu & Zhaofu Li & JunFei Du & Li Zhang, 2020. "Forecasting emerging technologies using data augmentation and deep learning," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 1-29, April.
    14. Bekkers, Rudi & Martinelli, Arianna & Tamagni, Federico, 2020. "The impact of including standards-related documentation in patent prior art: Evidence from an EPO policy change," Research Policy, Elsevier, vol. 49(7).
    15. Frakes, Michael D. & Wasserman, Melissa F., 2021. "Knowledge spillovers, peer effects, and telecommuting: Evidence from the U.S. Patent Office," Journal of Public Economics, Elsevier, vol. 198(C).
    16. Liegsalz, Johannes & Wagner, Stefan, 2013. "Patent examination at the State Intellectual Property Office in China," Research Policy, Elsevier, vol. 42(2), pages 552-563.
    17. Tong, Tony W. & Zhang, Kun & He, Zi-Lin & Zhang, Yuchen, 2018. "What determines the duration of patent examination in China? An outcome-specific duration analysis of invention patent applications at SIPO," Research Policy, Elsevier, vol. 47(3), pages 583-591.
    18. Joon Hyung Cho & Jungpyo Lee & So Young Sohn, 2021. "Predicting future technological convergence patterns based on machine learning using link prediction," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5413-5429, July.
    19. Ghoddusi, Hamed & Creamer, Germán G. & Rafizadeh, Nima, 2019. "Machine learning in energy economics and finance: A review," Energy Economics, Elsevier, vol. 81(C), pages 709-727.
    20. Marco, Alan C. & Sarnoff, Joshua D. & deGrazia, Charles A.W., 2019. "Patent claims and patent scope," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    21. Hur, Wonchang & Oh, Junbyoung, 2021. "A man is known by the company he keeps?: A structural relationship between backward citation and forward citation of patents," Research Policy, Elsevier, vol. 50(1).
    22. Chung, Park & Sohn, So Young, 2020. "Early detection of valuable patents using a deep learning model: Case of semiconductor industry," Technological Forecasting and Social Change, Elsevier, vol. 158(C).
    23. Youngjae Choi & Sanghyun Park & Sungjoo Lee, 2021. "Identifying emerging technologies to envision a future innovation ecosystem: A machine learning approach to patent data," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5431-5476, July.
    24. Mark A. Lemley & Bhaven Sampat, 2012. "Examiner Characteristics and Patent Office Outcomes," The Review of Economics and Statistics, MIT Press, vol. 94(3), pages 817-827, August.
    25. Kim, Juram & Lee, Gyumin & Lee, Seungbin & Lee, Changyong, 2022. "Towards expert–machine collaborations for technology valuation: An interpretable machine learning approach," Technological Forecasting and Social Change, Elsevier, vol. 183(C).
    26. Drivas, Kyriakos & Kaplanis, Ioannis, 2020. "The role of international collaborations in securing the patent grant," Journal of Informetrics, Elsevier, vol. 14(4).
    27. Guellec, Dominique & Pottelsberghe de la Potterie, Bruno v., 2000. "Applications, grants and the value of patent," Economics Letters, Elsevier, vol. 69(1), pages 109-114, October.
    28. de Rassenfosse, Gaétan & Palangkaraya, Alfons & Webster, Elizabeth, 2016. "Why do patents facilitate trade in technology? Testing the disclosure and appropriation effects," Research Policy, Elsevier, vol. 45(7), pages 1326-1336.
    29. Joon Mo Ahn & Letizia Mortara & Tim Minshall, 2018. "Dynamic capabilities and economic crises: has openness enhanced a firm's performance in an economic downturn?," Industrial and Corporate Change, Oxford University Press and the Associazione ICC, vol. 27(1), pages 49-63.
    30. Joshua Lerner, 1994. "The Importance of Patent Scope: An Empirical Analysis," RAND Journal of Economics, The RAND Corporation, vol. 25(2), pages 319-333, Summer.
    31. Climent, Francisco & Momparler, Alexandre & Carmona, Pedro, 2019. "Anticipating bank distress in the Eurozone: An Extreme Gradient Boosting approach," Journal of Business Research, Elsevier, vol. 101(C), pages 885-896.
    32. Bronwyn H. Hall & Dietmar Harhoff, 2012. "Recent Research on the Economics of Patents," Annual Review of Economics, Annual Reviews, vol. 4(1), pages 541-565, July.
    33. Sendhil Mullainathan & Jann Spiess, 2017. "Machine Learning: An Applied Econometric Approach," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 87-106, Spring.
    34. Dejing Kong & Jianzhong Yang & Lingfeng Li, 2020. "Early identification of technological convergence in numerical control machine tool: a deep learning approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 1983-2009, December.
    35. Kim, Yee Kyoung & Oh, Jun Byoung, 2017. "Examination workloads, grant decision bias and examination quality of patent office," Research Policy, Elsevier, vol. 46(5), pages 1005-1019.
    36. Mann, Ronald J. & Sager, Thomas W., 2007. "Patents, venture capital, and software start-ups," Research Policy, Elsevier, vol. 36(2), pages 193-208, March.
    37. Joshua S. Gans & David H. Hsu & Scott Stern, 2008. "The Impact of Uncertain Intellectual Property Rights on the Market for Ideas: Evidence from Patent Grant Delays," Management Science, INFORMS, vol. 54(5), pages 982-997, May.
    38. Schoenmakers, Wilfred & Duysters, Geert, 2010. "The technological origins of radical inventions," Research Policy, Elsevier, vol. 39(8), pages 1051-1059, October.
    39. Bhaven Sampat & Heidi L. Williams, 2019. "How Do Patents Affect Follow-On Innovation? Evidence from the Human Genome," American Economic Review, American Economic Association, vol. 109(1), pages 203-236, January.
    40. Krzysztof Klincewicz & Szymon Szumiał, 2022. "Successful patenting—not only how, but with whom: the importance of patent attorneys," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(9), pages 5111-5137, September.
    41. Dietmar Harhoff & Stefan Wagner, 2009. "The Duration of Patent Examination at the European Patent Office," Management Science, INFORMS, vol. 55(12), pages 1969-1984, December.
    42. Jeffrey M. Kuhn & Neil C. Thompson, 2019. "How to Measure and Draw Causal Inferences with Patent Scope," International Journal of the Economics of Business, Taylor & Francis Journals, vol. 26(1), pages 5-38, January.
    43. Harhoff, Dietmar & Scherer, Frederic M. & Vopel, Katrin, 2003. "Citations, family size, opposition and the value of patent rights," Research Policy, Elsevier, vol. 32(8), pages 1343-1363, September.
    44. Yang, Deli, 2008. "Pendency and grant ratios of invention patents: A comparative study of the US and China," Research Policy, Elsevier, vol. 37(6-7), pages 1035-1046, July.
    45. Gaétan de Rassenfosse & Reza Hosseini, 2020. "Discrimination against foreigners in the U.S. patent system," Journal of International Business Policy, Palgrave Macmillan, vol. 3(4), pages 349-366, December.
    46. Prithwiraj Choudhury & Martine R. Haas, 2018. "Scope versus speed: Team diversity, leader experience, and patenting outcomes for firms," Strategic Management Journal, Wiley Blackwell, vol. 39(4), pages 977-1002, April.
    47. Sun, Zhen & Wright, Brian D., 2022. "Citations backward and forward: Insights into the patent examiner's role," Research Policy, Elsevier, vol. 51(7).
    48. Elizabeth Webster & Paul H. Jensen & Alfons Palangkaraya, 2014. "Patent examination outcomes and the national treatment principle," RAND Journal of Economics, RAND Corporation, vol. 45(2), pages 449-469, June.
    49. Carmona, Pedro & Climent, Francisco & Momparler, Alexandre, 2019. "Predicting failure in the U.S. banking sector: An extreme gradient boosting approach," International Review of Economics & Finance, Elsevier, vol. 61(C), pages 304-323.
    50. Zhu, Kejia & Malhotra, Shavin & Li, Yaohan, 2022. "Technological diversity of patent applications and decision pendency," Research Policy, Elsevier, vol. 51(1).
    51. Zhao, Long, 2022. "On the grant rate of Patent Cooperation Treaty applications: Theory and evidence," Economic Modelling, Elsevier, vol. 117(C).
    52. Qingyuan Zhao & Trevor Hastie, 2021. "Causal Interpretations of Black-Box Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(1), pages 272-281, January.
    53. Novelli, Elena, 2015. "An examination of the antecedents and implications of patent scope," Research Policy, Elsevier, vol. 44(2), pages 493-507.
    54. Junbyoung Oh & Yee Kyoung Kim, 2017. "Examination workloads, grant decision bias and examination quality of patent office," Inha University IBER Working Paper Series 2017-3, Inha University, Institute of Business and Economic Research, revised Apr 2017.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gaétan De Rassenfosse & Paul H. Jensen & T'Mir Julius & Alfons Palangkaraya & Elizabeth Webster, 2023. "Is the Patent System an Even Playing Field? The Effect of Patent Attorney Firms," Journal of Industrial Economics, Wiley Blackwell, vol. 71(1), pages 124-142, March.
    2. Zhu, Kejia & Malhotra, Shavin & Li, Yaohan, 2022. "Technological diversity of patent applications and decision pendency," Research Policy, Elsevier, vol. 51(1).
    3. Manuel Acosta & Daniel Coronado & Esther Ferrándiz & Manuel Jiménez, 2022. "Effects of knowledge spillovers between competitors on patent quality: what patent citations reveal about a global duopoly," The Journal of Technology Transfer, Springer, vol. 47(5), pages 1451-1487, October.
    4. Benjamin Barber & Luis Diestre, 2022. "Can firms avoid tough patent examiners through examiner‐shopping? Strategic timing of citations in USPTO patent applications," Strategic Management Journal, Wiley Blackwell, vol. 43(9), pages 1854-1871, September.
    5. Hain, Daniel S. & Jurowetzki, Roman & Buchmann, Tobias & Wolf, Patrick, 2022. "A text-embedding-based approach to measuring patent-to-patent technological similarity," Technological Forecasting and Social Change, Elsevier, vol. 177(C).
    6. Gaétan de Rassenfosse & Emilio Raiteri, 2022. "Technology Protectionism and the Patent System: Evidence from China," Journal of Industrial Economics, Wiley Blackwell, vol. 70(1), pages 1-43, March.
    7. Andrew Eckert & Corinne Langinier, 2014. "A Survey Of The Economics Of Patent Systems And Procedures," Journal of Economic Surveys, Wiley Blackwell, vol. 28(5), pages 996-1015, December.
    8. Yang, Haodong & Zhang, Yumei & Liu, Li, 2024. "Patent priority examination under mission orientation: can speed and quality coexist?," Technology in Society, Elsevier, vol. 76(C).
    9. Petit, Elise & van Pottelsberghe de la Potterie, Bruno & Gimeno-Fabra, Lluis, 2023. "Are patent offices substitutes?," Research Policy, Elsevier, vol. 52(8).
    10. Jungpyo Lee & So Young Sohn, 2017. "What makes the first forward citation of a patent occur earlier?," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 279-298, October.
    11. Fabian Gaessler & Dietmar Harhoff & Stefan Sorg & Georg von Graevenitz, 2024. "Patents, Freedom to Operate, and Follow-on Innovation: Evidence from Post-Grant Opposition," Rationality and Competition Discussion Paper Series 494, CRC TRR 190 Rationality and Competition.
    12. Gaétan de Rassenfosse & Reza Hosseini, 2020. "Discrimination against foreigners in the U.S. patent system," Journal of International Business Policy, Palgrave Macmillan, vol. 3(4), pages 349-366, December.
    13. Caviggioli, Federico & De Marco, Antonio & Montobbio, Fabio & Ughetto, Elisa, 2020. "The licensing and selling of inventions by US universities," Technological Forecasting and Social Change, Elsevier, vol. 159(C).
    14. deGrazia, Charles A.W. & Pairolero, Nicholas A. & Teodorescu, Mike H.M., 2021. "Examination incentives, learning, and patent office outcomes: The use of examiner’s amendments at the USPTO," Research Policy, Elsevier, vol. 50(10).
    15. Sun, Zhen & Wright, Brian D., 2022. "Citations backward and forward: Insights into the patent examiner's role," Research Policy, Elsevier, vol. 51(7).
    16. Prithwiraj Choudhury & Martine R. Haas, 2018. "Scope versus speed: Team diversity, leader experience, and patenting outcomes for firms," Strategic Management Journal, Wiley Blackwell, vol. 39(4), pages 977-1002, April.
    17. Cesare Righi & Timothy Simcoe, 2022. "Patenting inventions or inventing patents? Continuation practice at the USPTO," Economics Working Papers 1820, Department of Economics and Business, Universitat Pompeu Fabra.
    18. Cesare Righi & Timothy Simcoe, 2022. "Patenting Inventions or Inventing Patents? Continuation Practice at the USPTO," Working Papers 1320, Barcelona School of Economics.
    19. Appio, Francesco Paolo & Baglieri, Daniela & Cesaroni, Fabrizio & Spicuzza, Lucia & Donato, Alessia, 2022. "Patent design strategies: Empirical evidence from European patents," Technological Forecasting and Social Change, Elsevier, vol. 181(C).
    20. Higham, Kyle & de Rassenfosse, Gaétan & Jaffe, Adam B., 2021. "Patent Quality: Towards a Systematic Framework for Analysis and Measurement," Research Policy, Elsevier, vol. 50(4).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:128:y:2023:i:9:d:10.1007_s11192-023-04736-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.