IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v13y2021i13p7147-d582138.html
   My bibliography  Save this article

Data-Driven Public R&D Project Performance Evaluation: Results from China

Author

Listed:
  • Hongbo Li

    (School of Management, Shanghai University, Shanghai 200044, China)

  • Bowen Yao

    (School of Management, Shanghai University, Shanghai 200044, China)

  • Xin Yan

    (School of Management, Shanghai University, Shanghai 200044, China)

Abstract

In public R&D projects, to improve the decision-making process and ensure the sustainability of public investment, it is indispensable to effectively evaluate the project performance. Currently, public R&D project management departments and various academic databases have accumulated a large number of project-related data. In view of this, we propose a data-driven performance evaluation framework for public R&D projects. In our framework, we collect structured and unstructured data related to completed projects from multiple websites. Then, these data are cleaned and fused to form a unified dataset. We train a project performance evaluation model by extracting the project performance information implicit in the dataset based on multi-classification supervised learning algorithms. When facing a new project that needs to be evaluated, its performance can be automatically predicted by inputting the characteristic information of the project into our performance evaluation model. Our framework is validated based on the project data of the National Natural Science Foundation of China (NSFC) in terms of four performance measures (i.e., Accuracy, Recall, Precision, F 1 score). In addition, we provide a case study that applies our framework to evaluate the project performance in the logistics and supply chain area of NSFC. In conclusion, this paper contributes to the body of knowledge in sustainability by developing a data-driven method that equips the decision-maker with an automated project performance evaluation tool to make sustainable project decisions.

Suggested Citation

  • Hongbo Li & Bowen Yao & Xin Yan, 2021. "Data-Driven Public R&D Project Performance Evaluation: Results from China," Sustainability, MDPI, vol. 13(13), pages 1-14, June.
    • Handle: RePEc:gam:jsusta:v:13:y:2021:i:13:p:7147-:d:582138
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/13/13/7147/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/13/13/7147/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ji-ping Gao & Cheng Su & Hai-yan Wang & Li-hua Zhai & Yun-tao Pan, 2019. "Research fund evaluation based on academic publication output analysis: the case of Chinese research fund evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 959-972, May.
    2. Johnes, Jill & Yu, Li, 2008. "Measuring the research performance of Chinese higher education institutions using data envelopment analysis," China Economic Review, Elsevier, vol. 19(4), pages 679-696, December.
    3. Woo Sung Kim & Kunsu Park & Sang Hoon Lee & Hongyoung Kim, 2018. "R&D Investments and Firm Value: Evidence from China," Sustainability, MDPI, vol. 10(11), pages 1-17, November.
    4. Hsu, Fang-Ming & Hsueh, Chao-Chih, 2009. "Measuring relative efficiency of government-sponsored R&D projects: A three-stage approach," Evaluation and Program Planning, Elsevier, vol. 32(2), pages 178-186, May.
    5. Karasakal, Esra & Aker, Pınar, 2017. "A multicriteria sorting approach based on data envelopment analysis for R&D project selection problem," Omega, Elsevier, vol. 73(C), pages 79-92.
    6. Jaehun Park & Joonyoung Kim & Si-Il Sung, 2017. "Performance Evaluation of Research and Business Development: A Case Study of Korean Public Organizations," Sustainability, MDPI, vol. 9(12), pages 1-16, December.
    7. Eilat, Harel & Golany, Boaz & Shtub, Avraham, 2008. "R&D project evaluation: An integrated DEA and balanced scorecard approach," Omega, Elsevier, vol. 36(5), pages 895-912, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hongbo Li & Rui Chen & Xianchao Zhang, 2022. "Uncertain Public R&D Project Portfolio Selection Considering Sectoral Balancing and Project Failure," Sustainability, MDPI, vol. 14(23), pages 1-13, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dalton Garcia Borges de Souza & Erivelton Antonio dos Santos & Nei Yoshihiro Soma & Carlos Eduardo Sanches da Silva, 2021. "MCDM-Based R&D Project Selection: A Systematic Literature Review," Sustainability, MDPI, vol. 13(21), pages 1-34, October.
    2. Kun Chen & Xian-tong Ren & Guo-liang Yang & Hai-bo Qin, 2022. "The other side of the coin: The declining of Chinese social science," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 127-143, January.
    3. Sungmin Park, 2015. "The R&D logic model: Does it really work? An empirical verification using successive binary logistic regression models," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1399-1439, December.
    4. Lu, Wen-Min & Kweh, Qian Long & Nourani, Mohammad & Huang, Feng-Wen, 2016. "Evaluating the efficiency of dual-use technology development programs from the R&D and socio-economic perspectives," Omega, Elsevier, vol. 62(C), pages 82-92.
    5. Li, Yan-Lai & Tang, Jia-Fu & Chin, Kwai-Sang & Jiang, Yu-Shi & Han, Yi & Pu, Yun, 2011. "Estimating the final priority ratings of engineering characteristics in mature-period product improvement by MDBA and AHP," International Journal of Production Economics, Elsevier, vol. 131(2), pages 575-586, June.
    6. Chen, Zhongfei & Wanke, Peter & Tsionas, Mike G., 2018. "Assessing the strategic fit of potential M&As in Chinese banking: A novel Bayesian stochastic frontier approach," Economic Modelling, Elsevier, vol. 73(C), pages 254-263.
    7. Hongwei Liu & Ronglu Yang & Zhixiang Zhou & Dacheng Huang, 2020. "Regional Green Eco-Efficiency in China: Considering Energy Saving, Pollution Treatment, and External Environmental Heterogeneity," Sustainability, MDPI, vol. 12(17), pages 1-19, August.
    8. Salimi, Negin & Rezaei, Jafar, 2018. "Evaluating firms’ R&D performance using best worst method," Evaluation and Program Planning, Elsevier, vol. 66(C), pages 147-155.
    9. Sung-Shun Weng & Yang Liu & Yen-Ching Chuang, 2019. "Reform of Chinese Universities in the Context of Sustainable Development: Teacher Evaluation and Improvement Based on Hybrid Multiple Criteria Decision-Making Model," Sustainability, MDPI, vol. 11(19), pages 1-23, October.
    10. Samoilenko, Sergey & Osei-Bryson, Kweku-Muata, 2013. "Using Data Envelopment Analysis (DEA) for monitoring efficiency-based performance of productivity-driven organizations: Design and implementation of a decision support system," Omega, Elsevier, vol. 41(1), pages 131-142.
    11. Fu, Tsu-Tan & See, Kok Fong, 2022. "An integrated analysis of quality and productivity growth in China’s and Taiwan’s higher education institutions," Economic Analysis and Policy, Elsevier, vol. 74(C), pages 234-249.
    12. Ding, Tao & Zhang, Yun & Zhang, Danlu & Li, Feng, 2023. "Performance evaluation of Chinese research universities: A parallel interactive network DEA approach with shared and fixed sum inputs," Socio-Economic Planning Sciences, Elsevier, vol. 87(PA).
    13. Xu, Jia & Wei, Jiuchang & Zhao, Dingtao, 2016. "Influence of social media on operational efficiency of national scenic spots in china based on three-stage DEA model," International Journal of Information Management, Elsevier, vol. 36(3), pages 374-388.
    14. Xianmei Wang & Hanhui Hu, 2017. "Sustainability in Chinese Higher Educational Institutions’ Social Science Research: A Performance Interface toward Efficiency," Sustainability, MDPI, vol. 9(11), pages 1-18, October.
    15. Claudia Capozza & Angela Stefania Bergantino & Angela De Carlo, 2013. "The Role of Transport Infrastructures in determining Technical Efficiency in R&D activity of Italian regions. A double-bootstrapped DEA procedure," ERSA conference papers ersa13p1052, European Regional Science Association.
    16. Sun, Yu & Yang, Feng & Wang, Dawei & Ang, Sheng, 2023. "Efficiency evaluation for higher education institutions in China considering unbalanced regional development: A meta-frontier Super-SBM model," Socio-Economic Planning Sciences, Elsevier, vol. 88(C).
    17. Rafael Lizarralde & Jaione Ganzarain & Mikel Zubizarreta, 2020. "Assessment and Selection of Technologies for the Sustainable Development of an R&D Center," Sustainability, MDPI, vol. 12(23), pages 1-23, December.
    18. George E. Halkos & Nickolaos G. Tzeremes & Stavros A. Kourtzidis, 2010. "An application of statistical interference in DEA models: An analysis of public owned university departments' efficiency," EERI Research Paper Series EERI_RP_2010_17, Economics and Econometrics Research Institute (EERI), Brussels.
    19. Zhang, Daqun & Banker, Rajiv D. & Li, Xiaoxuan & Liu, Wenbin, 2011. "Performance impact of research policy at the Chinese Academy of Sciences," Research Policy, Elsevier, vol. 40(6), pages 875-885, July.
    20. Liu, John S. & Lu, Louis Y.Y. & Lu, Wen-Min & Lin, Bruce J.Y., 2013. "A survey of DEA applications," Omega, Elsevier, vol. 41(5), pages 893-902.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:13:y:2021:i:13:p:7147-:d:582138. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.