IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v124y2020i1d10.1007_s11192-020-03479-5.html
   My bibliography  Save this article

Predicting the future success of scientific publications through social network and semantic analysis

Author

Listed:
  • Andrea Fronzetti Colladon

    (University of Perugia)

  • Ciriaco Andrea D’Angelo

    (University of Rome “Tor Vergata”)

  • Peter A. Gloor

    (MIT Center for Collective Intelligence)

Abstract

Citations acknowledge the impact a scientific publication has on subsequent work. At the same time, deciding how and when to cite a paper, is also heavily influenced by social factors. In this work, we conduct an empirical analysis based on a dataset of 2010–2012 global publications in chemical engineering. We use social network analysis and text mining to measure publication attributes and understand which variables can better help predicting their future success. Controlling for intrinsic quality of a publication and for the number of authors in the byline, we are able to predict scholarly impact of a paper in terms of citations received 6 years after publication with almost 80% accuracy. Results suggest that, all other things being equal, it is better to co-publish with rotating co-authors and write the papers’ abstract using more positive words, and a more complex, thus more informative, language. Publications that result from the collaboration of different social groups also attract more citations.

Suggested Citation

  • Andrea Fronzetti Colladon & Ciriaco Andrea D’Angelo & Peter A. Gloor, 2020. "Predicting the future success of scientific publications through social network and semantic analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 357-377, July.
    • Handle: RePEc:spr:scient:v:124:y:2020:i:1:d:10.1007_s11192-020-03479-5
    DOI: 10.1007/s11192-020-03479-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-020-03479-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-020-03479-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Guan, Jiancheng & Yan, Yan & Zhang, Jing Jing, 2017. "The impact of collaboration and knowledge networks on citations," Journal of Informetrics, Elsevier, vol. 11(2), pages 407-422.
    2. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2014. "How to improve the prediction based on citation impact percentiles for years shortly after the publication date?," Journal of Informetrics, Elsevier, vol. 8(1), pages 175-180.
    3. Yared H. Kidane & Peter A. Gloor, 2007. "Correlating temporal communication patterns of the Eclipse open source community with performance and creativity," Computational and Mathematical Organization Theory, Springer, vol. 13(1), pages 17-27, March.
    4. Stegehuis, Clara & Litvak, Nelly & Waltman, Ludo, 2015. "Predicting the long-term citation impact of recent publications," Journal of Informetrics, Elsevier, vol. 9(3), pages 642-657.
    5. Trivedi, Pravin K, 1993. "An Analysis of Publication Lags in Econometrics," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 8(1), pages 93-100, Jan.-Marc.
    6. Lakshmi Balachandran Nair & Michael Gibbert, 2016. "What makes a ‘good’ title and (how) does it matter for citations? A review and general model of article title attributes in management science," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1331-1359, June.
    7. Wuestman, Mignon L. & Hoekman, Jarno & Frenken, Koen, 2019. "The geography of scientific citations," Research Policy, Elsevier, vol. 48(7), pages 1771-1780.
    8. Jha, Yamini & Welch, Eric W., 2010. "Relational mechanisms governing multifaceted collaborative behavior of academic scientists in six fields of science and engineering," Research Policy, Elsevier, vol. 39(9), pages 1174-1184, November.
    9. Uddin, Shahadat & Khan, Arif, 2016. "The impact of author-selected keywords on citation counts," Journal of Informetrics, Elsevier, vol. 10(4), pages 1166-1177.
    10. Hirotaka Kawashima & Hiroyuki Tomizawa, 2015. "Accuracy evaluation of Scopus Author ID based on the largest funding database in Japan," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 1061-1071, June.
    11. Alexander Karlsson & Björn Hammarfelt & H. Joe Steinhauer & Göran Falkman & Nasrine Olson & Gustaf Nelhans & Jan Nolin, 2015. "Modeling uncertainty in bibliometrics and information retrieval: an information fusion approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(3), pages 2255-2274, March.
    12. Giovanni Abramo & Ciriaco Andrea D’Angelo & Tindaro Cicero, 2012. "What is the appropriate length of the publication period over which to assess research performance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 1005-1017, December.
    13. Hoekman, Jarno & Frenken, Koen & Tijssen, Robert J.W., 2010. "Research collaboration at a distance: Changing spatial patterns of scientific collaboration within Europe," Research Policy, Elsevier, vol. 39(5), pages 662-673, June.
    14. Gunther Eysenbach, 2006. "Citation Advantage of Open Access Articles," Working Papers id:626, eSocialSciences.
    15. Gloor, Peter & Fronzetti Colladon, Andrea & Giacomelli, Gianni & Saran, Tejasvita & Grippa, Francesca, 2017. "The impact of virtual mirroring on customer satisfaction," Journal of Business Research, Elsevier, vol. 75(C), pages 67-76.
    16. Stephan B. Bruns & David I. Stern, 2016. "Research assessment using early citation information," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(2), pages 917-935, August.
    17. James S Dietz, 2000. "Building a social capital model of research development: The case of the Experimental Program to Stimulate Competitive Research," Science and Public Policy, Oxford University Press, vol. 27(2), pages 137-145, April.
    18. Ong, David & Chan, Ho Fai & Torgler, Benno & Yang, Yu (Alan), 2018. "Collaboration incentives: Endogenous selection into single and coauthorships by surname initial in economics and management," Journal of Economic Behavior & Organization, Elsevier, vol. 147(C), pages 41-57.
    19. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Felici, Giovanni, 2019. "Predicting publication long-term impact through a combination of early citations and journal impact factor," Journal of Informetrics, Elsevier, vol. 13(1), pages 32-49.
    20. Benjamin Freeling & Zoë A. Doubleday & Sean D. Connell, 2019. "Opinion: How can we boost the impact of publications? Try better writing," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 116(2), pages 341-343, January.
    21. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Costa, 2016. "The effect of a country’s name in the title of a publication on its visibility and citability," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1895-1909, December.
    22. Guerrero-Bote, Vicente P. & Moya-Anegón, Félix, 2012. "A further step forward in measuring journals’ scientific prestige: The SJR2 indicator," Journal of Informetrics, Elsevier, vol. 6(4), pages 674-688.
    23. Anthony F. J. van Raan, 2004. "Sleeping Beauties in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 59(3), pages 467-472, March.
    24. Petersen, Alexander M. & Pan, Raj K. & Pammolli, Fabio & Fortunato, Santo, 2019. "Methods to account for citation inflation in research evaluation," Research Policy, Elsevier, vol. 48(7), pages 1855-1865.
    25. Franceschet, Massimo & Costantini, Antonio, 2010. "The effect of scholar collaboration on impact and quality of academic papers," Journal of Informetrics, Elsevier, vol. 4(4), pages 540-553.
    26. Wei Huang, 2015. "DO ABCs GET MORE CITATIONS THAN XYZs?," Economic Inquiry, Western Economic Association International, vol. 53(1), pages 773-789, January.
    27. Grit Laudel, 2002. "What do we measure by co-authorships?," Research Evaluation, Oxford University Press, vol. 11(1), pages 3-15, April.
    28. Li, Menghui & Wu, Jinshan & Wang, Dahui & Zhou, Tao & Di, Zengru & Fan, Ying, 2007. "Evolving model of weighted networks inspired by scientific collaboration networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 375(1), pages 355-364.
    29. Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2015. "The relationship between the number of authors of a publication, its citations and the impact factor of the publishing journal: Evidence from Italy," Journal of Informetrics, Elsevier, vol. 9(4), pages 746-761.
    30. Katz, J. Sylvan & Martin, Ben R., 1997. "What is research collaboration?," Research Policy, Elsevier, vol. 26(1), pages 1-18, March.
    31. Fatemeh Rostami & Asghar Mohammadpoorasl & Mohammad Hajizadeh, 2014. "The effect of characteristics of title on citation rates of articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(3), pages 2007-2010, March.
    32. Giovanni Abramo & Ciriaco Andrea D’Angelo & Emanuela Reale, 2019. "Peer review versus bibliometrics: Which method better predicts the scholarly impact of publications?," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 537-554, October.
    33. Julia Melkers & Agrita Kiopa, 2010. "The Social Capital of Global Ties in Science: The Added Value of International Collaboration," Review of Policy Research, Policy Studies Organization, vol. 27(4), pages 389-414, July.
    34. Murray, Catherine, 2005. "Social Capital and Cooperation in Central and Eastern Europe: A Theoretical Perspective," Institutional Change in Agriculture and Natural Resources Discussion Papers 18831, Humboldt University Berlin, Department of Agricultural Economics.
    35. Letchford, Adrian & Preis, Tobias & Moat, Helen Susannah, 2016. "The advantage of simple paper abstracts," Journal of Informetrics, Elsevier, vol. 10(1), pages 1-8.
    36. Mingers, John & Xu, Fang, 2010. "The drivers of citations in management science journals," European Journal of Operational Research, Elsevier, vol. 205(2), pages 422-430, September.
    37. Hamid R. Jamali & Mahsa Nikzad, 2011. "Article title type and its relation with the number of downloads and citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(2), pages 653-661, August.
    38. Mark Shevlin & Mark N. O. Davies, 1997. "Alphabetical listing and citation rates," Nature, Nature, vol. 388(6637), pages 14-14, July.
    39. Li, Eldon Y. & Liao, Chien Hsiang & Yen, Hsiuju Rebecca, 2013. "Co-authorship networks and research impact: A social capital perspective," Research Policy, Elsevier, vol. 42(9), pages 1515-1530.
    40. Sabine Brunswicker & Sorin Adam Matei & Michael Zentner & Lynn Zentner & Gerhard Klimeck, 2017. "Creating impact in the digital space: digital practice dependency in communities of digital scientific innovations," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 417-442, January.
    41. Defazio, Daniela & Lockett, Andy & Wright, Mike, 2009. "Funding incentives, collaborative dynamics and scientific productivity: Evidence from the EU framework program," Research Policy, Elsevier, vol. 38(2), pages 293-305, March.
    42. Perc, Matjaž, 2010. "Growth and structure of Slovenia’s scientific collaboration network," Journal of Informetrics, Elsevier, vol. 4(4), pages 475-482.
    43. Jonas Lundberg & Göran Tomson & Inger Lundkvist & John Sk?r & Mats Brommels, 2006. "Collaboration uncovered: Exploring the adequacy of measuring university-industry collaboration through co-authorship and funding," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(3), pages 575-589, December.
    44. Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2017. "Does your surname affect the citability of your publications?," Journal of Informetrics, Elsevier, vol. 11(1), pages 121-127.
    45. Jinseok Kim & Jana Diesner, 2015. "Coauthorship networks: A directed network approach considering the order and number of coauthors," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(12), pages 2685-2696, December.
    46. Wang, Jian, 2014. "Unpacking the Matthew effect in citations," Journal of Informetrics, Elsevier, vol. 8(2), pages 329-339.
    47. ., 1998. "Methodology of Scientific Research Programmes," Chapters, in: John B. Davis & D. W. Hands & Uskali Mäki (ed.), The Handbook of Economic Methodology, chapter 70, Edward Elgar Publishing.
    48. Peter Weingart, 2005. "Impact of bibliometrics upon the science system: Inadvertent consequences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 117-131, January.
    49. Barabási, A.L & Jeong, H & Néda, Z & Ravasz, E & Schubert, A & Vicsek, T, 2002. "Evolution of the social network of scientific collaborations," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 311(3), pages 590-614.
    50. Shahadat Uddin & Liaquat Hossain & Alireza Abbasi & Kim Rasmussen, 2012. "Trend and efficiency analysis of co-authorship network," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 687-699, February.
    51. Vincent Larivière & Yves Gingras & Cassidy R. Sugimoto & Andrew Tsou, 2015. "Team size matters: Collaboration and scientific impact since 1900," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(7), pages 1323-1332, July.
    52. Vedran Sekara & Pierre Deville & Sebastian E. Ahnert & Albert-László Barabási & Roberta Sinatra & Sune Lehmann, 2018. "The chaperone effect in scientific publishing," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 115(50), pages 12603-12607, December.
    53. ., 1998. "Scientific Explanation," Chapters, in: John B. Davis & D. W. Hands & Uskali Mäki (ed.), The Handbook of Economic Methodology, chapter 109, Edward Elgar Publishing.
    54. Donald deB. Beaver, 2004. "Does collaborative research have greater epistemic authority?," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(3), pages 399-408, August.
    55. Waltman, Ludo & van Eck, Nees Jan, 2015. "Field-normalized citation impact indicators and the choice of an appropriate counting method," Journal of Informetrics, Elsevier, vol. 9(4), pages 872-894.
    56. ., 1998. "Sociology of Scientific Knowledge, The," Chapters, in: John B. Davis & D. W. Hands & Uskali Mäki (ed.), The Handbook of Economic Methodology, chapter 118, Edward Elgar Publishing.
    57. Bozeman, Barry & Corley, Elizabeth, 2004. "Scientists' collaboration strategies: implications for scientific and technical human capital," Research Policy, Elsevier, vol. 33(4), pages 599-616, May.
    58. Matveeva, Nataliya & Poldin, Oleg, 2016. "Citation of scholars in co-authorship network: Analysis of Google Scholar data," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 44, pages 100-118.
    59. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2011. "Assessing the varying level of impact measurement accuracy as a function of the citation window length," Journal of Informetrics, Elsevier, vol. 5(4), pages 659-667.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhai, Li & Yan, Xiangbin, 2022. "A directed collaboration network for exploring the order of scientific collaboration," Journal of Informetrics, Elsevier, vol. 16(4).
    2. Katchanov, Yurij L. & Markova, Yulia V. & Shmatko, Natalia A., 2023. "Uncited papers in the structure of scientific communication," Journal of Informetrics, Elsevier, vol. 17(2).
    3. Wanjun Xia & Tianrui Li & Chongshou Li, 2023. "A review of scientific impact prediction: tasks, features and methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 543-585, January.
    4. Haochuan Cui & Tiewei Li & Cheng-Jun Wang, 2023. "Climbing up the ladder of abstraction: how to span the boundaries of knowledge space in the online knowledge market?," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-12, December.
    5. Anqi Ma & Yu Liu & Xiujuan Xu & Tao Dong, 2021. "A deep-learning based citation count prediction model with paper metadata semantic features," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6803-6823, August.
    6. Olivia Fischer & Loris T. Jeitziner & Dirk U. Wulff, 2024. "Affect in science communication: a data-driven analysis of TED Talks on YouTube," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-9, December.
    7. Don Watson & Manfred Krug & Claus-Christian Carbon, 2022. "The relationship between citations and the linguistic traits of specific academic discourse communities identified by using social network analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 1755-1781, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Martorell Cunil, Onofre & Otero González, Luis & Durán Santomil, Pablo & Mulet Forteza, Carlos, 2023. "How to accomplish a highly cited paper in the tourism, leisure and hospitality field," Journal of Business Research, Elsevier, vol. 157(C).
    2. Marian-Gabriel Hâncean & Matjaž Perc & Jürgen Lerner, 2021. "The coauthorship networks of the most productive European researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 201-224, January.
    3. Graf, Holger & Kalthaus, Martin, 2018. "International research networks: Determinants of country embeddedness," Research Policy, Elsevier, vol. 47(7), pages 1198-1214.
    4. Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2015. "The relationship between the number of authors of a publication, its citations and the impact factor of the publishing journal: Evidence from Italy," Journal of Informetrics, Elsevier, vol. 9(4), pages 746-761.
    5. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Di Costa, 2019. "The collaboration behavior of top scientists," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 215-232, January.
    6. Chen, Kaihua & Zhang, Yi & Fu, Xiaolan, 2019. "International research collaboration: An emerging domain of innovation studies?," Research Policy, Elsevier, vol. 48(1), pages 149-168.
    7. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    8. Elizabeth S. Vieira, 2023. "The influence of research collaboration on citation impact: the countries in the European Innovation Scoreboard," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3555-3579, June.
    9. Hongquan Shen & Juan Xie & Jiang Li & Ying Cheng, 2021. "The correlation between scientific collaboration and citation count at the paper level: a meta-analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3443-3470, April.
    10. Gao, Qiang & Liang, Zhentao & Wang, Ping & Hou, Jingrui & Chen, Xiuxiu & Liu, Manman, 2021. "Potential index: Revealing the future impact of research topics based on current knowledge networks," Journal of Informetrics, Elsevier, vol. 15(3).
    11. Anqi Ma & Yu Liu & Xiujuan Xu & Tao Dong, 2021. "A deep-learning based citation count prediction model with paper metadata semantic features," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6803-6823, August.
    12. Ana Fernández & Esther Ferrándiz & M. Dolores León, 2021. "Are organizational and economic proximity driving factors of scientific collaboration? Evidence from Spanish universities, 2001–2010," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 579-602, January.
    13. Chaocheng He & Jiang Wu & Qingpeng Zhang, 2021. "Characterizing research leadership on geographically weighted collaboration network," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(5), pages 4005-4037, May.
    14. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    15. Hugo Confraria & Fernando Vargas, 2019. "Scientific systems in Latin America: performance, networks, and collaborations with industry," The Journal of Technology Transfer, Springer, vol. 44(3), pages 874-915, June.
    16. Chin-Chang Tsai & Elizabeth A. Corley & Barry Bozeman, 2016. "Collaboration experiences across scientific disciplines and cohorts," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(2), pages 505-529, August.
    17. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Costa, 2019. "A gender analysis of top scientists’ collaboration behavior: evidence from Italy," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 405-418, August.
    18. Cimenler, Oguz & Reeves, Kingsley A. & Skvoretz, John, 2014. "A regression analysis of researchers’ social network metrics on their citation performance in a college of engineering," Journal of Informetrics, Elsevier, vol. 8(3), pages 667-682.
    19. Letina, Srebrenka, 2016. "Network and actor attribute effects on the performance of researchers in two fields of social science in a small peripheral community," Journal of Informetrics, Elsevier, vol. 10(2), pages 571-595.
    20. Anna Małgorzata Kamińska & Łukasz Opaliński & Łukasz Wyciślik, 2022. "The Landscapes of Sustainability in the Library and Information Science: Collaboration Insights," Sustainability, MDPI, vol. 14(24), pages 1-23, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:124:y:2020:i:1:d:10.1007_s11192-020-03479-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.