IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v121y2019i1d10.1007_s11192-019-03196-8.html
   My bibliography  Save this article

Building journal’s long-term impact: using indicators detected from the sustained active articles

Author

Listed:
  • Mingyang Wang

    (Northeast Forestry University)

  • Shijia Jiao

    (Northeast Forestry University)

  • Kah-Hin Chai

    (National University of Singapore)

  • Guangsheng Chen

    (Northeast Forestry University)

Abstract

The Journal’s Impact Factor is an appropriate measure of recent concern rather than an effective measure of long-term impact of journals. This paper is mainly to find indicators that can effectively quantify the long-term impact of journal, with the aim to provide more useful supplementary information for journal evaluation. By examining the correlation between articles’ past citations and their future citations in different time windows, we found that the articles which were referenced in the past years will yield useful information also in the future. The age characteristics of these sustained active articles in journals provide clues for establishing long-term impact metrics for journals. A new indicator: h1-index was proposed to extract the active articles with at least the same number of citations as the h1-index in the statistical year. On this basis, four indicators describing the age characteristics of active articles were proposed to quantify the long-term impact of journals. The experimental results show that these indicators have a high correlation with the journal’s total citations, indicating that it is appropriate for these indicators to express the impact of the journal. Combining the average age of the active articles with the impact factors of journals, we found that some journals with short-term attraction strategies can also build long-term impact. The indicators presented in this paper that describe the long-term impact of journals will be a useful complement to journal quality assessment.

Suggested Citation

  • Mingyang Wang & Shijia Jiao & Kah-Hin Chai & Guangsheng Chen, 2019. "Building journal’s long-term impact: using indicators detected from the sustained active articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 261-283, October.
    • Handle: RePEc:spr:scient:v:121:y:2019:i:1:d:10.1007_s11192-019-03196-8
    DOI: 10.1007/s11192-019-03196-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-019-03196-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-019-03196-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Tudorel Andrei & Daniel Teodorescu & Andreea Mirică, 2016. "Beyond the Impact Factor: measuring the international visibility of Romanian social sciences journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 1-20, July.
    2. Gad Saad, 2006. "Exploring the h-index at the author and journal levels using bibliometric data of productive consumer scholars and business-related journals respectively," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 117-120, October.
    3. Tolga Yuret, 2018. "Author-weighted impact factor and reference return ratio: can we attain more equality among fields?," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2097-2111, September.
    4. Guang Yu & Yi-Jun Li, 2010. "Identification of referencing and citation processes of scientific journals based on the citation distribution model," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 249-261, February.
    5. Glänzel, Wolfgang & Schubert, András, 2010. "Hirsch-type characteristics of the tail of distributions. The generalised h-index," Journal of Informetrics, Elsevier, vol. 4(1), pages 118-123.
    6. Mikko Packalen & Jay Bhattacharya, 2017. "Neophilia ranking of scientific journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 43-64, January.
    7. Wolfgang Glänzel & Bart Thijs, 2018. "The role of baseline granularity for benchmarking citation impact. The case of CSS profiles," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(1), pages 521-536, July.
    8. Guang Yu & Yi-Jun Li, 2007. "Parameter identification of the observed citation distribution," Scientometrics, Springer;Akadémiai Kiadó, vol. 71(2), pages 339-348, May.
    9. Jürgen Harald Jacob & Siegfried Lehrl & Andreas Wolfram Henkel, 2007. "Early recognition of high quality researchers of the German psychiatry by worldwide accessible bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 73(2), pages 117-130, November.
    10. Peter Vinkler, 2010. "The πv-index: a new indicator to characterize the impact of journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(3), pages 461-475, March.
    11. Wolfgang Glänzel & Henk F. Moed, 2002. "Journal impact measures in bibliometric research," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 171-193, February.
    12. Bornmann, Lutz & Williams, Richard, 2017. "Can the journal impact factor be used as a criterion for the selection of junior researchers? A large-scale empirical study based on ResearcherID data," Journal of Informetrics, Elsevier, vol. 11(3), pages 788-799.
    13. Zhang, Lin & Thijs, Bart & Glänzel, Wolfgang, 2011. "The diffusion of H-related literature," Journal of Informetrics, Elsevier, vol. 5(4), pages 583-593.
    14. Lucio Bertoli-Barsotti & Tommaso Lando, 2017. "The h-index as an almost-exact function of some basic statistics," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1209-1228, November.
    15. Juan E. Iglesias & Carlos Pecharromán, 2007. "Scaling the h-index for different scientific ISI fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 73(3), pages 303-320, December.
    16. Loet Leydesdorff, 2012. "Alternatives to the journal impact factor: I3 and the top-10% (or top-25%?) of the most-highly cited papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 355-365, August.
    17. Dorta-González, P. & Dorta-González, M.I., 2013. "Impact maturity times and citation time windows: The 2-year maximum journal impact factor," Journal of Informetrics, Elsevier, vol. 7(3), pages 593-602.
    18. Seyed Reza Mirnezami & Catherine Beaudry & Vincent Larivière, 2016. "What determines researchers’ scientific impact? A case study of Quebec researchers," Science and Public Policy, Oxford University Press, vol. 43(2), pages 262-274.
    19. Juan Miguel Campanario, 2018. "Are leaders really leading? Journals that are first in Web of Science subject categories in the context of their groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 111-130, April.
    20. Heidi Prozesky & Nelius Boshoff, 2012. "Bibliometrics as a tool for measuring gender-specific research performance: an example from South African invasion ecology," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 383-406, February.
    21. Vanclay, Jerome K., 2008. "Ranking forestry journals using the h-index," Journal of Informetrics, Elsevier, vol. 2(4), pages 326-334.
    22. Jaroslav Fiala & Jiří J. Mareš & Jaroslav Šesták, 2017. "Reflections on how to evaluate the professional value of scientific papers and their corresponding citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 697-709, July.
    23. Dong-Hui Yang & Xin Li & Xiaoxia Sun & Jie Wan, 2016. "Detecting impact factor manipulation with data mining techniques," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1989-2005, December.
    24. Ichiko Fuyuno & David Cyranoski, 2006. "Cash for papers: putting a premium on publication," Nature, Nature, vol. 441(7095), pages 792-792, June.
    25. Michael Mimouni & Motti Ratmansky & Yaron Sacher & Sharon Aharoni & Aviva Mimouni-Bloch, 2016. "Self-citation rate and impact factor in pediatrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(3), pages 1455-1460, September.
    26. Pascal Bador & Thierry Lafouge, 2010. "Comparative analysis between impact factor and h-index for pharmacology and psychiatry journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 65-79, July.
    27. Péter Vinkler, 2001. "An attempt for defining some basic categories of scientometrics and classifying the indicators of evaluative scientometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 50(3), pages 539-544, March.
    28. Schubert, András & Glänzel, Wolfgang, 2007. "A systematic analysis of Hirsch-type indices for journals," Journal of Informetrics, Elsevier, vol. 1(3), pages 179-184.
    29. Yawen Zou & Manfred D. Laubichler, 2017. "Measuring the contributions of Chinese scholars to the research field of systems biology from 2005 to 2013," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1615-1631, March.
    30. Jacek Bogocz & Andrzej Bak & Jaroslaw Polanski, 2014. "No free lunches in nature? An analysis of the regional distribution of the affiliations of Nature publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 547-568, October.
    31. Pilar Valderrama & Manuel Escabias & Evaristo Jiménez-Contreras & Alberto Rodríguez-Archilla & Mariano J. Valderrama, 2018. "Proposal of a stochastic model to determine the bibliometric variables influencing the quality of a journal: application to the field of Dentistry," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 1087-1095, May.
    32. Erjia Yan & Chaojiang Wu & Min Song, 2018. "The funding factor: a cross-disciplinary examination of the association between research funding and citation impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 369-384, April.
    33. Loet Leydesdorff, 2013. "An evaluation of impacts in “Nanoscience & nanotechnology”: steps towards standards for citation analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(1), pages 35-55, January.
    34. Adam Eyre-Walker & Nina Stoletzki, 2013. "The Assessment of Science: The Relative Merits of Post-Publication Review, the Impact Factor, and the Number of Citations," PLOS Biology, Public Library of Science, vol. 11(10), pages 1-8, October.
    35. James Hartley, 2012. "To cite or not to cite: author self-citations and the impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 313-317, August.
    36. Péter Vinkler, 2009. "Introducing the Current Contribution Index for characterizing the recent, relevant impact of journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(2), pages 409-420, May.
    37. Éric Archambault & Vincent Larivière, 2009. "History of the journal impact factor: Contingencies and consequences," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(3), pages 635-649, June.
    38. Anne‐Wil Harzing & Ron van der Wal, 2009. "A Google Scholar h‐index for journals: An alternative metric to measure journal impact in economics and business," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(1), pages 41-46, January.
    39. Gangan Prathap & S. Mini & P. Nishy, 2016. "Does high impact factor successfully predict future citations? An analysis using Peirce’s measure," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(3), pages 1043-1047, September.
    40. Pei-Shan Chi & Wolfgang Glänzel, 2018. "Comparison of citation and usage indicators in research assessment in scientific disciplines and journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(1), pages 537-554, July.
    41. Loet Leydesdorff, 2008. "Caveats for the use of citation indicators in research and journal evaluations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(2), pages 278-287, January.
    42. Peter Vinkler, 2004. "Characterization of the impact of sets of scientific papers: The Garfield (impact) factor," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 55(5), pages 431-435, March.
    43. Péter Vinkler, 2012. "The Garfield impact factor, one of the fundamental indicators in scientometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 471-483, August.
    44. Antonoyiannakis, Manolis, 2018. "Impact Factors and the Central Limit Theorem: Why citation averages are scale dependent," Journal of Informetrics, Elsevier, vol. 12(4), pages 1072-1088.
    45. Renata R. Gonçalves & Christian Kieling & Rodrigo A. Bressan & Jair J. Mari & Luis A. Rohde, 2009. "The evaluation of scientific productivity in Brazil: An assessment of the mental health field," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(2), pages 529-537, August.
    46. Keshra Sangwal, 2013. "Some citation-related characteristics of scientific journals published in individual countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 719-741, December.
    47. Lorelei R. Vinluan, 2012. "Research productivity in education and psychology in the Philippines and comparison with ASEAN countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(1), pages 277-294, April.
    48. Malesios, C., 2016. "Measuring the robustness of the journal h-index with respect to publication and citation values: A Bayesian sensitivity analysis," Journal of Informetrics, Elsevier, vol. 10(3), pages 719-731.
    49. Tibor Braun & Wolfgang Glänzel & András Schubert, 2006. "A Hirsch-type index for journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 169-173, October.
    50. Lucio Bertoli-Barsotti & Tommaso Lando, 2017. "A theoretical model of the relationship between the h-index and other simple citation indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1415-1448, June.
    51. Iman Tahamtan & Askar Safipour Afshar & Khadijeh Ahamdzadeh, 2016. "Factors affecting number of citations: a comprehensive review of the literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1195-1225, June.
    52. Péter Vinkler, 2017. "The size and impact of the elite set of publications in scientometric assessments," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 163-177, January.
    53. Guang Yu & Xiao-Hong Wang & Da-Ren Yu, 2005. "The influence of publication delays on impact factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(2), pages 235-246, August.
    54. Lutz Bornmann, 2017. "Confidence intervals for Journal Impact Factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1869-1871, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marcel Clermont & Johanna Krolak & Dirk Tunger, 2021. "Does the citation period have any effect on the informative value of selected citation indicators in research evaluations?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1019-1047, February.
    2. Wanjun Xia & Tianrui Li & Chongshou Li, 2023. "A review of scientific impact prediction: tasks, features and methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 543-585, January.
    3. Mingyang Wang & Jiaqi Zhang & Shijia Jiao & Xiangrong Zhang & Na Zhu & Guangsheng Chen, 2020. "Important citation identification by exploiting the syntactic and contextual information of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2109-2129, December.
    4. Weibin Wang & Zheng Wang & Tian Yu & CholMyong Pak & Guang Yu, 2020. "Research on citation mention times and contributions using a neural network," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2383-2400, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Péter Vinkler, 2011. "Application of the distribution of citations among publications in scientometric evaluations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(10), pages 1963-1978, October.
    2. Alonso, S. & Cabrerizo, F.J. & Herrera-Viedma, E. & Herrera, F., 2009. "h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, Elsevier, vol. 3(4), pages 273-289.
    3. William Cabos & Juan Miguel Campanario, 2018. "Exploring the Hjif-Index, an Analogue to the H-Like Index for Journal Impact Factors," Publications, MDPI, vol. 6(2), pages 1-11, April.
    4. Bertoli-Barsotti, Lucio & Lando, Tommaso, 2019. "How mean rank and mean size may determine the generalised Lorenz curve: With application to citation analysis," Journal of Informetrics, Elsevier, vol. 13(1), pages 387-396.
    5. Mingkun Wei, 2020. "Research on impact evaluation of open access journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 1027-1049, February.
    6. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    7. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    8. Lucio Bertoli-Barsotti & Tommaso Lando, 2017. "A theoretical model of the relationship between the h-index and other simple citation indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1415-1448, June.
    9. de Carvalho, Gustavo Dambiski Gomes & Sokulski, Carla Cristiane & da Silva, Wesley Vieira & de Carvalho, Hélio Gomes & de Moura, Rafael Vignoli & de Francisco, Antonio Carlos & da Veiga, Claudimar Per, 2020. "Bibliometrics and systematic reviews: A comparison between the Proknow-C and the Methodi Ordinatio," Journal of Informetrics, Elsevier, vol. 14(3).
    10. Juan Miguel Campanario, 2018. "Are leaders really leading? Journals that are first in Web of Science subject categories in the context of their groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 111-130, April.
    11. Mingyang Wang & Jiaqi Zhang & Shijia Jiao & Xiangrong Zhang & Na Zhu & Guangsheng Chen, 2020. "Important citation identification by exploiting the syntactic and contextual information of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2109-2129, December.
    12. Tingcan Ma & Gui-Fang Wang & Ke Dong & Mukun Cao, 2012. "The Journal’s Integrated Impact Index: a new indicator for journal evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 649-658, February.
    13. Petridis, Konstantinos & Malesios, Chrisovalantis & Arabatzis, Garyfallos & Thanassoulis, Emmanuel, 2013. "Efficiency analysis of forestry journals: Suggestions for improving journals’ quality," Journal of Informetrics, Elsevier, vol. 7(2), pages 505-521.
    14. Mingers, John & Yang, Liying, 2017. "Evaluating journal quality: A review of journal citation indicators and ranking in business and management," European Journal of Operational Research, Elsevier, vol. 257(1), pages 323-337.
    15. Juan Miguel Campanario, 2018. "Journals that Rise from the Fourth Quartile to the First Quartile in Six Years or Less: Mechanisms of Change and the Role of Journal Self-Citations," Publications, MDPI, vol. 6(4), pages 1-15, November.
    16. Wei, Shelia X. & Tong, Tong & Rousseau, Ronald & Wang, Wanru & Ye, Fred Y., 2022. "Relations among the h-, g-, ψ-, and p-index and offset-ability," Journal of Informetrics, Elsevier, vol. 16(4).
    17. Péter Vinkler, 2019. "Core journals and elite subsets in scientometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 241-259, October.
    18. Zhang, Lin & Thijs, Bart & Glänzel, Wolfgang, 2011. "The diffusion of H-related literature," Journal of Informetrics, Elsevier, vol. 5(4), pages 583-593.
    19. Petr Praus, 2019. "High-ranked citations percentage as an indicator of publications quality," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(1), pages 319-329, July.
    20. Vinkler, Péter, 2013. "Would it be possible to increase the Hirsch-index, π-index or CDS-index by increasing the number of publications or citations only by unity?," Journal of Informetrics, Elsevier, vol. 7(1), pages 72-83.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:121:y:2019:i:1:d:10.1007_s11192-019-03196-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.