IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v83y2010i1d10.1007_s11192-009-0078-y.html
   My bibliography  Save this article

Assessing the quality of scientific conferences based on bibliographic citations

Author

Listed:
  • Waister Silva Martins

    (Federal University of Minas Gerais)

  • Marcos André Gonçalves

    (Federal University of Minas Gerais)

  • Alberto H. F. Laender

    (Federal University of Minas Gerais)

  • Nivio Ziviani

    (Federal University of Minas Gerais)

Abstract

Assessing the quality of scientific conferences is an important and useful service that can be provided by digital libraries and similar systems. This is specially true for fields such as Computer Science and Electric Engineering, where conference publications are crucial. However, the majority of the existing quality metrics, particularly those relying on bibliographic citations, has been proposed for measuring the quality of journals. In this article we conduct a study about the relative performance of existing journal metrics in assessing the quality of scientific conferences. More importantly, departing from a deep analysis of the deficiencies of these metrics, we propose a new set of quality metrics especially designed to capture intrinsic and important aspects related to conferences, such as longevity, popularity, prestige, and periodicity. To demonstrate the effectiveness of the proposed metrics, we have conducted two sets of experiments that contrast their results against a “gold standard” produced by a large group of specialists. Our metrics obtained gains of more than 12% when compared to the most consistent journal quality metric and up to 58% when compared to standard metrics such as Thomson’s Impact Factor.

Suggested Citation

  • Waister Silva Martins & Marcos André Gonçalves & Alberto H. F. Laender & Nivio Ziviani, 2010. "Assessing the quality of scientific conferences based on bibliographic citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 133-155, April.
    • Handle: RePEc:spr:scient:v:83:y:2010:i:1:d:10.1007_s11192-009-0078-y
    DOI: 10.1007/s11192-009-0078-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-009-0078-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-009-0078-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Tibor Braun & Wolfgang Glänzel & András Schubert, 2006. "A Hirsch-type index for journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 169-173, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Danielle H. Lee, 2019. "Predictive power of conference-related factors on citation rates of conference papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 281-304, January.
    2. Peter Ingwersen & Birger Larsen, 2014. "Influence of a performance indicator on Danish research production and citation impact 2000–12," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1325-1344, November.
    3. Xiancheng Li & Wenge Rong & Haoran Shi & Jie Tang & Zhang Xiong, 2018. "The impact of conference ranking systems in computer science: a comparative regression analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(2), pages 879-907, August.
    4. Željko Stević & Irena Đalić & Dragan Pamučar & Zdravko Nunić & Slavko Vesković & Marko Vasiljević & Ilija Tanackov, 2019. "A new hybrid model for quality assessment of scientific conferences based on Rough BWM and SERVQUAL," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 1-30, April.
    5. Peep Küngas & Siim Karus & Svitlana Vakulenko & Marlon Dumas & Cristhian Parra & Fabio Casati, 2013. "Reverse-engineering conference rankings: what does it take to make a reputable conference?," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 651-665, August.
    6. D. R. Amancio & M. G. V. Nunes & O. N. Oliveira & L. F. Costa, 2012. "Using complex networks concepts to assess approaches for citations in scientific papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 827-842, June.
    7. Michael Eckmann & Anderson Rocha & Jacques Wainer, 2012. "Relationship between high-quality journals and conferences in computer vision," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 617-630, February.
    8. Vinicius da Silva Almendra & Denis Enăchescu & Cornelia Enăchescu, 2015. "Ranking computer science conferences using self-organizing maps with dynamic node splitting," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 267-283, January.
    9. Peter Ingwersen & Birger Larsen & J. Carlos Garcia-Zorita & Antonio Eleazar Serrano-López & Elias Sanz-Casado, 2014. "Influence of proceedings papers on citation impact in seven sub-fields of sustainable energy research 2005–2011," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1273-1292, November.
    10. Meho, Lokman I., 2019. "Using Scopus’s CiteScore for assessing the quality of computer science conferences," Journal of Informetrics, Elsevier, vol. 13(1), pages 419-433.
    11. Regina Negri Pagani & Bruno Pedroso & Celso Bilynkievycz Santos & Claudia Tania Picinin & João Luiz Kovaleski, 2023. "Methodi Ordinatio 2.0: revisited under statistical estimation, and presenting FInder and RankIn," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(5), pages 4563-4602, October.
    12. Nicholas Yee Liang Hing & Xin Ci Wong & Pei Xuan Kuan & Mohan Dass Pathmanathan & Mohd Aizuddin Abdul Rahman & Kalaiarasu M. Peariasamy, 2022. "Scientific Abstract to Full Paper: Publication Rate over a 3-Year Period in a Malaysian Clinical Research Conference," Publications, MDPI, vol. 10(4), pages 1-13, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rabishankar Giri & Sabuj Kumar Chaudhuri, 2021. "Ranking journals through the lens of active visibility," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(3), pages 2189-2208, March.
    2. Deming Lin & Tianhui Gong & Wenbin Liu & Martin Meyer, 2020. "An entropy-based measure for the evolution of h index research," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2283-2298, December.
    3. Feng Li & Yong Yi & Xiaolong Guo & Wei Qi, 2012. "Performance evaluation of research universities in Mainland China, Hong Kong and Taiwan: based on a two-dimensional approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 531-542, February.
    4. Hsieh, Pao-Nuan & Chang, Pao-Long, 2009. "An assessment of world-wide research productivity in production and operations management," International Journal of Production Economics, Elsevier, vol. 120(2), pages 540-551, August.
    5. Maziar Montazerian & Edgar Dutra Zanotto & Hellmut Eckert, 2019. "A new parameter for (normalized) evaluation of H-index: countries as a case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(3), pages 1065-1078, March.
    6. Lin Zhang & Wolfgang Glänzel, 2012. "Where demographics meets scientometrics: towards a dynamic career analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 617-630, May.
    7. Jean-Francois Molinari & Alain Molinari, 2008. "A new methodology for ranking scientific institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(1), pages 163-174, April.
    8. Schmal, W. Benedikt & Haucap, Justus & Knoke, Leon, 2023. "The role of gender and coauthors in academic publication behavior," Research Policy, Elsevier, vol. 52(10).
    9. Parul Khurana & Kiran Sharma, 2022. "Impact of h-index on author’s rankings: an improvement to the h-index for lower-ranked authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4483-4498, August.
    10. Judit Bar-Ilan & Mark Levene, 2015. "The hw-rank: an h-index variant for ranking web pages," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(3), pages 2247-2253, March.
    11. Pascal Bador & Thierry Lafouge, 2010. "Comparative analysis between impact factor and h-index for pharmacology and psychiatry journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 65-79, July.
    12. Wang, Jingjing & Xu, Shuqi & Mariani, Manuel S. & Lü, Linyuan, 2021. "The local structure of citation networks uncovers expert-selected milestone papers," Journal of Informetrics, Elsevier, vol. 15(4).
    13. Judit Bar-Ilan, 2012. "Journal report card," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 249-260, August.
    14. Anqi Ma & Yu Liu & Xiujuan Xu & Tao Dong, 2021. "A deep-learning based citation count prediction model with paper metadata semantic features," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6803-6823, August.
    15. Mingkun Wei, 2020. "Research on impact evaluation of open access journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 1027-1049, February.
    16. Fiorenzo Franceschini & Domenico Maisano, 2011. "Criticism on the hg-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(2), pages 339-346, February.
    17. András Schubert & Gábor Schubert, 2018. "Whatever happened to Garfield’s constant?," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 659-667, February.
    18. M. A. Martínez & M. Herrera & J. López-Gijón & E. Herrera-Viedma, 2014. "H-Classics: characterizing the concept of citation classics through H-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(3), pages 1971-1983, March.
    19. Tingcan Ma & Gui-Fang Wang & Ke Dong & Mukun Cao, 2012. "The Journal’s Integrated Impact Index: a new indicator for journal evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 649-658, February.
    20. Xia Gao & Jiancheng Guan, 2012. "Network model of knowledge diffusion," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 749-762, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:83:y:2010:i:1:d:10.1007_s11192-009-0078-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.