IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v107y2016i1d10.1007_s11192-016-1842-4.html
   My bibliography  Save this article

On the uncertainty of interdisciplinarity measurements due to incomplete bibliographic data

Author

Listed:
  • María del Carmen Calatrava Moreno

    (Vienna University of Technology)

  • Thomas Auzinger

    (Vienna University of Technology
    IST Austria)

  • Hannes Werthner

    (Vienna University of Technology)

Abstract

The accuracy of interdisciplinarity measurements is directly related to the quality of the underlying bibliographic data. Existing indicators of interdisciplinarity are not capable of reflecting the inaccuracies introduced by incorrect and incomplete records because correct and complete bibliographic data can rarely be obtained. This is the case for the Rao–Stirling index, which cannot handle references that are not categorized into disciplinary fields. We introduce a method that addresses this problem. It extends the Rao–Stirling index to acknowledge missing data by calculating its interval of uncertainty using computational optimization. The evaluation of our method indicates that the uncertainty interval is not only useful for estimating the inaccuracy of interdisciplinarity measurements, but it also delivers slightly more accurate aggregated interdisciplinarity measurements than the Rao–Stirling index.

Suggested Citation

  • María del Carmen Calatrava Moreno & Thomas Auzinger & Hannes Werthner, 2016. "On the uncertainty of interdisciplinarity measurements due to incomplete bibliographic data," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(1), pages 213-232, April.
    • Handle: RePEc:spr:scient:v:107:y:2016:i:1:d:10.1007_s11192-016-1842-4
    DOI: 10.1007/s11192-016-1842-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-016-1842-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-016-1842-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Stephen J. Bensman & Loet Leydesdorff, 2009. "Definition and identification of journals as bibliographic and subject entities: Librarianship versus ISI Journal Citation Reports methods and their effect on citation measures," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(6), pages 1097-1117, June.
    2. Andy Stirling, 2007. "A General Framework for Analysing Diversity in Science, Technology and Society," SPRU Working Paper Series 156, SPRU - Science Policy Research Unit, University of Sussex Business School.
    3. Alexander I. Pudovkin & Eugene Garfield, 2002. "Algorithmic procedure for finding semantically related journals," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 53(13), pages 1113-1119, November.
    4. Loet Leydesdorff & Ismael Rafols & Chaomei Chen, 2013. "Interactive overlays of journals and the measurement of interdisciplinarity on the basis of aggregated journal–journal citations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(12), pages 2573-2586, December.
    5. Kevin W. Boyack & Richard Klavans, 2010. "Co‐citation analysis, bibliographic coupling, and direct citation: Which citation approach represents the research front most accurately?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(12), pages 2389-2404, December.
    6. Chaomei Chen & Fidelia Ibekwe-SanJuan & Jianhua Hou, 2010. "The structure and dynamics of cocitation clusters: A multiple-perspective cocitation analysis," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(7), pages 1386-1409, July.
    7. Ismael Rafols & Loet Leydesdorff, 2009. "Content‐based and algorithmic classifications of journals: Perspectives on the dynamics of scientific communication and indexer effects," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(9), pages 1823-1835, September.
    8. Wagner, Caroline S. & Roessner, J. David & Bobb, Kamau & Klein, Julie Thompson & Boyack, Kevin W. & Keyton, Joann & Rafols, Ismael & Börner, Katy, 2011. "Approaches to understanding and measuring interdisciplinary scientific research (IDR): A review of the literature," Journal of Informetrics, Elsevier, vol. 5(1), pages 14-26.
    9. Stephen A. Rhoades, 1993. "The Herfindahl-Hirschman index," Federal Reserve Bulletin, Board of Governors of the Federal Reserve System (U.S.), issue Mar, pages 188-189.
    10. Alan L. Porter & Alex S. Cohen & J. David Roessner & Marty Perreault, 2007. "Measuring researcher interdisciplinarity," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(1), pages 117-147, July.
    11. Kevin W. Boyack & Richard Klavans, 2010. "Co-citation analysis, bibliographic coupling, and direct citation: Which citation approach represents the research front most accurately?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(12), pages 2389-2404, December.
    12. Loet Leydesdorff & Stephen Carley & Ismael Rafols, 2013. "Global maps of science based on the new Web-of-Science categories," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 589-593, February.
    13. Andreas Bjurström & Merritt Polk, 2011. "Climate change and interdisciplinarity: a co-citation analysis of IPCC Third Assessment Report," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 525-550, June.
    14. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    15. Alan L. Porter & Ismael Rafols, 2009. "Is science becoming more interdisciplinary? Measuring and mapping six research fields over time," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 719-745, December.
    16. Loet Leydesdorff, 2007. "Betweenness centrality as an indicator of the interdisciplinarity of scientific journals," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(9), pages 1303-1319, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gordon Rogers & Martin Szomszor & Jonathan Adams, 2020. "Sample size in bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 777-794, October.
    2. Ugo Moschini & Elena Fenialdi & Cinzia Daraio & Giancarlo Ruocco & Elisa Molinari, 2020. "A comparison of three multidisciplinarity indices based on the diversity of Scopus subject areas of authors’ documents, their bibliography and their citing papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1145-1158, November.
    3. Juste Raimbault, 2019. "Exploration of an interdisciplinary scientific landscape," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 617-641, May.
    4. R. Fileto Maciel & P. Saskia Bayerl & Marta Macedo Kerr Pinheiro, 2019. "Technical research innovations of the US national security system," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 539-565, August.
    5. Kevin M. Kniffin & Andrew S. Hanks, 2017. "Antecedents and near-term consequences for interdisciplinary dissertators," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1225-1250, June.
    6. Roland Barthel & Roman Seidl, 2017. "Interdisciplinary Collaboration between Natural and Social Sciences – Status and Trends Exemplified in Groundwater Research," PLOS ONE, Public Library of Science, vol. 12(1), pages 1-27, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ugo Moschini & Elena Fenialdi & Cinzia Daraio & Giancarlo Ruocco & Elisa Molinari, 2020. "A comparison of three multidisciplinarity indices based on the diversity of Scopus subject areas of authors’ documents, their bibliography and their citing papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1145-1158, November.
    2. Ricardo Arencibia-Jorge & Rosa Lidia Vega-Almeida & José Luis Jiménez-Andrade & Humberto Carrillo-Calvet, 2022. "Evolutionary stages and multidisciplinary nature of artificial intelligence research," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(9), pages 5139-5158, September.
    3. Ismael Rafols & Alan Porter & Loet Leydesdorff, 2009. "Overlay Maps of Science: a New Tool for Research Policy," SPRU Working Paper Series 179, SPRU - Science Policy Research Unit, University of Sussex Business School.
    4. Jorge Mañana Rodríguez, 2017. "Disciplinarity and interdisciplinarity in citation and reference dimensions: knowledge importation and exportation taxonomy of journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 617-642, February.
    5. Loet Leydesdorff & Caroline S. Wagner & Lutz Bornmann, 2018. "Betweenness and diversity in journal citation networks as measures of interdisciplinarity—A tribute to Eugene Garfield," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 567-592, February.
    6. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    7. Sándor Soós & Zsófia Vida & András Schubert, 2018. "Long-term trends in the multidisciplinarity of some typical natural and social sciences, and its implications on the SSH versus STM distinction," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 795-822, March.
    8. Lorenzo Cassi & Wilfriedo Mescheba & Élisabeth Turckheim, 2014. "How to evaluate the degree of interdisciplinarity of an institution?," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(3), pages 1871-1895, December.
    9. Shiji Chen & Yanhui Song & Fei Shu & Vincent Larivière, 2022. "Interdisciplinarity and impact: the effects of the citation time window," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2621-2642, May.
    10. Wolfram, Dietmar & Zhao, Yuehua, 2014. "A comparison of journal similarity across six disciplines using citing discipline analysis," Journal of Informetrics, Elsevier, vol. 8(4), pages 840-853.
    11. Alfonso Ávila-Robinson & Cristian Mejia & Shintaro Sengoku, 2021. "Are bibliometric measures consistent with scientists’ perceptions? The case of interdisciplinarity in research," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7477-7502, September.
    12. Xuefeng Wang & Zhinan Wang & Ying Huang & Yun Chen & Yi Zhang & Huichao Ren & Rongrong Li & Jinhui Pang, 2017. "Measuring interdisciplinarity of a research system: detecting distinction between publication categories and citation categories," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 2023-2039, June.
    13. Leydesdorff, Loet & Bornmann, Lutz & Zhou, Ping, 2016. "Construction of a pragmatic base line for journal classifications and maps based on aggregated journal-journal citation relations," Journal of Informetrics, Elsevier, vol. 10(4), pages 902-918.
    14. Carlos Olmeda-Gómez & Carlos Romá-Mateo & Maria-Antonia Ovalle-Perandones, 2019. "Overview of trends in global epigenetic research (2009–2017)," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1545-1574, June.
    15. Wang, Qi & Waltman, Ludo, 2016. "Large-scale analysis of the accuracy of the journal classification systems of Web of Science and Scopus," Journal of Informetrics, Elsevier, vol. 10(2), pages 347-364.
    16. Muh-Chyun Tang & Yun Jen Cheng & Kuang Hua Chen, 2017. "A longitudinal study of intellectual cohesion in digital humanities using bibliometric analyses," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 985-1008, November.
    17. Sander Zwanenburg & Maryam Nakhoda & Peter Whigham, 2022. "Toward greater consistency and validity in measuring interdisciplinarity: a systematic and conceptual evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7769-7788, December.
    18. Stephen Carley & Alan L. Porter, 2012. "A forward diversity index," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 407-427, February.
    19. Jon Garner & Alan L. Porter & Nils C. Newman, 2014. "Distance and velocity measures: using citations to determine breadth and speed of research impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(3), pages 687-703, September.
    20. Wagner, Caroline S. & Roessner, J. David & Bobb, Kamau & Klein, Julie Thompson & Boyack, Kevin W. & Keyton, Joann & Rafols, Ismael & Börner, Katy, 2011. "Approaches to understanding and measuring interdisciplinary scientific research (IDR): A review of the literature," Journal of Informetrics, Elsevier, vol. 5(1), pages 14-26.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:107:y:2016:i:1:d:10.1007_s11192-016-1842-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.