IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v128y2023i1d10.1007_s11192-022-04542-z.html
   My bibliography  Save this article

Evaluating the online impact of reporting guidelines for randomised trial reports and protocols: a cross-sectional web-based data analysis of CONSORT and SPIRIT initiatives

Author

Listed:
  • Enrique Orduña-Malea

    (Universitat Politècnica de València)

  • Adolfo Alonso-Arroyo

    (University of Valencia
    UISYS)

  • José-Antonio Ontalba-Ruipérez

    (Universitat Politècnica de València)

  • Ferrán Catalá-López

    (National School of Public Health, Institute of Health Carlos III
    University of Valencia/INCLIVA Health Research Institute and CIBERSAM
    Ottawa Hospital Research Institute)

Abstract

Reporting guidelines are tools to help improve the transparency, completeness, and clarity of published articles in health research. Specifically, the CONSORT (Consolidated Standards of Reporting Trials) and SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) statements provide evidence-based guidance on what to include in randomised trial articles and protocols to guarantee the efficacy of interventions. These guidelines are subsequently described and discussed in journal articles and used to produce checklists. Determining the online impact (i.e., number and type of links received) of these articles can provide insights into the dissemination of reporting guidelines in broader environments (web-at-large) than simply that of the scientific publications that cite them. To address the technical limitations of link analysis, here the Debug-Validate-Access-Find (DVAF) method is designed and implemented to measure different facets of the guidelines’ online impact. A total of 65 articles related to 38 reporting guidelines are taken as a baseline, providing 240,128 URL citations, which are then refined, analysed, and categorised using the DVAF method. A total of 15,582 links to journal articles related to the CONSORT and SPIRIT initiatives were identified. CONSORT 2010 and SPIRIT 2013 were the reporting guidelines that received most links (URL citations) from other online objects (5328 and 2190, respectively). Overall, the online impact obtained is scattered (URL citations are received by different article URL IDs, mainly from link-based DOIs), narrow (limited number of linking domain names, half of articles are linked from fewer than 29 domain names), concentrated (links come from just a few academic publishers, around 60% from publishers), non-reputed (84% of links come from dubious websites and fake domain names) and highly decayed (89% of linking domain names were not accessible at the time of the analysis). In light of these results, it is concluded that the online impact of these guidelines could be improved, and a set of recommendations are proposed to this end.

Suggested Citation

  • Enrique Orduña-Malea & Adolfo Alonso-Arroyo & José-Antonio Ontalba-Ruipérez & Ferrán Catalá-López, 2023. "Evaluating the online impact of reporting guidelines for randomised trial reports and protocols: a cross-sectional web-based data analysis of CONSORT and SPIRIT initiatives," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 407-440, January.
    • Handle: RePEc:spr:scient:v:128:y:2023:i:1:d:10.1007_s11192-022-04542-z
    DOI: 10.1007/s11192-022-04542-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-022-04542-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-022-04542-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mike Thelwall & Nabeil Maflahi, 2016. "Guideline references and academic citations as evidence of the clinical value of health research," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(4), pages 960-966, April.
    2. Enrique Orduña-Malea & Rodrigo Costas, 2021. "Link-based approach to study scientific software usage: the case of VOSviewer," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 8153-8186, September.
    3. Wallace Koehler, 1999. "An analysis of web page and web site constancy and permanence," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 50(2), pages 162-180.
    4. Franz Barjak & Mike Thelwall, 2008. "A statistical analysis of the web presences of European life sciences research teams," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(4), pages 628-643, February.
    5. Isidro F. Aguillo & José L. Ortega & Mario Fernández & Ana M. Utrilla, 2010. "Indicators for a webometric ranking of open access repositories," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(3), pages 477-486, March.
    6. Nigel Payne & Mike Thelwall, 2007. "A longitudinal study of academic webs: Growth and stabilisation," Scientometrics, Springer;Akadémiai Kiadó, vol. 71(3), pages 523-539, June.
    7. Enrique Orduna-Malea & Mike Thelwall & Kayvan Kousha, 2017. "Web citations in patents: Evidence of technological impact?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(8), pages 1967-1974, August.
    8. Mousumi Karmakar & Sumit Kumar Banshal & Vivek Kumar Singh, 2021. "A large-scale comparison of coverage and mentions captured by the two altmetric aggregators: Altmetric.com and PlumX," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(5), pages 4465-4489, May.
    9. José Luis Ortega, 2018. "Reliability and accuracy of altmetric providers: a comparison among Altmetric.com, PlumX and Crossref Event Data," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2123-2138, September.
    10. Liwen Vaughan & Mike Thelwall, 2003. "Scholarly use of the Web: What are the key inducers of links to journal Web sites?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 54(1), pages 29-38, January.
    11. Font-Julián, Cristina I & Ontalba-Ruipérez, José-Antonio & Orduña-Malea, Enrique & Thelwall, Mike, 2022. "Which types of online resource support US patent claims?," Journal of Informetrics, Elsevier, vol. 16(1).
    12. David Moher & Alessandro Liberati & Jennifer Tetzlaff & Douglas G Altman & The PRISMA Group, 2009. "Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement," PLOS Medicine, Public Library of Science, vol. 6(7), pages 1-6, July.
    13. Kenneth F Schulz & Douglas G Altman & David Moher & for the CONSORT Group, 2010. "CONSORT 2010 Statement: Updated Guidelines for Reporting Parallel Group Randomised Trials," PLOS Medicine, Public Library of Science, vol. 7(3), pages 1-7, March.
    14. Mike Thelwall & Kayvan Kousha, 2016. "Are citations from clinical trials evidence of higher impact research? An analysis of ClinicalTrials.gov," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1341-1351, November.
    15. Franz Barjak & Xuemei Li & Mike Thelwall, 2007. "Which factors explain the Web impact of scientists' personal homepages?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(2), pages 200-211, January.
    16. Mike Thelwall, 2006. "Interpreting social science link analysis research: A theoretical framework," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(1), pages 60-68, January.
    17. José-Antonio Ontalba-Ruipérez & Enrique Orduna-Malea & Adolfo Alonso-Arroyo, 2016. "Identifying institutional relationships in a geographically distributed public health system using interlinking and co-authorship methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 1167-1191, March.
    18. Martin Klein & Herbert Van de Sompel & Robert Sanderson & Harihar Shankar & Lyudmila Balakireva & Ke Zhou & Richard Tobin, 2014. "Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot," PLOS ONE, Public Library of Science, vol. 9(12), pages 1-39, December.
    19. Fatih Oguz & Wallace Koehler, 2016. "URL decay at year 20: A research note," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(2), pages 477-479, February.
    20. Mike Thelwall & Alesia Zuccala, 2008. "A university-centred European Union link analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(3), pages 407-420, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Enrique Orduña-Malea, 2021. "Dot-science top level domain: Academic websites or dumpsites?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3565-3591, April.
    2. Patrick Kenekayoro & Kevan Buckley & Mike Thelwall, 2015. "Clustering research group website homepages," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(3), pages 2023-2039, March.
    3. Benedetto Lepori & Isidro F. Aguillo & Marco Seeber, 2014. "Size of web domains and interlinking behavior of higher education institutions in Europe," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(2), pages 497-518, August.
    4. Enrique Orduña-Malea & Cristina I. Font-Julián, 2022. "Are patents linked on Twitter? A case study of Google patents," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6339-6362, November.
    5. Su Keng Tan & Wai Keung Leung & Alexander Tin Hong Tang & Roger A Zwahlen, 2017. "Effects of mandibular setback with or without maxillary advancement osteotomies on pharyngeal airways: An overview of systematic reviews," PLOS ONE, Public Library of Science, vol. 12(10), pages 1-20, October.
    6. Jovana Kuzmanovic Pficer & Slobodan Dodic & Vojkan Lazic & Goran Trajkovic & Natasa Milic & Biljana Milicic, 2017. "Occlusal stabilization splint for patients with temporomandibular disorders: Meta-analysis of short and long term effects," PLOS ONE, Public Library of Science, vol. 12(2), pages 1-21, February.
    7. Clovis Mariano Faggion Jr & Yun-Chun Wu & Moritz Scheidgen & Yu-Kang Tu, 2015. "Effect of Risk of Bias on the Effect Size of Meta-Analytic Estimates in Randomized Controlled Trials in Periodontology and Implant Dentistry," PLOS ONE, Public Library of Science, vol. 10(9), pages 1-9, September.
    8. Salvador Angosto & Jerónimo García-Fernández & Irena Valantine & Moisés Grimaldi-Puyana, 2020. "The Intention to Use Fitness and Physical Activity Apps: A Systematic Review," Sustainability, MDPI, vol. 12(16), pages 1-25, August.
    9. Libing Jiang & Shouyin Jiang & Mao Zhang & Zhongjun Zheng & Yuefeng Ma, 2014. "Albumin versus Other Fluids for Fluid Resuscitation in Patients with Sepsis: A Meta-Analysis," PLOS ONE, Public Library of Science, vol. 9(12), pages 1-21, December.
    10. Simone Belli & Carlos Gonzalo-Penela, 2020. "Science, research, and innovation infospheres in Google results of the Ibero-American countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 635-653, May.
    11. Mahesh Shumsher Rughooputh & Rui Zeng & Ying Yao, 2015. "Protein Diet Restriction Slows Chronic Kidney Disease Progression in Non-Diabetic and in Type 1 Diabetic Patients, but Not in Type 2 Diabetic Patients: A Meta-Analysis of Randomized Controlled Trials ," PLOS ONE, Public Library of Science, vol. 10(12), pages 1-17, December.
    12. Laith Hussain-Alkhateeb & Tatiana Rivera Ramírez & Axel Kroeger & Ernesto Gozzer & Silvia Runge-Ranzinger, 2021. "Early warning systems (EWSs) for chikungunya, dengue, malaria, yellow fever, and Zika outbreaks: What is the evidence? A scoping review," PLOS Neglected Tropical Diseases, Public Library of Science, vol. 15(9), pages 1-25, September.
    13. Diana Hicks & Julia Melkers & Kimberley R. Isett, 2019. "A characterization of professional media and its links to research," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 827-843, May.
    14. Shalin Lee Wan Fei & Khatijah L Abdullah, 2015. "Effect of turning vs. supine position under phototherapy on neonates with hyperbilirubinemia: a systematic review," Journal of Clinical Nursing, John Wiley & Sons, vol. 24(5-6), pages 672-682, March.
    15. Savita Bakhshi & Alison E. While, 2013. "Health Professionals’ Alcohol-Related Professional Practices and the Relationship between Their Personal Alcohol Attitudes and Behavior and Professional Practices: A Systematic Review," IJERPH, MDPI, vol. 11(1), pages 1-31, December.
    16. Finkelstein, Eric A. & Bilger, Marcel & Baid, Drishti, 2019. "Effectiveness and cost-effectiveness of incentives as a tool for prevention of non-communicable diseases: A systematic review," Social Science & Medicine, Elsevier, vol. 232(C), pages 340-350.
    17. João Carlos Belloti & Aldo Okamura & Jordana Scheeren & Flávio Faloppa & Vinícius Ynoe de Moraes, 2019. "A systematic review of the quality of distal radius systematic reviews: Methodology and reporting assessment," PLOS ONE, Public Library of Science, vol. 14(1), pages 1-12, January.
    18. Kevin Rudolf & Lea A. L. Dejonghe & Ingo Froböse & Florian Lammer & Lisa-Marie Rückel & Jessica Tetz & Andrea Schaller, 2019. "Effectiveness Studies in Health Promotion: A Review of the Methodological Quality of Studies Reporting Significant Effects on Physical Activity in Working Age Adults," IJERPH, MDPI, vol. 16(5), pages 1-17, March.
    19. Taweewat Wiangkham & Joan Duda & Sayeed Haque & Mohammad Madi & Alison Rushton, 2015. "The Effectiveness of Conservative Management for Acute Whiplash Associated Disorder (WAD) II: A Systematic Review and Meta-Analysis of Randomised Controlled Trials," PLOS ONE, Public Library of Science, vol. 10(7), pages 1-22, July.
    20. Mike Thelwall, 2012. "Journal impact evaluation: a webometric perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 429-441, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:128:y:2023:i:1:d:10.1007_s11192-022-04542-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.