IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v128y2023i11d10.1007_s11192-023-04826-y.html
   My bibliography  Save this article

How can policy document mentions to scholarly papers be interpreted? An analysis of the underlying mentioning process

Author

Listed:
  • Houqiang Yu

    (Sun Yat-Sen University)

  • Biegzat Murat

    (Leiden University)

  • Jiatong Li

    (Nanjing University of Science and Technology)

  • Longfei Li

    (Wuhan University)

Abstract

Policy document mentions are useful for assessing the societal impact of scholarly papers. However, how policy document mentions can be interpreted is unclear yet. In this study, content analysis was used to examine features (mentioned element, mentioning form, and mentioning location) and motivations of policy document mention to scholarly papers. 885 policy documents were sampled for analysis from the Altmetric.com database. Results reveal that: (1) The mentioned elements of policy document mentions can be divided as five categories, summarized content (26.9%) is the most frequent one. (2) We found five types of the mentioning form of policy document mentions, the major mentioning form is references (72.3%). (3) The mentioning locations in policy documents can be divided into twelve categories, expounding (47.4%) and review (22.6%) are the core mentioning locations. (4) Motivation of policy document mentions can be broken down into five major categories and seventeen minor categories, more than 30% of motivations are to support the policy argument by listing relevant work. Analysis of the mention features of the policy document mentions gives us another way to understand how it works and how policy document mentions are motivated, with these findings we can do more work to find out the relationship between scientific articles and policy documents.

Suggested Citation

  • Houqiang Yu & Biegzat Murat & Jiatong Li & Longfei Li, 2023. "How can policy document mentions to scholarly papers be interpreted? An analysis of the underlying mentioning process," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(11), pages 6247-6266, November.
  • Handle: RePEc:spr:scient:v:128:y:2023:i:11:d:10.1007_s11192-023-04826-y
    DOI: 10.1007/s11192-023-04826-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-023-04826-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-023-04826-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Ding, Ying & Liu, Xiaozhong & Guo, Chun & Cronin, Blaise, 2013. "The distribution of references across texts: Some implications for citation analysis," Journal of Informetrics, Elsevier, vol. 7(3), pages 583-592.
    2. Hadas Shema & Judit Bar-Ilan & Mike Thelwall, 2015. "How is research blogged? A content analysis approach," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(6), pages 1136-1149, June.
    3. Yu, Houqiang & Xu, Shenmeng & Xiao, Tingting & Hemminger, Brad M. & Yang, Siluo, 2017. "Global science discussed in local altmetrics: Weibo and its comparison with Twitter," Journal of Informetrics, Elsevier, vol. 11(2), pages 466-482.
    4. Robin Haunschild & Lutz Bornmann, 2017. "How many scientific papers are mentioned in policy-related documents? An empirical investigation using Web of Science and Altmetric data," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1209-1216, March.
    5. Houqiang Yu & Xueting Cao & Tingting Xiao & Zhenyi Yang, 2020. "How accurate are policy document mentions? A first look at the role of altmetrics database," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1517-1540, November.
    6. Hu, Zhigang & Chen, Chaomei & Liu, Zeyuan, 2013. "Where are citations located in the body of scientific articles? A study of the distributions of citation locations," Journal of Informetrics, Elsevier, vol. 7(4), pages 887-896.
    7. Lutz Bornmann & Robin Haunschild & Werner Marx, 2016. "Policy documents as sources for measuring societal impact: how often is climate change research mentioned in policy-related documents?," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1477-1495, December.
    8. Zhichao Fang & Rodrigo Costas, 2020. "Studying the accumulation velocity of altmetric data tracked by Altmetric.com," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1077-1101, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dorta-González, Pablo & Rodríguez-Caro, Alejandro & Dorta-González, María Isabel, 2024. "Societal and scientific impact of policy research: A large-scale empirical study of some explanatory factors using Altmetric and Overton," Journal of Informetrics, Elsevier, vol. 18(3).
    2. Houqiang Yu & Biegzat Murat & Longfei Li & Tingting Xiao, 2021. "How accurate are Twitter and Facebook altmetrics data? A comparative content analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(5), pages 4437-4463, May.
    3. Houqiang Yu & Xueting Cao & Tingting Xiao & Zhenyi Yang, 2020. "How accurate are policy document mentions? A first look at the role of altmetrics database," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1517-1540, November.
    4. Chao Lu & Ying Ding & Chengzhi Zhang, 2017. "Understanding the impact change of a highly cited article: a content-based citation analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 927-945, August.
    5. Mike Thelwall, 2021. "Measuring Societal Impacts Of Research With Altmetrics? Common Problems And Mistakes," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1302-1314, December.
    6. Bornmann, Lutz & Haunschild, Robin & Adams, Jonathan, 2019. "Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF)," Journal of Informetrics, Elsevier, vol. 13(1), pages 325-340.
    7. Kaltrina Nuredini, 2021. "Investigating Altmetric Information For The Top 1000 Journals From Handelsblatt Ranking In Economic And Business Studies," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1315-1343, December.
    8. Liyue Chen & Jielan Ding & Vincent Larivière, 2022. "Measuring the citation context of national self‐references," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(5), pages 671-686, May.
    9. Knut Blind & Alex Fenton, 2022. "Standard-relevant publications: evidence, processes and influencing factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 577-602, January.
    10. Chao Yang & Cui Huang, 2024. "Target-oriented policy diffusion analysis: a case study of China’s information technology policy," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(3), pages 1347-1376, March.
    11. Hamid R. Jamali & Majid Nabavi & Saeid Asadi, 2018. "How video articles are cited, the case of JoVE: Journal of Visualized Experiments," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1821-1839, December.
    12. John P A Ioannidis & Kevin Boyack & Paul F Wouters, 2016. "Citation Metrics: A Primer on How (Not) to Normalize," PLOS Biology, Public Library of Science, vol. 14(9), pages 1-7, September.
    13. Shengzhi Huang & Jiajia Qian & Yong Huang & Wei Lu & Yi Bu & Jinqing Yang & Qikai Cheng, 2022. "Disclosing the relationship between citation structure and future impact of a publication," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(7), pages 1025-1042, July.
    14. Jeon, Jong Hee & Jung, Jason J., 2024. "Article ranking with location-based weight in contextual citation network," Journal of Informetrics, Elsevier, vol. 18(4).
    15. CholMyong Pak & Guang Yu & Weibin Wang, 2018. "A study on the citation situation within the citing paper: citation distribution of references according to mention frequency," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 905-918, March.
    16. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    17. Cui Huang & Chao Yang & Jun Su, 2018. "Policy change analysis based on “policy target–policy instrument” patterns: a case study of China’s nuclear energy policy," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(2), pages 1081-1114, November.
    18. Weibin Wang & Zheng Wang & Tian Yu & CholMyong Pak & Guang Yu, 2020. "Research on citation mention times and contributions using a neural network," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2383-2400, December.
    19. Dorte Drongstrup & Shafaq Malik & Naif Radi Aljohani & Salem Alelyani & Iqra Safder & Saeed-Ul Hassan, 2020. "Can social media usage of scientific literature predict journal indices of AJG, SNIP and JCR? An altmetric study of economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1541-1558, November.
    20. Zhichao Fang & Rodrigo Costas & Wencan Tian & Xianwen Wang & Paul Wouters, 2020. "An extensive analysis of the presence of altmetric data for Web of Science publications across subject fields and research topics," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2519-2549, September.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:128:y:2023:i:11:d:10.1007_s11192-023-04826-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.