IDEAS home Printed from https://ideas.repec.org/a/gam/jpubli/v11y2023i2p33-d1162628.html
   My bibliography  Save this article

The Evaluation Gap in Astronomy—Explained through a Rational Choice Framework

Author

Listed:
  • Julia Heuritsch

    (Research Group “Reflexive Metrics”, Institut für Sozialwissenschaften, Humboldt Universität zu Berlin, Universitätsstraße 3b, 10117 Berlin, Germany)

Abstract

The concept of evaluation gaps captures potential discrepancies between what researchers value about their research, in particular research quality, and what metrics measure. The existence of evaluation gaps can give rise to questions about the relationship between intrinsic and extrinsic motivations to perform research, i.e., how field-specific notions of quality compete with notions captured via evaluation metrics, and consequently how researchers manage the balancing act between intrinsic values and requirements of evaluation procedures. This study analyses the evaluation gap from a rational choice point of view for the case of observational astronomers, based on a literature review and 19 semi-structured interviews with international astronomers. On the basis of the institutional norms and capital at play in academic astronomy, I shed light on the workings of the balancing act and its consequences on research quality in astronomy. I find that astronomers experience an anomie: they want to follow their intrinsic motivation to pursue science in order to push knowledge forward, while at the same time following their extrinsic motivation to comply with institutional norms. The balancing act is the art of serving performance indicators in order to stay in academia, while at the same time compromising research quality as little as possible. Gaming strategies shall give the appearance of compliance, while institutionalised means to achieve a good bibliometric record are used in innovative ways, such as salami slicing or going for easy publications. This leads to an overall decrease in research quality.

Suggested Citation

  • Julia Heuritsch, 2023. "The Evaluation Gap in Astronomy—Explained through a Rational Choice Framework," Publications, MDPI, vol. 11(2), pages 1-26, June.
    • Handle: RePEc:gam:jpubli:v:11:y:2023:i:2:p:33-:d:1162628
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2304-6775/11/2/33/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2304-6775/11/2/33/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Laudel, Grit & Gläser, Jochen, 2014. "Beyond breakthrough research: Epistemic properties of research and their consequences for research funding," Research Policy, Elsevier, vol. 43(7), pages 1204-1216.
    2. Blaise Cronin, 2001. "Hyperauthorship: A postmodern perversion or evidence of a structural shift in scholarly communication practices?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 52(7), pages 558-569.
    3. Peter Dahler-Larsen, 2014. "Constitutive Effects of Performance Indicators: Getting beyond unintended consequences," Public Management Review, Taylor & Francis Journals, vol. 16(7), pages 969-986, October.
    4. Stefan Thurner & Wenyuan Liu & Peter Klimek & Siew Ann Cheong, 2020. "The role of mainstreamness and interdisciplinarity for the relevance of scientific papers," PLOS ONE, Public Library of Science, vol. 15(4), pages 1-14, April.
    5. Michael J. Kurtz & Edwin A. Henneken, 2017. "Measuring metrics - a 40-year longitudinal cross-validation of citations, downloads, and peer review in astrophysics," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(3), pages 695-708, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Julia Heuritsch, 2021. "Reflexive Behaviour: How Publication Pressure Affects Research Quality in Astronomy," Publications, MDPI, vol. 9(4), pages 1-23, November.
    2. Alberto Baccini & Giuseppe De Nicolao & Eugenio Petrovich, 2019. "Citation gaming induced by bibliometric evaluation: A country-level comparative analysis," PLOS ONE, Public Library of Science, vol. 14(9), pages 1-16, September.
    3. Nadine Desrochers & Adèle Paul‐Hus & Jen Pecoskie, 2017. "Five decades of gratitude: A meta‐synthesis of acknowledgments research," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(12), pages 2821-2833, December.
    4. Waltman, Ludo, 2012. "An empirical analysis of the use of alphabetical authorship in scientific publishing," Journal of Informetrics, Elsevier, vol. 6(4), pages 700-711.
    5. Olle Persson & Wolfgang Glänzel, 2014. "Discouraging honorific authorship," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1417-1419, February.
    6. Jo Royle & Louisa Coles & Dorothy Williams & Paul Evans, 2007. "Publishing in international journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 71(1), pages 59-86, April.
    7. Rosenzweig, Stav & Grinstein, Amir & Ofek, Elie, 2016. "Social network utilization and the impact of academic research in marketing," International Journal of Research in Marketing, Elsevier, vol. 33(4), pages 818-839.
    8. Franceschini, Fiorenzo & Maisano, Domenico, 2011. "Structured evaluation of the scientific output of academic research groups by recent h-based indicators," Journal of Informetrics, Elsevier, vol. 5(1), pages 64-74.
    9. Aniruddha Maiti & Sai Shi & Slobodan Vucetic, 2023. "An ablation study on the use of publication venue quality to rank computer science departments," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4197-4218, August.
    10. Franceschet, Massimo & Costantini, Antonio, 2010. "The effect of scholar collaboration on impact and quality of academic papers," Journal of Informetrics, Elsevier, vol. 4(4), pages 540-553.
    11. Liming Liang & Junwan Liu & Ronald Rousseau, 2004. "Name order patterns of graduate candidates and supervisors in Chinese publications: A case study of three major Chinese universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 61(1), pages 3-18, September.
    12. Chung-Souk Han, 2011. "On the demographical changes of U.S. research doctorate awardees and corresponding trends in research fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(3), pages 845-865, December.
    13. Soo Jeung Lee & Christian Schneijderberg & Yangson Kim & Isabel Steinhardt, 2021. "Have Academics’ Citation Patterns Changed in Response to the Rise of World University Rankings? A Test Using First-Citation Speeds," Sustainability, MDPI, vol. 13(17), pages 1-19, August.
    14. Shanwu Tian & Xiurui Xu & Ping Li, 2021. "Acknowledgement network and citation count: the moderating role of collaboration network," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7837-7857, September.
    15. Fontdevila, Clara, 2023. "The politics of good enough data. Developments, dilemmas and deadlocks in the production of global learning metrics," International Journal of Educational Development, Elsevier, vol. 96(C).
    16. Xuan Zhen Liu & Hui Fang, 2014. "Scientific group leaders’ authorship preferences: an empirical investigation," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 909-925, February.
    17. Perianes-Rodriguez, Antonio & Waltman, Ludo & van Eck, Nees Jan, 2016. "Constructing bibliometric networks: A comparison between full and fractional counting," Journal of Informetrics, Elsevier, vol. 10(4), pages 1178-1195.
    18. Conor O’Kane & Jing A. Zhang & Jarrod Haar & James A. Cunningham, 2023. "How scientists interpret and address funding criteria: value creation and undesirable side effects," Small Business Economics, Springer, vol. 61(2), pages 799-826, August.
    19. Arsev U. Aydinoglu & Suzie Allard & Chad Mitchell, 2016. "Measuring diversity in disciplinary collaboration in research teams: An ecological perspective," Research Evaluation, Oxford University Press, vol. 25(1), pages 18-36.
    20. Jeongeun Kim, 2018. "The Functions and Dysfunctions of College Rankings: An Analysis of Institutional Expenditure," Research in Higher Education, Springer;Association for Institutional Research, vol. 59(1), pages 54-87, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jpubli:v:11:y:2023:i:2:p:33-:d:1162628. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.