IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v15y2021i3s1751157721000675.html
   My bibliography  Save this article

What should be rewarded? Gender and evaluation criteria for tenure and promotion

Author

Listed:
  • Cruz-Castro, Laura
  • Sanz-Menendez, Luis

Abstract

Criteria for assessing candidates are essential elements for the functioning of evaluation practices in academia. This article addresses a relevant issue of academia: the preference for evaluation criteria for tenure and promotion, as reported by female and male academics employed at Spanish universities. We use survey data from 4,460 faculty members, testing whether there are differences in the evaluation criteria that women and men prefer and exploring the factors that account for such preferences. Our focus is on bibliometric evaluation criteria. We propose an analytical model that considers the influence of career and quality factors, values about universalism and the mission of universities, and beliefs about meritocracy in the context of the academic evaluation system. We use a binary logistic model to explain the preference for bibliometric criteria and develop the comparisons by gender using predicted probabilities and marginal effects for estimating the difference. We find that female academics do not have the same preferences as men and report lower preferences for bibliometrics. However, women at the highest research quality levels have similar probabilities than males to prefer bibliometric criteria for evaluation.

Suggested Citation

  • Cruz-Castro, Laura & Sanz-Menendez, Luis, 2021. "What should be rewarded? Gender and evaluation criteria for tenure and promotion," Journal of Informetrics, Elsevier, vol. 15(3).
  • Handle: RePEc:eee:infome:v:15:y:2021:i:3:s1751157721000675
    DOI: 10.1016/j.joi.2021.101196
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157721000675
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.joi.2021.101196?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ernesto Reuben & Matthew Wiswall & Basit Zafar, 2017. "Preferences and Biases in Educational Choices and Labour Market Expectations: Shrinking the Black Box of Gender," Economic Journal, Royal Economic Society, vol. 127(604), pages 2153-2186, September.
    2. Cathelijn J F Waaijer & Hans Sonneveld & Simone E Buitendijk & Cornelis A van Bochove & Inge C M van der Weijden, 2016. "The Role of Gender in the Employment, Career Perception and Research Performance of Recent PhD Graduates from Dutch Universities," PLOS ONE, Public Library of Science, vol. 11(10), pages 1-16, October.
    3. Stefanie Haustein & Vincent Larivière, 2015. "The Use of Bibliometrics for Assessing Research: Possibilities, Limitations and Adverse Effects," Springer Books, in: Isabell M. Welpe & Jutta Wollersheim & Stefanie Ringelhan & Margit Osterloh (ed.), Incentives and Performance, edition 127, pages 121-139, Springer.
    4. Wang, Jian & Veugelers, Reinhilde & Stephan, Paula, 2017. "Bias against novelty in science: A cautionary tale for users of bibliometric indicators," Research Policy, Elsevier, vol. 46(8), pages 1416-1436.
    5. Catherine Paradeise & Jean-Claude Thoenig, 2013. "Academic Institutions in Search of Quality: Local Orders and Global Standards," Post-Print halshs-00871625, HAL.
    6. Thelwall, Mike, 2018. "Do females create higher impact research? Scopus citations and Mendeley readers for articles from five countries," Journal of Informetrics, Elsevier, vol. 12(4), pages 1031-1041.
    7. Mansfield, Edward D. & Mutz, Diana C., 2009. "Support for Free Trade: Self-Interest, Sociotropic Politics, and Out-Group Anxiety," International Organization, Cambridge University Press, vol. 63(3), pages 425-457, July.
    8. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    9. Ginther, Donna K. & Heggeness, Misty L., 2020. "Administrative discretion in scientific funding: Evidence from a prestigious postdoctoral training program✰," Research Policy, Elsevier, vol. 49(4).
    10. Duncan A Thomas & Maria Nedeva & Mayra M Tirado & Merle Jacob, 2020. "Changing research on research evaluation: A critical literature review to revisit the agenda," Research Evaluation, Oxford University Press, vol. 29(3), pages 275-288.
    11. Richard B. Freeman & Daniel Goroff, 2009. "Science and Engineering Careers in the United States: An Analysis of Markets and Employment," NBER Books, National Bureau of Economic Research, Inc, number free09-1, March.
    12. Muriel Niederle & Lise Vesterlund, 2011. "Gender and Competition," Annual Review of Economics, Annual Reviews, vol. 3(1), pages 601-630, September.
    13. Bornmann, Lutz & Williams, Richard, 2013. "How to calculate the practical significance of citation impact differences? An empirical example from evaluative institutional bibliometrics using adjusted predictions and marginal effects," Journal of Informetrics, Elsevier, vol. 7(2), pages 562-574.
    14. James S. Fairweather, 2005. "Beyond the Rhetoric: Trends in the Relative Value of Teaching and Research in Faculty Salaries," The Journal of Higher Education, Taylor & Francis Journals, vol. 76(4), pages 401-422, July.
    15. Cruz-Castro, Laura & Sanz-Menéndez, Luis, 2010. "Mobility versus job stability: Assessing tenure and productivity outcomes," Research Policy, Elsevier, vol. 39(1), pages 27-38, February.
    16. Thomas Buser & Muriel Niederle & Hessel Oosterbeek, 2014. "Gender, Competitiveness, and Career Choices," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(3), pages 1409-1447.
    17. Carmichael, H Lorne, 1988. "Incentives in Academics: Why Is There Tenure?," Journal of Political Economy, University of Chicago Press, vol. 96(3), pages 453-472, June.
    18. Nielsen, Mathias Wullum, 2017. "Gender and citation impact in management research," Journal of Informetrics, Elsevier, vol. 11(4), pages 1213-1228.
    19. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2015. "Should the research performance of scientists be distinguished by gender?," Journal of Informetrics, Elsevier, vol. 9(1), pages 25-38.
    20. Freeman, Richard B. & Goroff, Daniel L (ed.), 2009. "Science and Engineering Careers in the United States," National Bureau of Economic Research Books, University of Chicago Press, number 9780226261898, October.
    21. Andersen, Jens Peter & Nielsen, Mathias Wullum, 2018. "Google Scholar and Web of Science: Examining gender differences in citation coverage across five scientific disciplines," Journal of Informetrics, Elsevier, vol. 12(3), pages 950-959.
    22. Tahamtan, Iman & Bornmann, Lutz, 2018. "Core elements in the process of citing publications: Conceptual overview of the literature," Journal of Informetrics, Elsevier, vol. 12(1), pages 203-216.
    23. Barbara F. Reskin & Denise D. Bielby, 2005. "A Sociological Perspective on Gender and Career Outcomes," Journal of Economic Perspectives, American Economic Association, vol. 19(1), pages 71-86, Winter.
    24. Elizabeth L. Pier & Markus Brauer & Amarette Filut & Anna Kaatz & Joshua Raclaw & Mitchell J. Nathan & Cecilia E. Ford & Molly Carnes, 2018. "Low agreement among reviewers evaluating the same NIH grant applications," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 115(12), pages 2952-2957, March.
    25. Carole J. Lee & Cassidy R. Sugimoto & Guo Zhang & Blaise Cronin, 2013. "Bias in peer review," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(1), pages 2-17, January.
    26. repec:dau:papers:123456789/12525 is not listed on IDEAS
    27. Rachel Croson & Uri Gneezy, 2009. "Gender Differences in Preferences," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 448-474, June.
    28. Heilman, Madeline E. & Manzi, Francesca & Caleo, Suzette, 2019. "Updating impressions: The differential effects of new performance information on evaluations of women and men," Organizational Behavior and Human Decision Processes, Elsevier, vol. 152(C), pages 105-121.
    29. Richard Williams, 2012. "Using the margins command to estimate and interpret adjusted predictions and marginal effects," Stata Journal, StataCorp LP, vol. 12(2), pages 308-331, June.
    30. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    31. Bornmann, Lutz & Mutz, Rüdiger & Daniel, Hans-Dieter, 2008. "How to detect indications of potential sources of bias in peer review: A generalized latent variable modeling approach exemplified by a gender study," Journal of Informetrics, Elsevier, vol. 2(4), pages 280-287.
    32. Conti, Annamaria & Visentin, Fabiana, 2015. "A revealed preference analysis of PhD students’ choices over employment outcomes," Research Policy, Elsevier, vol. 44(10), pages 1931-1947.
    33. Donna K. Ginther & Misty L. Heggeness, 2020. "Administrative Discretion in Scientific Funding: Evidence from a Prestigious Postdoctoral Training Program," NBER Working Papers 26841, National Bureau of Economic Research, Inc.
    34. Bornmann, Lutz & Mutz, Rüdiger & Daniel, Hans-Dieter, 2007. "Gender differences in grant peer review: A meta-analysis," Journal of Informetrics, Elsevier, vol. 1(3), pages 226-238.
    35. Isabel Gómez & María Bordons & M. Teresa Fernández & Fernanda Morillo, 2009. "Structure and research performance of Spanish universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(1), pages 131-146, April.
    36. Carole J. Lee & Cassidy R. Sugimoto & Guo Zhang & Blaise Cronin, 2013. "Bias in peer review," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(1), pages 2-17, January.
    37. Sarah de Rijcke & Paul F. Wouters & Alex D. Rushforth & Thomas P. Franssen & Björn Hammarfelt, 2016. "Evaluation practices and effects of indicator use—a literature review," Research Evaluation, Oxford University Press, vol. 25(2), pages 161-169.
    38. Broder, Ivy E, 1993. "Review of NSF Economics Proposals: Gender and Institutional Patterns," American Economic Review, American Economic Association, vol. 83(4), pages 964-970, September.
    39. Janet H. Lawrence & Sergio Celis & Molly Ott, 2014. "Is the Tenure Process Fair? What Faculty Think," The Journal of Higher Education, Taylor & Francis Journals, vol. 85(2), pages 155-192, March.
    40. Lutz Bornmann & Werner Marx, 2014. "How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 487-509, January.
    41. Mary Fox & Wenbin Xiao, 2013. "Perceived chances for promotion among women associate professors in computing: individual, departmental, and entrepreneurial factors," The Journal of Technology Transfer, Springer, vol. 38(2), pages 135-152, April.
    42. Donna K. Ginther & Shulamit Kahn, 2009. "Does Science Promote Women? Evidence from Academia 1973-2001," NBER Chapters, in: Science and Engineering Careers in the United States: An Analysis of Markets and Employment, pages 163-194, National Bureau of Economic Research, Inc.
    43. Salil Gunashekar & Steven Wooding & Susan Guthrie, 2017. "How do NIHR peer review panels use bibliometric information to support their decisions?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1813-1835, September.
    44. Ted I. K. Youn & Tanya M. Price, 2009. "Learning from the Experience of Others: The Evolution of Faculty Tenure and Promotion Rules in Comprehensive Institutions," The Journal of Higher Education, Taylor & Francis Journals, vol. 80(2), pages 204-237, March.
    45. Thelwall, Mike & Bailey, Carol & Tobin, Catherine & Bradshaw, Noel-Ann, 2019. "Gender differences in research areas, methods and topics: Can people and thing orientations explain the results?," Journal of Informetrics, Elsevier, vol. 13(1), pages 149-169.
    46. Xavier Bosch, 2001. "Tenure-track plan aims to end university inbreeding," Nature, Nature, vol. 410(6832), pages 1014-1014, April.
    47. Diana Hicks & Paul Wouters & Ludo Waltman & Sarah de Rijcke & Ismael Rafols, 2015. "Bibliometrics: The Leiden Manifesto for research metrics," Nature, Nature, vol. 520(7548), pages 429-431, April.
    48. Musselin, Christine, 2013. "How peer review empowers the academic profession and university managers: Changes in relationships between the state, universities and the professoriate," Research Policy, Elsevier, vol. 42(5), pages 1165-1173.
    49. Gaby Haddow & Björn Hammarfelt, 2019. "Quality, impact, and quantification: Indicators and metrics use by social scientists," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 70(1), pages 16-26, January.
    50. Isabel Fernandez-Mateo & Sarah Kaplan, 2018. "Gender and Organization Science: Introduction to a Virtual Special Issue," Organization Science, INFORMS, vol. 29(6), pages 1229-1236, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Corsi, Marcella & D’Ippoliti, Carlo & Zacchia, Giulia, 2019. "Diversity of backgrounds and ideas: The case of research evaluation in economics," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    2. Wang, Yukai & Yang, Zhongkai & Liu, Lanjian & Wang, Xianwen, 2020. "Gender bias in patenting process," Journal of Informetrics, Elsevier, vol. 14(3).
    3. Kok, Holmer & Faems, Dries & de Faria, Pedro, 2022. "Pork Barrel or Barrel of Gold? Examining the performance implications of earmarking in public R&D grants," Research Policy, Elsevier, vol. 51(7).
    4. Nunkoo, Robin & Hall, C. Michael & Rughoobur-Seetah, Soujata & Teeroovengadum, Viraiyan, 2019. "Citation practices in tourism research: Toward a gender conscientious engagement," Annals of Tourism Research, Elsevier, vol. 79(C).
    5. Eugenio Petrovich, 2022. "Bibliometrics in Press. Representations and uses of bibliometric indicators in the Italian daily newspapers," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2195-2233, May.
    6. Lutz Bornmann & Julian N. Marewski, 2019. "Heuristics as conceptual lens for understanding and studying the usage of bibliometrics in research evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 419-459, August.
    7. David A. Pendlebury, 2019. "Charting a path between the simple and the false and the complex and unusable: Review of Henk F. Moed, Applied Evaluative Informetrics [in the series Qualitative and Quantitative Analysis of Scientifi," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 549-560, April.
    8. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    9. Laura Hospido & Carlos Sanz, 2021. "Gender Gaps in the Evaluation of Research: Evidence from Submissions to Economics Conferences," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 83(3), pages 590-618, June.
    10. Bradford Demarest & Guo Freeman & Cassidy R. Sugimoto, 2014. "The reviewer in the mirror: examining gendered and ethnicized notions of reciprocity in peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 717-735, October.
    11. Thomas Buser & Noemi Peter & Stefan C. Wolter, 2022. "Willingness to compete, gender and career choices along the whole ability distribution," Experimental Economics, Springer;Economic Science Association, vol. 25(5), pages 1299-1326, November.
    12. Kuhn, Andreas & Wolter, Stefan C., 2022. "Things versus People: Gender Differences in Vocational Interests and in Occupational Preferences," Journal of Economic Behavior & Organization, Elsevier, vol. 203(C), pages 210-234.
    13. Lovász, Anna & Bat-Erdene, Boldmaa & Cukrowska-Torzewska, Ewa & Rigó, Mariann & Szabó-Morvai, Ágnes, 2023. "Competition, subjective feedback, and gender gaps in performance," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 102(C).
    14. Sven E. Hug & Mirjam Aeschbach, 2020. "Criteria for assessing grant applications: a systematic review," Palgrave Communications, Palgrave Macmillan, vol. 6(1), pages 1-15, December.
    15. Buser, Thomas & van den Assem, Martijn J. & van Dolder, Dennie, 2023. "Gender and willingness to compete for high stakes," Journal of Economic Behavior & Organization, Elsevier, vol. 206(C), pages 350-370.
    16. Nathan Barrymore & Cristian L. Dezső & Benjamin C. King, 2022. "Gender and competitiveness when earning for others: Experimental evidence and implications for sponsorship," Strategic Management Journal, Wiley Blackwell, vol. 43(5), pages 905-934, May.
    17. Laura Hospido & Carlos Sanz, 2019. "Gender gaps in the evaluation of research: evidence from submissions to economics conferences (Updated March 2020)," Working Papers 1918, Banco de España, revised Mar 2020.
    18. José de Sousa & Guillaume Hollard, 2021. "From Micro to Macro Gender Differences: Evidence from Field Tournaments," Post-Print hal-03389151, HAL.
    19. Dreber, Anna & Heikensten, Emma & Säve-Söderbergh, Jenny, 2022. "Why do women ask for less?," Labour Economics, Elsevier, vol. 78(C).
    20. Cornaglia, Francesca & Drouvelis, Michalis & Masella, Paolo, 2019. "Competition and the role of group identity," Journal of Economic Behavior & Organization, Elsevier, vol. 162(C), pages 136-145.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:15:y:2021:i:3:s1751157721000675. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.