IDEAS home Printed from https://ideas.repec.org/a/eee/respol/v53y2024i2s0048733323002238.html

External review letters in academic promotion and tenure decisions are reflective of reviewer characteristics

Author

Listed:
  • Madera, Juan M.
  • Spitzmueller, Christiane
  • Yu, Heyao
  • Edema-Sillo, Ebenezer
  • Clarke, Mark S.F.

Abstract

We examine validity and bias in external review letters (ERLs) in academic settings. ERLs play a critical role in promotion and tenure (P&T) decisions across the globe, ending careers in some cases while allowing other scientists' careers to flourish. We coded and analyzed 995 ERLs submitted by letter writers at various institutions as part of the P&T portfolios of 195 candidates at an R1 university. We examined their relationship with P&T committees' percent of positive votes by department, college, and university committees. We investigated how ERL linguistic features, letter writer characteristics, and candidate characteristics (productivity and gender) relate to P&T decision-making. Results show writer characteristics are more strongly related than candidate characteristics to ERL linguistic features associated with P&T decisions at the department and university levels. We develop recommendations for policymakers, including changes in the use of ERLs for the P&T process.

Suggested Citation

  • Madera, Juan M. & Spitzmueller, Christiane & Yu, Heyao & Edema-Sillo, Ebenezer & Clarke, Mark S.F., 2024. "External review letters in academic promotion and tenure decisions are reflective of reviewer characteristics," Research Policy, Elsevier, vol. 53(2).
    • Handle: RePEc:eee:respol:v:53:y:2024:i:2:s0048733323002238
    DOI: 10.1016/j.respol.2023.104939
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0048733323002238
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.respol.2023.104939?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Isabel Fernandez-Mateo & Roberto M. Fernandez, 2016. "Bending the Pipeline? Executive Search and Gender Inequality in Hiring for Top Management Jobs," Management Science, INFORMS, vol. 62(12), pages 3636-3655, December.
    2. J. E. Hirsch, 2010. "An index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(3), pages 741-754, December.
    3. Judge, Timothy A. & Higgins, Chad A., 1998. "Affective Disposition and the Letter of Reference," Organizational Behavior and Human Decision Processes, Elsevier, vol. 75(3), pages 207-221, September.
    4. Stephen J. Sauer & Melissa C. Thomas-Hunt & Patrick A. Morris, 2010. "Too Good to Be True? The Unintended Signaling Effects of Educational Prestige on External Expectations of Team Performance," Organization Science, INFORMS, vol. 21(5), pages 1108-1120, October.
    5. Mike Thelwall & Kayvan Kousha, 2021. "Researchers’ attitudes towards the h-index on Twitter 2007–2020: criticism and acceptance," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 5361-5368, June.
    6. Buunk, Abraham P. & Gibbons, Frederick X., 2007. "Social comparison: The end of a theory and the emergence of a field," Organizational Behavior and Human Decision Processes, Elsevier, vol. 102(1), pages 3-21, January.
    7. Wang, Feng & Liu, Xuefeng & Fang, Eric (Er), 2015. "User Reviews Variance, Critic Reviews Variance, and Product Sales: An Exploration of Customer Breadth and Depth Effects," Journal of Retailing, Elsevier, vol. 91(3), pages 372-389.
    8. Greenberg, Jerald & Ashton-James, Claire E. & Ashkanasy, Neal M., 2007. "Social comparison processes in organizations," Organizational Behavior and Human Decision Processes, Elsevier, vol. 102(1), pages 22-41, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Theodore Masters-Waage & Christiane Spitzmueller & Ebenezer Edema-Sillo & Ally St. Aubin & Michelle Penn-Marshall & Erika Henderson & Peggy Lindner & Cynthia Werner & Tracey Rizzuto & Juan Madera, 2024. "Underrepresented minority faculty in the USA face a double standard in promotion and tenure decisions," Nature Human Behaviour, Nature, vol. 8(11), pages 2107-2118, November.
    2. Masters-Waage, Theodore & Madera, Juan & St. Aubin, Ally & Ash, Joshua & Edema-Sillo, Ebenezer & Spitzmueller, Christiane, 2025. "Women advocates and men critics: How referees' gender influences candidates' likelihood of receiving a promotion," Research Policy, Elsevier, vol. 54(5).
    3. Theodore C. Masters-Waage & Juan M. Madera & Ebenezer Edema-Sillo & Ally St. Aubin & Peggy Lindner & Maritza Gaytan & Heyao Yu & Christiane Spitzmueller, 2026. "Evaluating multiple candidates simultaneously reduces racial disparities in promotion and tenure," Nature Communications, Nature, vol. 17(1), pages 1-13, December.
    4. Mona Farouk Ali, 2025. "Investigating shifts in publication patterns after launching scientometric evaluation at Egyptian universities: an analysis of submitted research for promotion," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(3), pages 1751-1787, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang, Feifan & Chu, Rongwei & Chen, Zhijun, 2026. "Bitter or better: how social comparison emotions drive coworker altruism and ostracism in response to job crafting," Journal of Business Research, Elsevier, vol. 205(C).
    2. Matt Grimes, 2010. "Strategic Sensemaking within Funding Relationships: The Effects of Performance Measurement on Organizational Identity in the Social Sector," Entrepreneurship Theory and Practice, , vol. 34(4), pages 763-783, July.
    3. Miaomiao Li & Xiaofeng Xu & Ho Kwong Kwan, 2023. "The antecedents and consequences of workplace envy: A meta-analytic review," Asia Pacific Journal of Management, Springer, vol. 40(1), pages 1-35, March.
    4. Ken Cheng & Limin Guo & Jinlian Luo, 2023. "The more you exploit, the more expedient I will be: A moral disengagement and Chinese traditionality examination of exploitative leadership and employee expediency," Asia Pacific Journal of Management, Springer, vol. 40(1), pages 151-167, March.
    5. Breidenthal, Amy P. & Liu, Dong & Bai, Yuntao & Mao, Yina, 2020. "The dark side of creativity: Coworker envy and ostracism as a response to employee creativity," Organizational Behavior and Human Decision Processes, Elsevier, vol. 161(C), pages 242-254.
    6. Goodman, Paul S. & Haisley, Emily, 2007. "Social comparison processes in an organizational context: New directions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 102(1), pages 109-125, January.
    7. Hernández-Ortega, Blanca, 2020. "When the performance comes into play: The influence of positive online consumer reviews on individuals' post-consumption responses," Journal of Business Research, Elsevier, vol. 113(C), pages 422-435.
    8. Matthew J. Quade & Rebecca L. Greenbaum & Mary B. Mawritz, 2019. "“If Only My Coworker Was More Ethical”: When Ethical and Performance Comparisons Lead to Negative Emotions, Social Undermining, and Ostracism," Journal of Business Ethics, Springer, vol. 159(2), pages 567-586, October.
    9. Ilanit SimanTov-Nachlieli, 2025. "More to Lose: The Adverse Effect of High Performance Ranking on Employees’ Preimplementation Attitudes Toward the Integration of Powerful AI Aids," Organization Science, INFORMS, vol. 36(1), pages 1-20, January.
    10. Zi Wang & Guiquan Li, 2018. "You don’t actually want to get closer to the star: How LMX leads to workplace ostracism," Frontiers of Business Research in China, Springer, vol. 12(1), pages 1-13, December.
    11. Natalie J. Shin & Jonathan C. Ziegert & Miriam Muethel, 2022. "The Detrimental Effects of Ethical Incongruence in Teams: An Interactionist Perspective of Ethical Fit on Relationship Conflict and Information Sharing," Journal of Business Ethics, Springer, vol. 179(1), pages 259-272, August.
    12. Antonio Fernandez-Cano & Inés M. Fernández-Guerrero, 2017. "A multivariate model for evaluating emergency medicine journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 991-1003, February.
    13. Yingjin Song & Ruiyi Li & Guanyi Chen & Beibei Yan & Lei Zhong & Yuxin Wang & Yihang Li & Jinlei Li & Yingxiu Zhang, 2021. "Bibliometric Analysis of Current Status on Bioremediation of Petroleum Contaminated Soils during 2000–2019," IJERPH, MDPI, vol. 18(16), pages 1-20, August.
    14. Deming Lin & Tianhui Gong & Wenbin Liu & Martin Meyer, 2020. "An entropy-based measure for the evolution of h index research," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2283-2298, December.
    15. Oliver Fabel & Sandra Mauser & Yingchao Zhang, 2024. "Performance contests and merit pay with empathic employees," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 45(1), pages 353-372, January.
    16. Gao, Qiang & Liang, Zhentao & Wang, Ping & Hou, Jingrui & Chen, Xiuxiu & Liu, Manman, 2021. "Potential index: Revealing the future impact of research topics based on current knowledge networks," Journal of Informetrics, Elsevier, vol. 15(3).
    17. Kalyani Ganpat Midgule & Prasoom Dwivedi, 2026. "Research hotspots in agricultural finance research: a bibliometric exploration study," Future Business Journal, Springer, vol. 12(1), pages 1-19, December.
    18. Ash Mohammad Abbas, 2011. "Weighted indices for evaluating the quality of research with multiple authorship," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 107-131, July.
    19. Jiang Wu, 2013. "Geographical knowledge diffusion and spatial diversity citation rank," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(1), pages 181-201, January.
    20. Cheng-Wen Lee & Min-Ying Cheng, 2024. "The Impact of Ancient Traditional Culture on Earnings Quality: The Moderating Role of Marketization Index in China's A-Share Market," Advances in Management and Applied Economics, SCIENPRESS Ltd, vol. 14(6), pages 1-20.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:53:y:2024:i:2:s0048733323002238. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/respol .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.