IDEAS home Printed from https://ideas.repec.org/a/inm/oropre/v63y2015i6p1246-1261.html
   My bibliography  Save this article

OR Forum—Tenure Analytics: Models for Predicting Research Impact

Author

Listed:
  • Dimitris Bertsimas

    (Sloan School of Management and Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139)

  • Erik Brynjolfsson

    (Sloan School of Management, Initiative on the Digital Economy, and Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139)

  • Shachar Reichman

    (Recanati Business School, Tel Aviv University, Tel Aviv 6997801, Israel; and Sloan School of Management, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139)

  • John Silberholz

    (Sloan School of Management and Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139)

Abstract

Tenure decisions, key decisions in academic institutions, are primarily based on subjective assessments of candidates. Using a large-scale bibliometric database containing 198,310 papers published 1975–2012 in the field of operations research (OR), we propose prediction models of whether a scholar would perform well on a number of future success metrics using statistical models trained with data from the scholar’s first five years of publication, a subset of the information available to tenure committees. These models, which use network centrality of the citation network, coauthorship network, and a dual network combining the two, significantly outperform simple predictive models based on citation counts alone. Using a data set of the 54 scholars who obtained a Ph.D. after 1995 and held an assistant professorship at a top-10 OR program in 2003 or earlier, these statistical models, using data up to five years after the scholar became an assistant professor and constrained to tenure the same number of candidates as tenure committees did, made a different decision than the tenure committees for 16 (30%) of the candidates. This resulted in a set of scholars with significantly better future A-journal paper counts, citation counts, and h -indexes than the scholars actually selected by tenure committees. These results show that analytics can complement the tenure decision-making process in academia and improve the prediction of academic impact.

Suggested Citation

  • Dimitris Bertsimas & Erik Brynjolfsson & Shachar Reichman & John Silberholz, 2015. "OR Forum—Tenure Analytics: Models for Predicting Research Impact," Operations Research, INFORMS, vol. 63(6), pages 1246-1261, December.
  • Handle: RePEc:inm:oropre:v:63:y:2015:i:6:p:1246-1261
    DOI: 10.1287/opre.2015.1447
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/opre.2015.1447
    Download Restriction: no

    File URL: https://libkey.io/10.1287/opre.2015.1447?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Pierre Azoulay & Joshua S. Graff Zivin & Gustavo Manso, 2011. "Incentives and creativity: evidence from the academic life sciences," RAND Journal of Economics, RAND Corporation, vol. 42(3), pages 527-554, September.
    2. Currie, Russell R. & Pandher, Gurupdesh S., 2011. "Finance journal rankings and tiers: An Active Scholar Assessment methodology," Journal of Banking & Finance, Elsevier, vol. 35(1), pages 7-20, January.
    3. Theoharakis V. & Skordia M., 2003. "How Do Statisticians Perceive Statistics Journals?," The American Statistician, American Statistical Association, vol. 57, pages 115-123, May.
    4. Lutz Bornmann & Rüdiger Mutz & Hans‐Dieter Daniel, 2008. "Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(5), pages 830-837, March.
    5. Klaus Ritzberger, 2008. "A Ranking of Journals in Economics and Related Fields," German Economic Review, Verein für Socialpolitik, vol. 9, pages 402-430, November.
    6. Johannes Hönekopp & Julie Khan, 2012. "Future publication success in science is better predicted by traditional measures than by the h index," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 843-853, March.
    7. Daniel E. Acuna & Stefano Allesina & Konrad P. Kording, 2012. "Predicting scientific success," Nature, Nature, vol. 489(7415), pages 201-202, September.
    8. Ritzberger Klaus, 2008. "A Ranking of Journals in Economics and Related Fields," German Economic Review, De Gruyter, vol. 9(4), pages 402-430, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. James J. Heckman & Sidharth Moktan, 2020. "Publishing and promotion in economics - The tyranny of the Top Five," Vox eBook Chapters, in: Sebastian Galliani & Ugo Panizza (ed.), Publishing and Measuring Success in Economics, edition 1, volume 1, chapter 1, pages 23-32, Centre for Economic Policy Research.
    2. Matthias Kuppler, 2022. "Predicting the future impact of Computer Science researchers: Is there a gender bias?," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6695-6732, November.
    3. Zuo, Zhiya & Zhao, Kang & Ni, Chaoqun, 2019. "Standing on the shoulders of giants?—Faculty hiring in information schools," Journal of Informetrics, Elsevier, vol. 13(1), pages 341-353.
    4. Hird, Mackenzie D. & Pfotenhauer, Sebastian M., 2017. "How complex international partnerships shape domestic research clusters: Difference-in-difference network formation and research re-orientation in the MIT Portugal Program," Research Policy, Elsevier, vol. 46(3), pages 557-572.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Claudio Vitari, 2014. "Electronic currencies for purposive degrowth?," Working paper serie RMT - Grenoble Ecole de Management hal-00975432, HAL.
    2. Cristiano Varin & Manuela Cattelan & David Firth, 2016. "Statistical modelling of citation exchange between statistics journals," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(1), pages 1-63, January.
    3. Claudio Vitari, 2016. "Electronic Currencies: a literature review [Monnaies électroniques : a revue de la littérature]," Post-Print halshs-01924191, HAL.
    4. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    5. Danielle H. Lee, 2019. "Predicting the research performance of early career scientists," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(3), pages 1481-1504, December.
    6. Claudio Vitari, 2014. "Can virtual currencies drive sustainable de-growth?," Post-Print halshs-01924234, HAL.
    7. Claudio Vitari, 2014. "Can virtual currencies drive sustainable de-growth?," Grenoble Ecole de Management (Post-Print) halshs-01924234, HAL.
    8. Zhang, Lin & Thijs, Bart & Glänzel, Wolfgang, 2011. "The diffusion of H-related literature," Journal of Informetrics, Elsevier, vol. 5(4), pages 583-593.
    9. Claudio Vitari, 2016. "Electronic Currencies: a literature review [Monnaies électroniques : a revue de la littérature]," Grenoble Ecole de Management (Post-Print) halshs-01924191, HAL.
    10. Claudio Vitari, 2014. "Electronic currencies for purposive degrowth?," Working Papers hal-00975432, HAL.
    11. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    12. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2011. "An updated ranking of academic journals in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 44(4), pages 1525-1538, November.
    13. Justus Haucap & Johannes Muck, 2015. "What drives the relevance and reputation of economics journals? An update from a survey among economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 849-877, June.
    14. J. Paul Elhorst & Katarina Zigova, 2011. "Evidence of Competition in Research Activity among Economic Department using Spatial Econometric Techniques," Working Paper Series of the Department of Economics, University of Konstanz 2011-04, Department of Economics, University of Konstanz.
    15. Eduardo A. Haddad & Jesus P. Mena-Chalco, Otávio J.G. Sidone, 2016. "Produção Científica e Redes de Colaboração dos Docentes Vinculados aos Programas de Pós-graduação em Economia no Brasil," Working Papers, Department of Economics 2016_10, University of São Paulo (FEA-USP).
    16. Ho Fai Chan & Bruno S. Frey & Jana Gallus & Benno Torgler, 2013. "Does the John Bates Clark Medal boost subsequent productivity and citation success?," ECON - Working Papers 111, Department of Economics - University of Zurich.
    17. David D. Fleet & Michael W. Woolverton & James G. Beierlein & Ronald W. Cotterill, 2017. "Agribusiness: An International Journal's First 30 Volumes," Agribusiness, John Wiley & Sons, Ltd., vol. 33(1), pages 3-15, January.
    18. Damien Besancenot & Abdelghani Maddi, 2019. "Should citations be weighted to assess the influence of an academic article?," Economics Bulletin, AccessEcon, vol. 435(1), pages 435-445.
    19. Samreen Ayaz & Nayyer Masood & Muhammad Arshad Islam, 2018. "Predicting scientific impact based on h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 993-1010, March.
    20. Halkos, George & Tzeremes, Nickolaos, 2012. "Ranking agricultural, environmental and natural resource economics journals: A note," MPRA Paper 36233, University Library of Munich, Germany.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:oropre:v:63:y:2015:i:6:p:1246-1261. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.