IDEAS home Printed from https://ideas.repec.org/a/aea/jecper/v31y2017i1p231-44.html
   My bibliography  Save this article

How to Write an Effective Referee Report and Improve the Scientific Review Process

Author

Listed:
  • Jonathan B. Berk
  • Campbell R. Harvey
  • David Hirshleifer

Abstract

The review process for academic journals in economics has grown vastly more extensive over time. Journals demand more revisions, and papers have become bloated with numerous robustness checks and extensions. Even if the extra resulting revisions do on average lead to improved papers--a claim that is debatable--the cost is enormous. We argue that much of the time involved in these revisions is a waste of research effort. Another cause for concern is the level of disagreement amongst referees, a pattern that suggests a high level of arbitrariness in the review process. To identify and highlight what is going right and what is going wrong in the reviewing process, we wrote to a sample of former editors of the American Economic Review, the Journal of Political Economy, the Quarterly Journal of Economics, Econometrica, the Review of Economic Studies, and the Journal of Financial Economics, and asked them for their thoughts about what might improve the process. We found a rough consensus that referees for top journals in economics tend to make similar, correctable mistakes. The italicized quotations throughout this paper are drawn from our correspondence with these editors and our own experience. Their insights are consistent with our own experiences as editors at the Journal of Finance and the Review of Financial Studies. Our objective is to highlight these mistakes and provide a roadmap for how to avoid them.

Suggested Citation

  • Jonathan B. Berk & Campbell R. Harvey & David Hirshleifer, 2017. "How to Write an Effective Referee Report and Improve the Scientific Review Process," Journal of Economic Perspectives, American Economic Association, vol. 31(1), pages 231-244, Winter.
    • Handle: RePEc:aea:jecper:v:31:y:2017:i:1:p:231-44
    Note: DOI: 10.1257/jep.31.1.231
    as

    Download full text from publisher

    File URL: https://www.aeaweb.org/articles?id=10.1257/jep.31.1.231
    Download Restriction: no
    File URL: https://www.aeaweb.org/articles/attachments?retrieve=DQw4IdKLKTPDc_SMiD-Lc4cSG4bZtlUH
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. David Hirshleifer, 2015. "Editor's Choice Editorial: Cosmetic Surgery in the Academic Review Process," The Review of Financial Studies, Society for Financial Studies, vol. 28(3), pages 637-649.
    2. Glenn Ellison, 2002. "Evolving Standards for Academic Publishing: A q-r Theory," Journal of Political Economy, University of Chicago Press, vol. 110(5), pages 994-1034, October.
    3. David N. Laband, 1990. "Is There Value-Added from the Review Process in Economics?: Preliminary Evidence from Authors," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 105(2), pages 341-352.
    4. Daniel S. Hamermesh, 1994. "Facts and Myths about Refereeing," Journal of Economic Perspectives, American Economic Association, vol. 8(1), pages 153-163, Winter.
    5. David Card & Stefano DellaVigna, 2013. "Nine Facts about Top Journals in Economics," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 144-161, March.
    6. Szenberg, Michael, editor & Ramrattan, Lall, editor, 2014. "Secrets of Economics Editors," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262525461, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Muhammad Asali, 2019. "A tale of two tracks," Education Economics, Taylor & Francis Journals, vol. 27(3), pages 323-337, May.
    2. Hengel, E., 2017. "Publishing while Female. Are women held to higher standards? Evidence from peer review," Cambridge Working Papers in Economics 1753, Faculty of Economics, University of Cambridge.
    3. Abdul Azim Islahi, 2021. "How to Ensure Research Quality in Islamic Economics? كيف نضمن جودة البحث في الاقتصاد الإسلامي؟," Journal of King Abdulaziz University: Islamic Economics, King Abdulaziz University, Islamic Economics Institute., vol. 34(2), pages 147-156, July.
    4. Gehrig, Thomas & Stenbacka, Rune, 2021. "Journal competition and the quality of published research: Simultaneous versus sequential screening," International Journal of Industrial Organization, Elsevier, vol. 76(C).
    5. Franklin Allen & Alex Edmans, 2017. "Editorial," Review of Finance, European Finance Association, vol. 21(1), pages 1-6.
    6. Maurice Doyon & Stéphane Bergeron & Lota Tamini, 2017. "Policy relevance of applied economist: Examining sensitivity and inferences," CIRANO Working Papers 2017s-12, CIRANO.
    7. David Card & Stefano DellaVigna, 2017. "What do Editors Maximize? Evidence from Four Leading Economics Journals," NBER Working Papers 23282, National Bureau of Economic Research, Inc.
    8. Islahi, Abdul Azim, 2017. "Quality control and improvements in research of Islamic Economics," MPRA Paper 80208, University Library of Munich, Germany.
    9. Asali, Muhammad, 2018. "A Tale of Two Academic Tracks," IZA Discussion Papers 11423, Institute of Labor Economics (IZA).
    10. Berninger, Marc & Kiesel, Florian & Schiereck, Dirk & Gaar, Eduard, 2021. "Citations and the readers’ information-extracting costs of finance articles," Journal of Banking & Finance, Elsevier, vol. 131(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sascha Baghestanian & Sergey V. Popov, 2018. "On publication, refereeing and working hard," Canadian Journal of Economics, Canadian Economics Association, vol. 51(4), pages 1419-1459, November.
    2. Gehrig, Thomas & Stenbacka, Rune, 2021. "Journal competition and the quality of published research: Simultaneous versus sequential screening," International Journal of Industrial Organization, Elsevier, vol. 76(C).
    3. Ofer H. Azar, 2006. "The Academic Review Process: How Can We Make it More Efficient?," The American Economist, Sage Publications, vol. 50(1), pages 37-50, March.
    4. Azar, Ofer H., 2008. "Evolution of social norms with heterogeneous preferences: A general model and an application to the academic review process," Journal of Economic Behavior & Organization, Elsevier, vol. 65(3-4), pages 420-435, March.
    5. Glenn Ellison, 2002. "The Slowdown of the Economics Publishing Process," Journal of Political Economy, University of Chicago Press, vol. 110(5), pages 947-993, October.
    6. David Card & Stefano DellaVigna, 2017. "What do Editors Maximize? Evidence from Four Leading Economics Journals," NBER Working Papers 23282, National Bureau of Economic Research, Inc.
    7. Aboozar Hadavand & Daniel S. Hamermesh & Wesley W. Wilson, 2021. "Publishing Economics: How Slow? Why Slow? Is Slow Productive? Fixing Slow?," NBER Working Papers 29147, National Bureau of Economic Research, Inc.
    8. Giuseppe Pernagallo, 2023. "Science in the mist: A model of asymmetric information for the research market," Metroeconomica, Wiley Blackwell, vol. 74(2), pages 390-415, May.
    9. Ofer H. Azar, 2005. "The Review Process in Economics: Is It Too Fast?," Southern Economic Journal, John Wiley & Sons, vol. 72(2), pages 482-491, October.
    10. Ofer Azar, 2003. "Rejections and the Importance of First Response Times (Or: How Many Rejections Do Others Receive?)," General Economics and Teaching 0309002, University Library of Munich, Germany.
    11. Yushan Hu & Ben G. Li, 2021. "The production economics of economics production," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 30(1), pages 228-255, February.
    12. Seidl, Christian & Schmidt, Ulrich & Grösche, Peter, 2005. "The Performance of Peer Review and a Beauty Contest of Referee Processes of Economics Journals/," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 23, pages 505-551, Diciembre.
    13. Justus Haucap & Johannes Muck, 2015. "What drives the relevance and reputation of economics journals? An update from a survey among economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 849-877, June.
    14. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2020. "Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics (RM/19/029-revised-)," Research Memorandum 014, Maastricht University, Graduate School of Business and Economics (GSBE).
    15. Bruno Frey, 2005. "Problems with Publishing: Existing State and Solutions," European Journal of Law and Economics, Springer, vol. 19(2), pages 173-190, April.
    16. Besancenot, Damien & Vranceanu, Radu, 2008. "Can incentives for research harm research? A business schools' tale," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 37(3), pages 1248-1265, June.
    17. Craig W. Holden, 2017. "Do Acceptance and Publication Times Differ Across Finance Journals?," The Review of Corporate Finance Studies, Society for Financial Studies, vol. 6(1), pages 102-126.
    18. Muhammad Asali, 2019. "A tale of two tracks," Education Economics, Taylor & Francis Journals, vol. 27(3), pages 323-337, May.
    19. Hadavand, Aboozar & Hamermesh, Daniel S. & Wilson, Wesley W., 2019. "Is Scholarly Refereeing Productive (at the Margin)?," IZA Discussion Papers 12866, Institute of Labor Economics (IZA).
    20. Lorenzo Ductor & Bauke Visser, 2023. "Concentration of power at the editorial boards of economics journals," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 189-238, April.

    More about this item

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:jecper:v:31:y:2017:i:1:p:231-44. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Michael P. Albert (email available below). General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.