IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Login to save this paper or follow this series

Can We Test for Bias in Scientific Peer-Review?

  • Oswald, Andrew J.

    ()

    (University of Warwick)

Science rests upon the reliability of peer review. This paper suggests a way to test for bias. It is able to avoid the fallacy – one seen in the popular press and the research literature – that to measure discrimination it is sufficient to study averages within two populations. The paper’s contribution is primarily methodological, but I apply it, as an illustration, to data from the field of economics. No scientific bias or favoritism is found (although the Journal of Political Economy discriminates against its own Chicago authors). The test’s methodology is applicable in most scholarly disciplines.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://ftp.iza.org/dp3665.pdf
Download Restriction: no

Paper provided by Institute for the Study of Labor (IZA) in its series IZA Discussion Papers with number 3665.

as
in new window

Length: 27 pages
Date of creation: Aug 2008
Date of revision:
Handle: RePEc:iza:izadps:dp3665
Contact details of provider: Postal: IZA, P.O. Box 7240, D-53072 Bonn, Germany
Phone: +49 228 3894 223
Fax: +49 228 3894 180
Web page: http://www.iza.org

Order Information: Postal: IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany
Email:


References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. C. Mirjam van Praag & Bernard M.S. van Praag, 2007. "The Benefits of Being Economics Professor A (and not Z)," Jena Economic Research Papers 2007-004, Friedrich-Schiller-University Jena, Max-Planck-Institute of Economics.
  2. COUPE, Tom & GINSBURGH, Victor & NOURY, Abdul, . "Are leading papers of better quality? Evidence from a natural experiment," CORE Discussion Papers RP -2221, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
  3. Daniel S. Hamermesh & Peter Schmidt, 2003. "The Determinants of Econometric Society Fellows Elections," Econometrica, Econometric Society, vol. 71(1), pages 399-407, January.
  4. Scott Smart & Joel Waldfogel, 1996. "A Citation-Based Test for Discrimination at Economics and Finance Journals," NBER Working Papers 5460, National Bureau of Economic Research, Inc.
  5. Glenn Ellison, 2011. "Is Peer Review In Decline?," Economic Inquiry, Western Economic Association International, vol. 49(3), pages 635-657, 07.
  6. Justin Wolfers, 2006. "Diagnosing Discrimination: Stock Returns and CEO Gender," Journal of the European Economic Association, MIT Press, vol. 4(2-3), pages 531-541, 04-05.
  7. Daniel S. Hamermesh, 2002. "International Labor Economics," Journal of Labor Economics, University of Chicago Press, vol. 20(4), pages 709-732, October.
  8. Oswald, Andrew J., 2006. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-makers," The Warwick Economics Research Paper Series (TWERPS) 744, University of Warwick, Department of Economics.
  9. C. Mirjam Van Praag & Bernard M.S. Van Praag, 2008. "The Benefits of Being Economics Professor A (rather than Z)," Economica, London School of Economics and Political Science, vol. 75(300), pages 782-796, November.
  10. Shamena Anwar & Hanming Fang, 2005. "An Alternative Test of Racial Prejudice in Motor Vehicle Searches: Theory and Evidence," NBER Working Papers 11264, National Bureau of Economic Research, Inc.
  11. Liran Einav & Leeat Yariv, 2006. "What's in a Surname? The Effects of Surname Initials on Academic Success," Journal of Economic Perspectives, American Economic Association, vol. 20(1), pages 175-187, Winter.
  12. J. Peter Neary & James A. Mirrlees & Jean Tirole, 2003. "Evaluating Economics Research in Europe: An Introduction," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1239-1249, December.
  13. Claudia Goldin & Cecilia Rouse, 1997. "Orchestrating Impartiality: The Impact of "Blind" Auditions on Female Musicians," NBER Working Papers 5903, National Bureau of Economic Research, Inc.
  14. Amanda H Goodall, 2005. "Should Research Universities be Led by Top Researchers? Part 1: Are they?," HEW 0506003, EconWPA.
  15. Laband, David N & Tollison, Robert D & Karahan, Gokhan R, 2002. "Quality Control in Economics," Kyklos, Wiley Blackwell, vol. 55(3), pages 315-34.
  16. Eric W. K. Tsang & Bruno S. Frey, 2006. "The as-is journal review process: Let authors own their ideas," IEW - Working Papers 280, Institute for Empirical Research in Economics - University of Zurich.
  17. Timothy Clark & Mike Wright, 2007. "Reviewing Journal Rankings and Revisiting Peer Reviews: Editorial Perspectives," Journal of Management Studies, Wiley Blackwell, vol. 44(4), pages 612-621, 06.
  18. Frey, Bruno S, 2003. " Publishing as Prostitution?--Choosing between One's Own Ideas and Academic Success," Public Choice, Springer, vol. 116(1-2), pages 205-23, July.
  19. JS Armstrong, 2004. "Peer Review for Journals: Evidence on Quality Control, Fairness, and Innovation," General Economics and Teaching 0412027, EconWPA.
  20. Stephen Wu, 2007. "Recent publishing trends at the AER, JPE and QJE," Applied Economics Letters, Taylor & Francis Journals, vol. 14(1), pages 59-63.
Full references (including those not matched with items on IDEAS)

This item is featured on the following reading lists or Wikipedia pages:

  1. Economic Logic blog

When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp3665. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mark Fallak)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.