Advanced Search
MyIDEAS: Login to save this paper or follow this series

Can We Test for Bias in Scientific Peer-Review?

Contents:

Author Info

  • Oswald, Andrew J.

    ()
    (University of Warwick)

Abstract

Science rests upon the reliability of peer review. This paper suggests a way to test for bias. It is able to avoid the fallacy – one seen in the popular press and the research literature – that to measure discrimination it is sufficient to study averages within two populations. The paper’s contribution is primarily methodological, but I apply it, as an illustration, to data from the field of economics. No scientific bias or favoritism is found (although the Journal of Political Economy discriminates against its own Chicago authors). The test’s methodology is applicable in most scholarly disciplines.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://ftp.iza.org/dp3665.pdf
Download Restriction: no

Bibliographic Info

Paper provided by Institute for the Study of Labor (IZA) in its series IZA Discussion Papers with number 3665.

as in new window
Length: 27 pages
Date of creation: Aug 2008
Date of revision:
Handle: RePEc:iza:izadps:dp3665

Contact details of provider:
Postal: IZA, P.O. Box 7240, D-53072 Bonn, Germany
Phone: +49 228 3894 223
Fax: +49 228 3894 180
Web page: http://www.iza.org

Order Information:
Postal: IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany
Email:

Related research

Keywords: discrimination; citations; science; peer-review system;

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Justin Wolfers, 2006. "Diagnosing Discrimination: Stock Returns and CEO Gender," NBER Working Papers 11989, National Bureau of Economic Research, Inc.
  2. Glenn Ellison, 2007. "Is Peer Review in Decline?," NBER Working Papers 13272, National Bureau of Economic Research, Inc.
  3. Tom Coupé & Victor Ginsburgh & Abdul Ghafar Noury, 2008. "Are Leading Papers of Better Quality? Evidence from a Natural Experiment," Working Papers ECARES 2008_014, ULB -- Universite Libre de Bruxelles.
  4. Shamena Anwar & Hanming Fang, 2005. "An Alternative Test of Racial Prejudice in Motor Vehicle Searches: Theory and Evidence," NBER Working Papers 11264, National Bureau of Economic Research, Inc.
  5. Amanda Goodall, 2005. "Should Research Universities be led by top researchers? Part 1: Are they?," CEE Discussion Papers, Centre for the Economics of Education, LSE 0051, Centre for the Economics of Education, LSE.
  6. Benno Torgler, & Sascha L. Schmidt & Bruno S. Frey, 2006. "The as-is journal review process: Let authors own their ideas," IEW - Working Papers 282, Institute for Empirical Research in Economics - University of Zurich.
  7. Daniel S. Hamermesh, 2002. "International Labor Economics," Journal of Labor Economics, University of Chicago Press, University of Chicago Press, vol. 20(4), pages 709-732, October.
  8. Frey, Bruno S, 2003. " Publishing as Prostitution?--Choosing between One's Own Ideas and Academic Success," Public Choice, Springer, vol. 116(1-2), pages 205-23, July.
  9. J. Peter Neary & James A. Mirrlees & Jean Tirole, 2003. "Evaluating Economics Research in Europe: An Introduction," Journal of the European Economic Association, MIT Press, MIT Press, vol. 1(6), pages 1239-1249, December.
  10. Cecilia Rouse & Claudia Goldin, 2000. "Orchestrating Impartiality: The Impact of "Blind" Auditions on Female Musicians," American Economic Review, American Economic Association, vol. 90(4), pages 715-741, September.
  11. Mirjam van Praag & Bernard M.S. van Praag, 2007. "The Benefits of Being Economics Professor A (and not Z)," CESifo Working Paper Series 1948, CESifo Group Munich.
  12. Stephen Wu, 2007. "Recent publishing trends at the AER, JPE and QJE," Applied Economics Letters, Taylor & Francis Journals, vol. 14(1), pages 59-63.
  13. Andrew J. Oswald, 2007. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-Makers," Economica, London School of Economics and Political Science, London School of Economics and Political Science, vol. 74(293), pages 21-31, 02.
  14. Timothy Clark & Mike Wright, 2007. "Reviewing Journal Rankings and Revisiting Peer Reviews: Editorial Perspectives," Journal of Management Studies, Wiley Blackwell, vol. 44(4), pages 612-621, 06.
  15. Laband, David N & Tollison, Robert D & Karahan, Gokhan R, 2002. "Quality Control in Economics," Kyklos, Wiley Blackwell, vol. 55(3), pages 315-34.
  16. Scott Smart & Joel Waldfogel, 1996. "A Citation-Based Test for Discrimination at Economics and Finance Journals," NBER Working Papers 5460, National Bureau of Economic Research, Inc.
  17. C. Mirjam Van Praag & Bernard M.S. Van Praag, 2008. "The Benefits of Being Economics Professor A (rather than Z)," Economica, London School of Economics and Political Science, London School of Economics and Political Science, vol. 75(300), pages 782-796, November.
  18. Liran Einav & Leeat Yariv, 2006. "What's in a Surname? The Effects of Surname Initials on Academic Success," Journal of Economic Perspectives, American Economic Association, vol. 20(1), pages 175-187, Winter.
  19. Daniel S. Hamermesh & Peter Schmidt, 2003. "The Determinants of Econometric Society Fellows Elections," Econometrica, Econometric Society, Econometric Society, vol. 71(1), pages 399-407, January.
  20. JS Armstrong, 2004. "Peer Review for Journals: Evidence on Quality Control, Fairness, and Innovation," General Economics and Teaching, EconWPA 0412027, EconWPA.
Full references (including those not matched with items on IDEAS)

Citations

Blog mentions

As found by EconAcademics.org, the blog aggregator for Economics research:
  1. A poor test for scientific bias in peer review
    by Economic Logician in Economic Logic on 2009-03-25 14:30:00
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Cardoso, Ana Rute & Guimaraes, Paulo & Zimmermann, Klaus F., 2008. "Comparing the Early Research Performance of PhD Graduates in Labor Economics in Europe and the USA," IZA Discussion Papers 3898, Institute for the Study of Labor (IZA).
  2. Brogaard, Jonathan & Engelberg, Joseph & Parsons, Christopher A., 2014. "Networks and productivity: Causal evidence from editor rotations," Journal of Financial Economics, Elsevier, vol. 111(1), pages 251-270.

Lists

This item is featured on the following reading lists or Wikipedia pages:
  1. Economic Logic blog

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp3665. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mark Fallak).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.