Advanced Search
MyIDEAS: Login

Can We Test for Bias in Scientific Peer-Review?

Contents:

Author Info

  • Oswald, Andrew J.

    ()
    (University of Warwick)

Abstract

Science rests upon the reliability of peer review. This paper suggests a way to test for bias. It is able to avoid the fallacy – one seen in the popular press and the research literature – that to measure discrimination it is sufficient to study averages within two populations. The paper’s contribution is primarily methodological, but I apply it, as an illustration, to data from the field of economics. No scientific bias or favoritism is found (although the Journal of Political Economy discriminates against its own Chicago authors). The test’s methodology is applicable in most scholarly disciplines.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://ftp.iza.org/dp3665.pdf
Download Restriction: no

Bibliographic Info

Paper provided by Institute for the Study of Labor (IZA) in its series IZA Discussion Papers with number 3665.

as in new window
Length: 27 pages
Date of creation: Aug 2008
Date of revision:
Handle: RePEc:iza:izadps:dp3665

Contact details of provider:
Postal: IZA, P.O. Box 7240, D-53072 Bonn, Germany
Phone: +49 228 3894 223
Fax: +49 228 3894 180
Web page: http://www.iza.org

Order Information:
Postal: IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany
Email:

Related research

Keywords: discrimination; citations; science; peer-review system;

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. JS Armstrong, 2004. "Peer Review for Journals: Evidence on Quality Control, Fairness, and Innovation," General Economics and Teaching 0412027, EconWPA.
  2. Eric W. K. Tsang & Bruno S. Frey, 2006. "The as-is journal review process: Let authors own their ideas," IEW - Working Papers 280, Institute for Empirical Research in Economics - University of Zurich.
  3. Shamena Anwar & Hanming Fang, 2006. "An Alternative Test of Racial Prejudice in Motor Vehicle Searches: Theory and Evidence," American Economic Review, American Economic Association, vol. 96(1), pages 127-151, March.
  4. van Praag, Mirjam & van Praag, Bernard M. S., 2007. "The Benefits of Being Economics Professor A (and not Z)," IZA Discussion Papers 2673, Institute for the Study of Labor (IZA).
  5. Frey, Bruno S, 2003. " Publishing as Prostitution?--Choosing between One's Own Ideas and Academic Success," Public Choice, Springer, vol. 116(1-2), pages 205-23, July.
  6. Laband, David N & Tollison, Robert D & Karahan, Gokhan R, 2002. "Quality Control in Economics," Kyklos, Wiley Blackwell, vol. 55(3), pages 315-34.
  7. Daniel S. Hamermesh & Peter Schmidt, 2003. "The Determinants of Econometric Society Fellows Elections," Econometrica, Econometric Society, vol. 71(1), pages 399-407, January.
  8. Oswald, Andrew J., 2006. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-makers," The Warwick Economics Research Paper Series (TWERPS) 744, University of Warwick, Department of Economics.
  9. Tom Coupé & Victor Ginsburgh & Abdul Noury, 2010. "Are leading papers of better quality? Evidence from a natural experiment," Oxford Economic Papers, Oxford University Press, vol. 62(1), pages 1-11, January.
  10. Amanda H Goodall, 2005. "Should Research Universities be Led by Top Researchers? Part 1: Are they?," HEW 0506003, EconWPA.
  11. Wolfers, Justin, 2006. "Diagnosing Discrimination: Stock Returns and CEO Gender," IZA Discussion Papers 1944, Institute for the Study of Labor (IZA).
  12. Daniel S. Hamermesh, 2002. "International Labor Economics," Journal of Labor Economics, University of Chicago Press, vol. 20(4), pages 709-732, October.
  13. Glenn Ellison, 2007. "Is Peer Review in Decline?," NBER Working Papers 13272, National Bureau of Economic Research, Inc.
  14. Liran Einav & Leeat Yariv, 2006. "What's in a Surname? The Effects of Surname Initials on Academic Success," Journal of Economic Perspectives, American Economic Association, vol. 20(1), pages 175-187, Winter.
  15. Claudia Goldin & Cecilia Rouse, 1997. "Orchestrating Impartiality: The Impact of "Blind" Auditions on Female Musicians," NBER Working Papers 5903, National Bureau of Economic Research, Inc.
  16. Stephen Wu, 2007. "Recent publishing trends at the AER, JPE and QJE," Applied Economics Letters, Taylor & Francis Journals, vol. 14(1), pages 59-63.
  17. J. Peter Neary & James A. Mirrlees & Jean Tirole, 2003. "Evaluating Economics Research in Europe: An Introduction," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1239-1249, December.
  18. Scott Smart & Joel Waldfogel, 1996. "A Citation-Based Test for Discrimination at Economics and Finance Journals," NBER Working Papers 5460, National Bureau of Economic Research, Inc.
  19. C. Mirjam Van Praag & Bernard M.S. Van Praag, 2008. "The Benefits of Being Economics Professor A (rather than Z)," Economica, London School of Economics and Political Science, vol. 75(300), pages 782-796, November.
  20. Timothy Clark & Mike Wright, 2007. "Reviewing Journal Rankings and Revisiting Peer Reviews: Editorial Perspectives," Journal of Management Studies, Wiley Blackwell, vol. 44(4), pages 612-621, 06.
Full references (including those not matched with items on IDEAS)

Citations

Blog mentions

As found by EconAcademics.org, the blog aggregator for Economics research:
  1. A poor test for scientific bias in peer review
    by Economic Logician in Economic Logic on 2009-03-25 14:30:00
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Ana Rute Cardoso & Paulo Guimarães & Klaus F. Zimmermann, 2008. "Comparing the Early Research Performance of PhD Graduates in Labor Economics in Europe and the USA," Discussion Papers of DIW Berlin 850, DIW Berlin, German Institute for Economic Research.
  2. Brogaard, Jonathan & Engelberg, Joseph & Parsons, Christopher A., 2014. "Networks and productivity: Causal evidence from editor rotations," Journal of Financial Economics, Elsevier, vol. 111(1), pages 251-270.

Lists

This item is featured on the following reading lists or Wikipedia pages:
  1. Economic Logic blog

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp3665. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mark Fallak).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.