IDEAS home Printed from https://ideas.repec.org/a/bla/jecsur/v19y2005i3p309-345.html
   My bibliography  Save this article

Beyond Publication Bias

Author

Listed:
  • T. D. Stanley

Abstract

. This review considers several meta‐regression and graphical methods that can differentiate genuine empirical effect from publication bias. Publication selection exists when editors, reviewers, or researchers have a preference for statistically significant results. Because all areas of empirical research are susceptible to publication selection, any average or tally of significant/insignificant studies is likely to be biased and potentially misleading. Meta‐regression analysis can see through the murk of random sampling error and selected misspecification bias to identify the underlying statistical structures that characterize genuine empirical effect. Meta‐significance testing and precision‐effect testing (PET) are offered as a means to identify empirical effect beyond publication bias and are applied to four areas of empirical economics research – minimum wage effects, union‐productivity effects, price elasticities, and tests of the natural rate hypothesis.

Suggested Citation

  • T. D. Stanley, 2005. "Beyond Publication Bias," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 309-345, July.
  • Handle: RePEc:bla:jecsur:v:19:y:2005:i:3:p:309-345
    DOI: 10.1111/j.0950-0804.2005.00250.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.0950-0804.2005.00250.x
    Download Restriction: no

    File URL: https://libkey.io/10.1111/j.0950-0804.2005.00250.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Robert J. Gordon, 1997. "The Time-Varying NAIRU and Its Implications for Economic Policy," Journal of Economic Perspectives, American Economic Association, vol. 11(1), pages 11-32, Winter.
    2. T. D. Stanley & Stephen B. Jarrell, 2005. "Meta‐Regression Analysis: A Quantitative Method of Literature Surveys," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 299-308, July.
    3. Jasper M. Dalhuisen & Raymond J. G. M. Florax & JHenri L. F. de Groot & Peter Nijkamp, 2003. "Price and Income Elasticities of Residential Water Demand: A Meta-Analysis," Land Economics, University of Wisconsin Press, vol. 79(2), pages 292-308.
    4. Stephen B. Jarrell & T. D. Stanley, 2004. "Declining Bias and Gender Wage Discrimination? A Meta-Regression Analysis," Journal of Human Resources, University of Wisconsin Press, vol. 39(3).
    5. Simonetta Longhi & Peter Nijkamp & Jacques Poot, 2005. "A Meta‐Analytic Assessment of the Effect of Immigration on Wages," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 451-477, July.
    6. Colin B. Begg & Jesse A. Berlin, 1988. "Publication Bias: A Problem in Interpreting Medical Data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 151(3), pages 419-445, May.
    7. Leamer, Edward E, 1983. "Let's Take the Con Out of Econometrics," American Economic Review, American Economic Association, vol. 73(1), pages 31-43, March.
    8. Orley Ashenfelter & Colm Harmon & Hessel Oosterbeek, 1999. "A Review of Estimates of the Schooling/Earnings Relationship, with Tests for Publication Bias," Working Papers 804, Princeton University, Department of Economics, Industrial Relations Section..
    9. Laroche, P., 2000. "What do Unions do to Productivity? A Meta-Analysis," Papers 2000-5, Groupe de recherche en économie financière et en gestion des entreprises, Universite Nancy 2.
    10. Thomas C. Leonard, 2000. "The Very Idea of Applying Economics: The Modern Minimum-Wage Controversy and Its Antecedents," History of Political Economy, Duke University Press, vol. 32(5), pages 117-144, Supplemen.
    11. De Long, J Bradford & Lang, Kevin, 1992. "Are All Economic Hypotheses False?," Journal of Political Economy, University of Chicago Press, vol. 100(6), pages 1257-1272, December.
    12. Joseph Stiglitz, 1997. "Reflections on the Natural Rate Hypothesis," Journal of Economic Perspectives, American Economic Association, vol. 11(1), pages 3-10, Winter.
    13. Holger Görg & Eric Strobl, 2016. "Multinational Companies And Productivity Spillovers: A Meta-Analysis," World Scientific Book Chapters, in: MULTINATIONAL ENTERPRISES AND HOST COUNTRY DEVELOPMENT Volume 53: World Scientific Studies in International Economics, chapter 8, pages 145-161, World Scientific Publishing Co. Pte. Ltd..
    14. repec:fth:prinin:425 is not listed on IDEAS
    15. Jakob Madsen, 1998. "The NAIRU and Classical Unemployment in the OECD Countries," International Review of Applied Economics, Taylor & Francis Journals, vol. 12(2), pages 165-185.
    16. Chris Doucouliagos, 2005. "Publication Bias in the Economic Freedom and Economic Growth Literature," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 367-387, July.
    17. Alan B. Krueger, 2003. "Economic Considerations and Class Size," Economic Journal, Royal Economic Society, vol. 113(485), pages 34-63, February.
    18. James J. Heckman, 2001. "Micro Data, Heterogeneity, and the Evaluation of Public Policy: Nobel Lecture," Journal of Political Economy, University of Chicago Press, vol. 109(4), pages 673-748, August.
    19. Simon Rottenberg, 1981. "The Economics of Legal Minimum Wages," Books, American Enterprise Institute, number 971842, September.
    20. James K. Galbraith, 1997. "Time to Ditch the NAIRU," Journal of Economic Perspectives, American Economic Association, vol. 11(1), pages 93-108, Winter.
    21. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    22. Ashenfelter, Orley & Harmon, Colm & Oosterbeek, Hessel, 1999. "A review of estimates of the schooling/earnings relationship, with tests for publication bias," Labour Economics, Elsevier, vol. 6(4), pages 453-470, November.
    23. T. D. Stanley, 2001. "Wheat from Chaff: Meta-analysis as Quantitative Literature Review," Journal of Economic Perspectives, American Economic Association, vol. 15(3), pages 131-150, Summer.
    24. Wojciech W. Charemza & Derek F. Deadman, 1997. "New Directions In Econometric Practice, Second Edition," Books, Edward Elgar Publishing, number 1139.
    25. T.D. Stanley & Stephen B. Jarrell, 1998. "Gender Wage Discrimination Bias? A Meta-Regression Analysis," Journal of Human Resources, University of Wisconsin Press, vol. 33(4), pages 947-973.
    26. T. D. Stanley, 2004. "Does unemployment hysteresis falsify the natural rate hypothesis? a meta‐regression analysis," Journal of Economic Surveys, Wiley Blackwell, vol. 18(4), pages 589-612, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stanley, T.D. & Doucouliagos, Chris & Jarrell, Stephen B., 2008. "Meta-regression analysis as the socio-economics of economics research," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 37(1), pages 276-292, February.
    2. Yannick BINEAU, 2010. "A Empirical Assessment of the Feldstein and Horioka Literature," EcoMod2010 259600030, EcoMod.
    3. Hristos Doucouliagos & T. D. Stanley, 2009. "Publication Selection Bias in Minimum‐Wage Research? A Meta‐Regression Analysis," British Journal of Industrial Relations, London School of Economics, vol. 47(2), pages 406-428, June.
    4. Doris Weichselbaumer & Rudolf Winter‐Ebmer, 2005. "A Meta‐Analysis of the International Gender Wage Gap," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 479-511, July.
    5. Neves, Pedro Cunha & Afonso, Óscar & Silva, Sandra Tavares, 2016. "A Meta-Analytic Reassessment of the Effects of Inequality on Growth," World Development, Elsevier, vol. 78(C), pages 386-400.
    6. Christopher Snyder & Ran Zhuo, 2018. "Sniff Tests as a Screen in the Publication Process: Throwing out the Wheat with the Chaff," NBER Working Papers 25058, National Bureau of Economic Research, Inc.
    7. Rosenberger, Randall S. & Stanley, Tom D., 2006. "Measurement, generalization, and publication: Sources of error in benefit transfers and their management," Ecological Economics, Elsevier, vol. 60(2), pages 372-378, December.
    8. T. D. Stanley, 2004. "Does unemployment hysteresis falsify the natural rate hypothesis? a meta‐regression analysis," Journal of Economic Surveys, Wiley Blackwell, vol. 18(4), pages 589-612, September.
    9. Hristos Doucouliagos & Janto Haman & T.D. Stanley, 2012. "Pay for Performance and Corporate Governance Reform," Industrial Relations: A Journal of Economy and Society, Wiley Blackwell, vol. 51(3), pages 670-703, July.
    10. Markus Hang & Jerome Geyer‐Klingeberg & Andreas Rathgeber & Stefan Stöckl, 2018. "Economic Development Matters: A Meta‐Regression Analysis on the Relation between Environmental Management and Financial Performance," Journal of Industrial Ecology, Yale University, vol. 22(4), pages 720-744, August.
    11. Maria Cipollina & Luca Salvatici, 2010. "Reciprocal Trade Agreements in Gravity Models: A Meta‐Analysis," Review of International Economics, Wiley Blackwell, vol. 18(1), pages 63-80, February.
    12. Alan S. Gerber & Neil Malhotra, 2008. "Publication Bias in Empirical Sociological Research," Sociological Methods & Research, , vol. 37(1), pages 3-30, August.
    13. de Linde Leonard, Megan & Stanley, T.D., 2015. "Married with children: What remains when observable biases are removed from the reported male marriage wage premium," Labour Economics, Elsevier, vol. 33(C), pages 72-80.
    14. Melo, Patricia C. & Graham, Daniel J. & Noland, Robert B., 2009. "A meta-analysis of estimates of urban agglomeration economies," Regional Science and Urban Economics, Elsevier, vol. 39(3), pages 332-342, May.
    15. Andrew K. Rose & T. D. Stanley, 2005. "A Meta‐Analysis of the Effect of Common Currencies on International Trade," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 347-365, July.
    16. Tomas Havranek, 2013. "Publication Bias in Measuring Intertemporal Substitution," Working Papers IES 2013/15, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Oct 2013.
    17. Stanley, T. D. & Doucouliagos, Hristos, 2011. "Meta-regression approximations to reduce publication selection bias," Working Papers eco_2011_4, Deakin University, Department of Economics.
    18. Chris Doucouliagos & T.D. Stanley, 2013. "Are All Economic Facts Greatly Exaggerated? Theory Competition And Selectivity," Journal of Economic Surveys, Wiley Blackwell, vol. 27(2), pages 316-339, April.
    19. Tomáš Havránek, 2009. "Rose Effect and the Euro: The Magic is Gone," Working Papers IES 2009/20, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Aug 2009.
    20. Hang, Markus & Geyer-Klingeberg, Jerome & Rathgeber, Andreas W. & Stöckl, Stefan, 2018. "Measurement matters—A meta-study of the determinants of corporate capital structure," The Quarterly Review of Economics and Finance, Elsevier, vol. 68(C), pages 211-225.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jecsur:v:19:y:2005:i:3:p:309-345. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0950-0804 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.