IDEAS home Printed from https://ideas.repec.org/a/aea/jeclit/v56y2018i3p920-80.html
   My bibliography  Save this article

Transparency, Reproducibility, and the Credibility of Economics Research

Author

Listed:
  • Garret Christensen
  • Edward Miguel

Abstract

There is growing interest in enhancing research transparency and reproducibility in economics and other scientific fields. We survey existing work on these topics within economics, and discuss the evidence suggesting that publication bias, inability to replicate, and specification searching remain widespread in the discipline. We next discuss recent progress in this area, including through improved research design, study registration and pre-analysis plans, disclosure standards, and open sharing of data and materials, drawing on experiences in both economics and other social sciences. We discuss areas where consensus is emerging on new practices, as well as approaches that remain controversial, and speculate about the most effective ways to make economics research more credible in the future.

Suggested Citation

  • Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
  • Handle: RePEc:aea:jeclit:v:56:y:2018:i:3:p:920-80
    Note: DOI: 10.1257/jel.20171350
    as

    Download full text from publisher

    File URL: https://www.aeaweb.org/articles?id=10.1257/jel.20171350
    Download Restriction: no

    File URL: https://www.aeaweb.org/articles/attachments?retrieve=OguLtmDSyXZBGgA4X-XQgduddHB3VRc5
    Download Restriction: Access to full text is restricted to AEA members and institutional subscribers.
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Stephen T. Ziliak & Deirdre N. McCloskey, 2004. "Size Matters: The Standard Error of Regressions in the American Economic Review," Econ Journal Watch, Econ Journal Watch, vol. 1(2), pages 331-358, August.
    2. T.D. Stanley & Hristos Doucouliagos, 2010. "Picture This: A Simple Graph That Reveals Much Ado About Research," Journal of Economic Surveys, Wiley Blackwell, vol. 24(1), pages 170-191, February.
    3. Yoav Benjamini & Abba M. Krieger & Daniel Yekutieli, 2006. "Adaptive linear step-up procedures that control the false discovery rate," Biometrika, Biometrika Trust, vol. 93(3), pages 491-507, September.
    4. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    5. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    6. Colin J. Roberts, 2005. "Issues in Meta‐Regression Analysis: An Overview," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 295-298, July.
    7. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    8. Alex Eble & Peter Boone & Diana Elbourne, 2017. "On Minimizing the Risk of Bias in Randomized Controlled Trials in Economics," World Bank Economic Review, World Bank Group, vol. 31(3), pages 687-707.
    9. Mansfield, Edwin & Schwartz, Mark & Wagner, Samuel, 1981. "Imitation Costs and Patents: An Empirical Study," Economic Journal, Royal Economic Society, vol. 91(364), pages 907-918, December.
    10. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    11. Gerber, Alan & Arceneaux, Kevin & Boudreau, Cheryl & Dowling, Conor & Hillygus, Sunshine & Palfrey, Thomas & Biggers, Daniel R. & Hendry, David J., 2014. "Reporting Guidelines for Experimental Research: A Report from the Experimental Research Section Standards Committee," Journal of Experimental Political Science, Cambridge University Press, vol. 1(1), pages 81-98, April.
    12. Lars Ivar Oppedal Berge & Kjetil Bjorvatn & Simon Galle & Edward Miguel & Daniel N. Posner & Bertil Tungodden & Kelly Zhang, 2015. "How Strong are Ethnic Preferences?," NBER Working Papers 21715, National Bureau of Economic Research, Inc.
    13. Leimer, Dean R & Lesnoy, Selig D, 1982. "Social Security and Private Saving: New Time-Series Evidence," Journal of Political Economy, University of Chicago Press, vol. 90(3), pages 606-629, June.
    14. Maria Abreu & Henri L. F. de Groot & Raymond J. G. M. Florax, 2005. "A Meta‐Analysis of β‐Convergence: the Legendary 2%," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 389-420, July.
    15. B. D. McCullough & H. D. Vinod, 2003. "Verifying the Solution from a Nonlinear Solver: A Case Study," American Economic Review, American Economic Association, vol. 93(3), pages 873-892, June.
    16. Hunt Allcott & Dmitry Taubinsky, 2015. "Evaluating Behaviorally Motivated Policy: Experimental Evidence from the Lightbulb Market," American Economic Review, American Economic Association, vol. 105(8), pages 2501-2538, August.
    17. Orley Ashenfelter & Michael Greenstone, 2004. "Estimating the Value of a Statistical Life: The Importance of Omitted Variables and Publication Bias," American Economic Review, American Economic Association, vol. 94(2), pages 454-460, May.
    18. Christian Zimmermann, 2015. "On the Need for a Replication Journal," Working Papers 2015-16, Federal Reserve Bank of St. Louis.
    19. Ori Heffetz & Katrina Ligett, 2014. "Privacy and Data-Based Research," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 75-98, Spring.
    20. W. Kip Viscusi, 2015. "The Role of Publication Selection Bias in Estimates of the Value of a Statistical Life," American Journal of Health Economics, University of Chicago Press, vol. 1(1), pages 27-52, Winter.
    21. Don Husereau & Michael Drummond & Stavros Petrou & Chris Carswell & David Moher & Dan Greenberg & Federico Augustovski & Andrew Briggs & Josephine Mauskopf & Elizabeth Loder, 2013. "Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(3), pages 367-372, June.
    22. Ryan Sullivan & Allan Timmermann & Halbert White, 1999. "Data‐Snooping, Technical Trading Rule Performance, and the Bootstrap," Journal of Finance, American Finance Association, vol. 54(5), pages 1647-1691, October.
    23. David Y. Albouy, 2012. "The Colonial Origins of Comparative Development: An Empirical Investigation: Comment," American Economic Review, American Economic Association, vol. 102(6), pages 3059-3076, October.
    24. David M. Drukker & Vince Wiggins, 2004. "Verifying the Solution from a Nonlinear Solver: A Case Study: Comment," American Economic Review, American Economic Association, vol. 94(1), pages 397-399, March.
    25. Daniel S. Hamermesh, 2007. "Viewpoint: Replication in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 40(3), pages 715-733, August.
    26. Rachel Glennerster & Kudzai Takavarasha, 2013. "Running Randomized Evaluations: A Practical Guide," Economics Books, Princeton University Press, edition 1, number 10085, October.
    27. Peter Nijkamp & Jacques Poot, 2005. "The Last Word on the Wage Curve?," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 421-450, July.
    28. Emeric Henry, 2009. "Strategic Disclosure of Research Results: The Cost of Proving Your Honesty," Economic Journal, Royal Economic Society, vol. 119(539), pages 1036-1064, July.
    29. Heléne Lundqvist & Matz Dahlberg & Eva Mörk, 2014. "Stimulating Local Public Employment: Do General Grants Work?," American Economic Journal: Economic Policy, American Economic Association, vol. 6(1), pages 167-192, February.
    30. Simonetta Longhi & Peter Nijkamp & Jacques Poot, 2005. "A Meta‐Analytic Assessment of the Effect of Immigration on Wages," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 451-477, July.
    31. Bellavance, Franois & Dionne, Georges & Lebeau, Martin, 2009. "The value of a statistical life: A meta-analysis with a mixed effects regression model," Journal of Health Economics, Elsevier, vol. 28(2), pages 444-464, March.
    32. Katherine Casey & Rachel Glennerster & Edward Miguel, 2012. "Reshaping Institutions: Evidence on Aid Impacts Using a Preanalysis Plan," The Quarterly Journal of Economics, Oxford University Press, vol. 127(4), pages 1755-1812.
    33. Denton, Frank T, 1985. "Data Mining as an Industry," The Review of Economics and Statistics, MIT Press, vol. 67(1), pages 124-127, February.
    34. Daron Acemoglu & Simon Johnson & James A. Robinson, 2012. "The Colonial Origins of Comparative Development: An Empirical Investigation: Reply," American Economic Review, American Economic Association, vol. 102(6), pages 3077-3110, October.
    35. Ron Shachar & Barry Nalebuff, 2004. "Verifying the Solution from a Nonlinear Solver: A Case Study: Comment," American Economic Review, American Economic Association, vol. 94(1), pages 382-390, March.
    36. Croke, Kevin & Hicks, Joan Hamory & Hsu, Eric & Kremer, Michael & Miguel, Edward, 2016. "Does Mass Deworming Affect Child Nutrition? Meta-analysis, Cost-Effectiveness, and Statistical Power," CEPR Discussion Papers 11458, C.E.P.R. Discussion Papers.
    37. Antonio Ciccone, 2011. "Economic Shocks and Civil Conflict: A Comment," American Economic Journal: Applied Economics, American Economic Association, vol. 3(4), pages 215-227, October.
    38. B. D. McCullough & H. D. Vinod, 2004. "Verifying the Solution from a Nonlinear Solver: A Case Study: Reply," American Economic Review, American Economic Association, vol. 94(1), pages 400-406, March.
    39. Neumark, David & Salas, J.M. Ian & Wascher, William, 2013. "Revisiting the Minimum Wage-Employment Debate: Throwing Out the Baby with the Bathwater?," IZA Discussion Papers 7166, Institute of Labor Economics (IZA).
    40. Sarojini Hirshleifer & David McKenzie & Rita Almeida & Cristobal Ridao‐Cano, 2016. "The Impact of Vocational Training for the Unemployed: Experimental Evidence from Turkey," Economic Journal, Royal Economic Society, vol. 126(597), pages 2115-2146, November.
    41. Martin Feldstein, 1982. "The Welfare Cost of Social Security's Impact on Private Saving," NBER Working Papers 0969, National Bureau of Economic Research, Inc.
    42. Leamer, Edward E & Leonard, Herman B, 1983. "Reporting the Fragility of Regression Estimates," The Review of Economics and Statistics, MIT Press, vol. 65(2), pages 306-317, May.
    43. Feldstein, Martin S, 1982. "Social Security and Private Saving: Reply," Journal of Political Economy, University of Chicago Press, vol. 90(3), pages 630-642, June.
    44. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, January.
    45. Soohyung Lee & Azeem M. Shaikh, 2014. "Multiple Testing And Heterogeneous Treatment Effects: Re‐Evaluating The Effect Of Progresa On School Enrollment," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(4), pages 612-626, June.
    46. Xavier Sala-I-Martin & Gernot Doppelhofer & Ronald I. Miller, 2004. "Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach," American Economic Review, American Economic Association, vol. 94(4), pages 813-835, September.
    47. Mitch Kunce & Shelby Gerking & William Morgan, 2002. "Effects of Environmental and Land Use Regulation in the Oil and Gas Industry Using the Wyoming Checkerboard as an Experimental Design," American Economic Review, American Economic Association, vol. 92(5), pages 1588-1593, December.
    48. Christophe Bontemps & Grayham E. Mizon, 2008. "Encompassing: Concepts and Implementation," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 70(s1), pages 721-750, December.
    49. G�nther Fink & Margaret McConnell & Sebastian Vollmer, 2014. "Testing for heterogeneous treatment effects in experimental data: false discovery risks and correction procedures," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(1), pages 44-57, January.
    50. Amy Finkelstein & Sarah Taubman & Bill Wright & Mira Bernstein & Jonathan Gruber & Joseph P. Newhouse & Heidi Allen & Katherine Baicker, 2012. "The Oregon Health Insurance Experiment: Evidence from the First Year," The Quarterly Journal of Economics, Oxford University Press, vol. 127(3), pages 1057-1106.
    51. Erik von Elm & Douglas G Altman & Matthias Egger & Stuart J Pocock & Peter C Gøtzsche & Jan P Vandenbroucke & for the STROBE Initiative, 2007. "The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies," PLOS Medicine, Public Library of Science, vol. 4(10), pages 1-5, October.
    52. B.D. McCullough & Kerry Anne McGeary & Teresa D. Harrison, 2008. "Do economics journal archives promote replicable research?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 41(4), pages 1406-1420, November.
    53. Berger, Ulrich, 2009. "The convergence of fictitious play in games with strategic complementarities: A Comment," MPRA Paper 20241, University Library of Munich, Germany.
    54. Orley Ashenfelter & Colm Harmon & Hessel Oosterbeek, 1999. "A Review of Estimates of the Schooling/Earnings Relationship, with Tests for Publication Bias," Working Papers 804, Princeton University, Department of Economics, Industrial Relations Section..
    55. Benjamin A. Olken, 2015. "Promises and Perils of Pre-analysis Plans," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 61-80, Summer.
    56. Hristos Doucouliagos & T. D. Stanley, 2009. "Publication Selection Bias in Minimum‐Wage Research? A Meta‐Regression Analysis," British Journal of Industrial Relations, London School of Economics, vol. 47(2), pages 406-428, June.
    57. Hashem Dezhbakhsh & Paul H. Rubin & Joanna M. Shepherd, 2003. "Does Capital Punishment Have a Deterrent Effect? New Evidence from Postmoratorium Panel Data," American Law and Economics Review, Oxford University Press, vol. 5(2), pages 344-376, August.
    58. Orley Ashenfelter & Michael Greenstone, 2004. "Estimating the Value of a Statistical Life: The Importance of Omitted Variables and Publication Bias," American Economic Review, American Economic Association, vol. 94(2), pages 454-460, May.
    59. John Ifcher & Homa Zarghamee, 2011. "Happiness and Time Preference: The Effect of Positive Affect in a Random-Assignment Experiment," American Economic Review, American Economic Association, vol. 101(7), pages 3109-3129, December.
    60. Ashenfelter, Orley & Harmon, Colm & Oosterbeek, Hessel, 1999. "A review of estimates of the schooling/earnings relationship, with tests for publication bias," Labour Economics, Elsevier, vol. 6(4), pages 453-470, November.
    61. Orley Ashenfelter & Michael Greenstone, 2004. "Estimating the Value of a Statistical Life: The Importance of Omitted Variables and Publication Bias," Working Papers 858, Princeton University, Department of Economics, Industrial Relations Section..
    62. Jonathan A. Schwabish, 2014. "An Economist's Guide to Visualizing Data," Journal of Economic Perspectives, American Economic Association, vol. 28(1), pages 209-234, Winter.
    63. Lucas C. Coffman & Muriel Niederle, 2015. "Pre-analysis Plans Have Limited Upside, Especially Where Replications Are Feasible," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 81-98, Summer.
    64. Andreoli-Versbach, Patrick & Mueller-Langer, Frank, 2014. "Open access to data: An ideal professed but not practised," Research Policy, Elsevier, vol. 43(9), pages 1621-1633.
    65. Jeffrey R Kling & Jeffrey B Liebman & Lawrence F Katz, 2007. "Experimental Analysis of Neighborhood Effects," Econometrica, Econometric Society, vol. 75(1), pages 83-119, January.
    66. Holger Görg & Eric Strobl, 2016. "Multinational Companies And Productivity Spillovers: A Meta-Analysis," World Scientific Book Chapters, in: MULTINATIONAL ENTERPRISES AND HOST COUNTRY DEVELOPMENT Volume 53: World Scientific Studies in International Economics, chapter 8, pages 145-161, World Scientific Publishing Co. Pte. Ltd..
    67. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    68. Irving Kirsch & Brett J Deacon & Tania B Huedo-Medina & Alan Scoboria & Thomas J Moore & Blair T Johnson, 2008. "Initial Severity and Antidepressant Benefits: A Meta-Analysis of Data Submitted to the Food and Drug Administration," PLOS Medicine, Public Library of Science, vol. 5(2), pages 1-9, February.
    69. Neumark, David & Wascher, William, 1998. "Is the Time-Series Evidence on Minimum Wage Effects Contaminated by Publication Bias?," Economic Inquiry, Western Economic Association International, vol. 36(3), pages 458-470, July.
    70. Walter Enders & Gary Hoover, 2006. "Plagiarism in the Economics Profession: A Survey," Challenge, Taylor & Francis Journals, vol. 49(5), pages 92-107.
    71. Bruce Wydick & Elizabeth Katz & Brendan Janet, 2014. "Do in-kind transfers damage local markets? The case of TOMS shoe donations in El Salvador," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(3), pages 249-267, September.
    72. Antonio Ciccone, 2013. "Estimating the Effect of Transitory Economic Shocks on Civil Conflict," Review of Economics and Institutions, Università di Perugia, vol. 4(2).
    73. Gary A. Hoover, 2004. "Whose Line Is It? Plagiarism in Economics," Journal of Economic Literature, American Economic Association, vol. 42(2), pages 487-493, June.
    74. Mizon, Grayham E & Richard, Jean-Francois, 1986. "The Encompassing Principle and Its Application to Testing Non-nested Hypotheses," Econometrica, Econometric Society, vol. 54(3), pages 657-678, May.
    75. Edward Miguel & Shanker Satyanath & Ernest Sergenti, 2004. "Economic Shocks and Civil Conflict: An Instrumental Variables Approach," Journal of Political Economy, University of Chicago Press, vol. 112(4), pages 725-753, August.
    76. Maren Duvendack & Richard Palmer-Jones, 2013. "Replication of quantitative work in development studies: Experiences and suggestions," Progress in Development Studies, , vol. 13(4), pages 307-322, October.
    77. Welch, Finis, 1974. "Minimum Wage Legislation in the United States," Economic Inquiry, Western Economic Association International, vol. 12(3), pages 285-318, September.
    78. Katherine Baicker & Amy Finkelstein & Jae Song & Sarah Taubman, 2014. "The Impact of Medicaid on Labor Market Activity and Program Participation: Evidence from the Oregon Health Insurance Experiment," American Economic Review, American Economic Association, vol. 104(5), pages 322-328, May.
    79. Roger Koenker & Achim Zeileis, 2009. "On reproducible econometric research," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 24(5), pages 833-847.
    80. Feldstein, Martin S, 1974. "Social Security, Induced Retirement, and Aggregate Capital Accumulation," Journal of Political Economy, University of Chicago Press, vol. 82(5), pages 905-926, Sept./Oct.
    81. repec:fth:prinin:425 is not listed on IDEAS
    82. Larry V. Hedges & Jack L. Vevea, 1996. "Estimating Effect Size Under Publication Bias: Small Sample Properties and Robustness of a Random Effects Selection Model," Journal of Educational and Behavioral Statistics, , vol. 21(4), pages 299-332, December.
    83. Larry A. Chenault, 1984. "A Note on the Stability Limitations in "A Stable Price Adjustment Process"," The Quarterly Journal of Economics, Oxford University Press, vol. 99(2), pages 385-386.
    84. Chris Doucouliagos & T.D. Stanley, 2013. "Are All Economic Facts Greatly Exaggerated? Theory Competition And Selectivity," Journal of Economic Surveys, Wiley Blackwell, vol. 27(2), pages 316-339, April.
    85. Arindrajit Dube & T. William Lester & Michael Reich, 2010. "Minimum Wage Effects Across State Borders: Estimates Using Contiguous Counties," The Review of Economics and Statistics, MIT Press, vol. 92(4), pages 945-964, November.
    86. Chris Doucouliagos, 2005. "Publication Bias in the Economic Freedom and Economic Growth Literature," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 367-387, July.
    87. Graham A. Davis, 2013. "Replicating Sachs and Warner's Working Papers on the Resource Curse," Journal of Development Studies, Taylor & Francis Journals, vol. 49(12), pages 1615-1630, December.
    88. Shelby Gerking & William E. Morgan, 2007. "Effects of Environmental and Land Use Regulation in the Oil and Gas Industry Using the Wyoming Checkerboard as a Natural Experiment: Retraction," American Economic Review, American Economic Association, vol. 97(3), pages 1032-1032, June.
    89. Hunter, John E, 2001. "The Desperate Need for Replications," Journal of Consumer Research, Oxford University Press, vol. 28(1), pages 149-158, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christensen, Garret & Miguel, Edward & Sturdy, Jennifer, 2017. "Transparency, Reproducibility, and the Credibility of Economics Research," MetaArXiv 9a3rw, Center for Open Science.
    2. Opoku-Agyemang, Kweku A., 2017. "A Human-Computer Interaction Approach for Integrity in Economics," SocArXiv ra3cs, Center for Open Science.
    3. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    4. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    5. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    6. Cristina Blanco-Perez & Abel Brodeur, 2020. "Publication Bias and Editorial Statement on Negative Findings," Economic Journal, Royal Economic Society, vol. 130(629), pages 1226-1247.
    7. Jarle Møen & Helge Sandvig Thorsen, 2017. "Publication Bias in the Returns to R&D Literature," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 8(3), pages 987-1013, September.
    8. Alex Eble & Peter Boone & Diana Elbourne, 2017. "On Minimizing the Risk of Bias in Randomized Controlled Trials in Economics," World Bank Economic Review, World Bank Group, vol. 31(3), pages 687-707.
    9. Burlig, Fiona, 2018. "Improving transparency in observational social science research: A pre-analysis plan approach," Economics Letters, Elsevier, vol. 168(C), pages 56-60.
    10. Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
    11. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," EconStor Open Access Articles, ZBW - Leibniz Information Centre for Economics, pages 62-83.
    12. Annette N. Brown & Drew B. Cameron & Benjamin D. K. Wood, 2014. "Quality evidence for policymaking: I'll believe it when I see the replication," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(3), pages 215-235, September.
    13. Marek Rusnak & Tomas Havranek & Roman Horvath, 2013. "How to Solve the Price Puzzle? A Meta-Analysis," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 45(1), pages 37-70, February.
    14. Maren Duvendack & Richard W. Palmer-Jones & W. Robert Reed, 2015. "Replications in Economics: A Progress Report," Econ Journal Watch, Econ Journal Watch, vol. 12(2), pages 164–191-1, May.
    15. Daniel S. Hamermesh, 2007. "Viewpoint: Replication in economics," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 40(3), pages 715-733, August.
    16. Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2018. "Methods Matter: P-Hacking and Causal Inference in Economics," IZA Discussion Papers 11796, Institute of Labor Economics (IZA).
    17. Furukawa, Chishio, 2019. "Publication Bias under Aggregation Frictions: Theory, Evidence, and a New Correction Method," EconStor Preprints 194798, ZBW - Leibniz Information Centre for Economics.
    18. Hippolyte W. Balima & Eric G. Kilama & Rene Tapsoba, 2017. "Settling the Inflation Targeting Debate: Lights from a Meta-Regression Analysis," IMF Working Papers 2017/213, International Monetary Fund.
    19. Michael A. Clemens, 2017. "The Meaning Of Failed Replications: A Review And Proposal," Journal of Economic Surveys, Wiley Blackwell, vol. 31(1), pages 326-342, February.
    20. Christopher Snyder & Ran Zhuo, 2018. "Sniff Tests as a Screen in the Publication Process: Throwing out the Wheat with the Chaff," NBER Working Papers 25058, National Bureau of Economic Research, Inc.

    More about this item

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:jeclit:v:56:y:2018:i:3:p:920-80. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Michael P. Albert). General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.