IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/22989.html
   My bibliography  Save this paper

Transparency, Reproducibility, and the Credibility of Economics Research

Author

Listed:
  • Garret S. Christensen
  • Edward Miguel

Abstract

There is growing interest in enhancing research transparency and reproducibility in economics and other scientific fields. We survey existing work on these topics within economics, and discuss the evidence suggesting that publication bias, inability to replicate, and specification searching remain widespread in the discipline. We next discuss recent progress in this area, including through improved research design, study registration and pre-analysis plans, disclosure standards, and open sharing of data and materials, drawing on experiences in both economics and other social sciences. We discuss areas where consensus is emerging on new practices, as well as approaches that remain controversial, and speculate about the most effective ways to make economics research more credible in the future.

Suggested Citation

  • Garret S. Christensen & Edward Miguel, 2016. "Transparency, Reproducibility, and the Credibility of Economics Research," NBER Working Papers 22989, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:22989
    Note: DEV LS TWP
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w22989.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. T.D. Stanley & Hristos Doucouliagos, 2010. "Picture This: A Simple Graph That Reveals Much Ado About Research," Journal of Economic Surveys, Wiley Blackwell, vol. 24(1), pages 170-191, February.
    2. Yoav Benjamini & Abba M. Krieger & Daniel Yekutieli, 2006. "Adaptive linear step-up procedures that control the false discovery rate," Biometrika, Biometrika Trust, vol. 93(3), pages 491-507, September.
    3. Anderson, Michael L, 2008. "Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects," Department of Agricultural & Resource Economics, UC Berkeley, Working Paper Series qt15n8j26f, Department of Agricultural & Resource Economics, UC Berkeley.
    4. Stefano DellaVigna & Devin Pope, 2018. "Predicting Experimental Results: Who Knows What?," Journal of Political Economy, University of Chicago Press, vol. 126(6), pages 2410-2456.
    5. Alex Eble & Peter Boone & Diana Elbourne, 2017. "On Minimizing the Risk of Bias in Randomized Controlled Trials in Economics," The World Bank Economic Review, World Bank, vol. 31(3), pages 687-707.
    6. Emeric Henry, 2009. "Strategic Disclosure of Research Results: The Cost of Proving Your Honesty," Economic Journal, Royal Economic Society, vol. 119(539), pages 1036-1064, July.
    7. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    8. Hunt Allcott & Dmitry Taubinsky, 2015. "Evaluating Behaviorally Motivated Policy: Experimental Evidence from the Lightbulb Market," American Economic Review, American Economic Association, vol. 105(8), pages 2501-2538, August.
    9. David Y. Albouy, 2012. "The Colonial Origins of Comparative Development: An Empirical Investigation: Comment," American Economic Review, American Economic Association, vol. 102(6), pages 3059-3076, October.
    10. Ryan Sullivan & Allan Timmermann & Halbert White, 1999. "Data‐Snooping, Technical Trading Rule Performance, and the Bootstrap," Journal of Finance, American Finance Association, vol. 54(5), pages 1647-1691, October.
    11. Peter Nijkamp & Jacques Poot, 2005. "The Last Word on the Wage Curve?," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 421-450, July.
    12. Simonetta Longhi & Peter Nijkamp & Jacques Poot, 2005. "A Meta‐Analytic Assessment of the Effect of Immigration on Wages," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 451-477, July.
    13. Orley Ashenfelter & Michael Greenstone, 2004. "Estimating the Value of a Statistical Life: The Importance of Omitted Variables and Publication Bias," American Economic Review, American Economic Association, vol. 94(2), pages 454-460, May.
    14. Denton, Frank T, 1985. "Data Mining as an Industry," The Review of Economics and Statistics, MIT Press, vol. 67(1), pages 124-127, February.
    15. Heléne Lundqvist & Matz Dahlberg & Eva Mörk, 2014. "Stimulating Local Public Employment: Do General Grants Work?," American Economic Journal: Economic Policy, American Economic Association, vol. 6(1), pages 167-192, February.
    16. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    17. Benjamin A. Olken, 2015. "Promises and Perils of Pre-analysis Plans," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 61-80, Summer.
    18. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    19. List, John A, et al, 2001. "Academic Economists Behaving Badly? A Survey on Three Areas of Unethical Behavior," Economic Inquiry, Western Economic Association International, vol. 39(1), pages 162-170, January.
    20. Don Husereau & Michael Drummond & Stavros Petrou & Chris Carswell & David Moher & Dan Greenberg & Federico Augustovski & Andrew Briggs & Josephine Mauskopf & Elizabeth Loder, 2013. "Consolidated Health Economic Evaluation Reporting Standards (CHEERS) Statement," PharmacoEconomics, Springer, vol. 31(5), pages 361-367, May.
    21. Jeffrey R Kling & Jeffrey B Liebman & Lawrence F Katz, 2007. "Experimental Analysis of Neighborhood Effects," Econometrica, Econometric Society, vol. 75(1), pages 83-119, January.
    22. Irving Kirsch & Brett J Deacon & Tania B Huedo-Medina & Alan Scoboria & Thomas J Moore & Blair T Johnson, 2008. "Initial Severity and Antidepressant Benefits: A Meta-Analysis of Data Submitted to the Food and Drug Administration," PLOS Medicine, Public Library of Science, vol. 5(2), pages 1-9, February.
    23. Neumark, David & Wascher, William, 1998. "Is the Time-Series Evidence on Minimum Wage Effects Contaminated by Publication Bias?," Economic Inquiry, Western Economic Association International, vol. 36(3), pages 458-470, July.
    24. Bruce Wydick & Elizabeth Katz & Brendan Janet, 2014. "Do in-kind transfers damage local markets? The case of TOMS shoe donations in El Salvador," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(3), pages 249-267, September.
    25. Michael Clemens, 2015. "The Meaning of Failed Replications: A Review and Proposal - Working Paper 399," Working Papers 399, Center for Global Development.
    26. Gary A. Hoover, 2004. "Whose Line Is It? Plagiarism in Economics," Journal of Economic Literature, American Economic Association, vol. 42(2), pages 487-493, June.
    27. Mizon, Grayham E & Richard, Jean-Francois, 1986. "The Encompassing Principle and Its Application to Testing Non-nested Hypotheses," Econometrica, Econometric Society, vol. 54(3), pages 657-678, May.
    28. Anderson, Michael L., 2008. "Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1481-1495.
    29. Daniel S. Hamermesh, 2007. "Replication in Economics," NBER Working Papers 13026, National Bureau of Economic Research, Inc.
    30. Antonio Ciccone, 2013. "Estimating the Effect of Transitory Economic Shocks on Civil Conflict," Review of Economics and Institutions, Università di Perugia, vol. 4(2).
    31. Andreoli-Versbach, Patrick & Mueller-Langer, Frank, 2014. "Open access to data: An ideal professed but not practised," Research Policy, Elsevier, vol. 43(9), pages 1621-1633.
    32. G�nther Fink & Margaret McConnell & Sebastian Vollmer, 2014. "Testing for heterogeneous treatment effects in experimental data: false discovery risks and correction procedures," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(1), pages 44-57, January.
    33. Benjamin A. Olken & Junko Onishi & Susan Wong, 2012. "Should Aid Reward Performance? Evidence from a Field Experiment on Health and Education in Indonesia," NBER Working Papers 17892, National Bureau of Economic Research, Inc.
    34. Chris Doucouliagos, 2005. "Publication Bias in the Economic Freedom and Economic Growth Literature," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 367-387, July.
    35. Graham A. Davis, 2013. "Replicating Sachs and Warner's Working Papers on the Resource Curse," Journal of Development Studies, Taylor & Francis Journals, vol. 49(12), pages 1615-1630, December.
    36. repec:hal:wpspec:info:hdl:2441/eu4vqp9ompqllr09iatr74eao is not listed on IDEAS
    37. Colin J. Roberts, 2005. "Issues in Meta‐Regression Analysis: An Overview," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 295-298, July.
    38. Bellavance, Franois & Dionne, Georges & Lebeau, Martin, 2009. "The value of a statistical life: A meta-analysis with a mixed effects regression model," Journal of Health Economics, Elsevier, vol. 28(2), pages 444-464, March.
    39. Mansfield, Edwin & Schwartz, Mark & Wagner, Samuel, 1981. "Imitation Costs and Patents: An Empirical Study," Economic Journal, Royal Economic Society, vol. 91(364), pages 907-918, December.
    40. Ziliak, Stephen T. & McCloskey, Deirdre N., 2004. "Size matters: the standard error of regressions in the American Economic Review," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 33(5), pages 527-546, November.
    41. Dewald, William G & Thursby, Jerry G & Anderson, Richard G, 1986. "Replication in Empirical Economics: The Journal of Money, Credit and Banking Project," American Economic Review, American Economic Association, vol. 76(4), pages 587-603, September.
    42. B. D. McCullough & H. D. Vinod, 2003. "Verifying the Solution from a Nonlinear Solver: A Case Study," American Economic Review, American Economic Association, vol. 93(3), pages 873-892, June.
    43. Christian Zimmermann, 2015. "On the Need for a Replication Journal," Working Papers 2015-16, Federal Reserve Bank of St. Louis.
    44. Ori Heffetz & Katrina Ligett, 2014. "Privacy and Data-Based Research," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 75-98, Spring.
    45. David M. Drukker & Vince Wiggins, 2004. "Verifying the Solution from a Nonlinear Solver: A Case Study: Comment," American Economic Review, American Economic Association, vol. 94(1), pages 397-399, March.
    46. Daron Acemoglu & Simon Johnson & James A. Robinson, 2012. "The Colonial Origins of Comparative Development: An Empirical Investigation: Reply," American Economic Review, American Economic Association, vol. 102(6), pages 3077-3110, October.
    47. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, January.
    48. Mitch Kunce & Shelby Gerking & William Morgan, 2002. "Effects of Environmental and Land Use Regulation in the Oil and Gas Industry Using the Wyoming Checkerboard as an Experimental Design," American Economic Review, American Economic Association, vol. 92(5), pages 1588-1593, December.
    49. Hristos Doucouliagos & T. D. Stanley, 2009. "Publication Selection Bias in Minimum‐Wage Research? A Meta‐Regression Analysis," British Journal of Industrial Relations, London School of Economics, vol. 47(2), pages 406-428, June.
    50. Orley Ashenfelter & Michael Greenstone, 2004. "Estimating the Value of a Statistical Life: The Importance of Omitted Variables and Publication Bias," American Economic Review, American Economic Association, vol. 94(2), pages 454-460, May.
    51. Sarojini Hirshleifer & David McKenzie & Rita Almeida & Cristobal Ridao‐Cano, 2016. "The Impact of Vocational Training for the Unemployed: Experimental Evidence from Turkey," Economic Journal, Royal Economic Society, vol. 126(597), pages 2115-2146, November.
    52. Jonathan A. Schwabish, 2014. "An Economist's Guide to Visualizing Data," Journal of Economic Perspectives, American Economic Association, vol. 28(1), pages 209-234, Winter.
    53. Lucas C. Coffman & Muriel Niederle, 2015. "Pre-analysis Plans Have Limited Upside, Especially Where Replications Are Feasible," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 81-98, Summer.
    54. Marianne Bertrand & Sendhil Mullainathan, 2004. "Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination," American Economic Review, American Economic Association, vol. 94(4), pages 991-1013, September.
    55. Lars Ivar Oppedal Berge & Kjetil Bjorvatn & Simon Galle & Edward Miguel & Daniel N. Posner & Bertil Tungodden & Kelly Zhang, 2015. "How Strong are Ethnic Preferences?," NBER Working Papers 21715, National Bureau of Economic Research, Inc.
    56. Ashenfelter, Orley & Harmon, Colm & Oosterbeek, Hessel, 1999. "A review of estimates of the schooling/earnings relationship, with tests for publication bias," Labour Economics, Elsevier, vol. 6(4), pages 453-470, November.
    57. T. D. Stanley, 2008. "Meta‐Regression Methods for Detecting and Estimating Empirical Effects in the Presence of Publication Selection," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 70(1), pages 103-127, February.
    58. Holger Görg & Eric Strobl, 2016. "Multinational Companies And Productivity Spillovers: A Meta-Analysis," World Scientific Book Chapters, in: MULTINATIONAL ENTERPRISES AND HOST COUNTRY DEVELOPMENT Volume 53: World Scientific Studies in International Economics, chapter 8, pages 145-161, World Scientific Publishing Co. Pte. Ltd..
    59. Amy Finkelstein & Sarah Taubman & Bill Wright & Mira Bernstein & Jonathan Gruber & Joseph P. Newhouse & Heidi Allen & Katherine Baicker, 2012. "The Oregon Health Insurance Experiment: Evidence from the First Year," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 127(3), pages 1057-1106.
    60. Antonio Ciccone, 2011. "Economic Shocks and Civil Conflict: A Comment," American Economic Journal: Applied Economics, American Economic Association, vol. 3(4), pages 215-227, October.
    61. Roger Koenker & Achim Zeileis, 2009. "On reproducible econometric research," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 24(5), pages 833-847.
    62. Arindrajit Dube & T. William Lester & Michael Reich, 2010. "Minimum Wage Effects Across State Borders: Estimates Using Contiguous Counties," The Review of Economics and Statistics, MIT Press, vol. 92(4), pages 945-964, November.
    63. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    64. Maria Abreu & Henri L. F. de Groot & Raymond J. G. M. Florax, 2005. "A Meta‐Analysis of β‐Convergence: the Legendary 2%," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 389-420, July.
    65. W. Kip Viscusi, 2015. "The Role of Publication Selection Bias in Estimates of the Value of a Statistical Life," American Journal of Health Economics, University of Chicago Press, vol. 1(1), pages 27-52, Winter.
    66. Daniel S. Hamermesh, 2007. "Viewpoint: Replication in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 40(3), pages 715-733, August.
    67. Rachel Glennerster & Kudzai Takavarasha, 2013. "Running Randomized Evaluations: A Practical Guide," Economics Books, Princeton University Press, edition 1, number 10085.
    68. Katherine Casey & Rachel Glennerster & Edward Miguel, 2012. "Reshaping Institutions: Evidence on Aid Impacts Using a Preanalysis Plan," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 127(4), pages 1755-1812.
    69. Monogan, James E., 2013. "A Case for Registering Studies of Political Outcomes: An Application in the 2010 House Elections," Political Analysis, Cambridge University Press, vol. 21(1), pages 21-37, January.
    70. Ron Shachar & Barry Nalebuff, 2004. "Verifying the Solution from a Nonlinear Solver: A Case Study: Comment," American Economic Review, American Economic Association, vol. 94(1), pages 382-390, March.
    71. Neumark, David & Salas, J.M. Ian & Wascher, William, 2013. "Revisiting the Minimum Wage-Employment Debate: Throwing Out the Baby with the Bathwater?," IZA Discussion Papers 7166, Institute of Labor Economics (IZA).
    72. Feldstein, Martin S, 1982. "Social Security and Private Saving: Reply," Journal of Political Economy, University of Chicago Press, vol. 90(3), pages 630-642, June.
    73. Soohyung Lee & Azeem M. Shaikh, 2014. "Multiple Testing And Heterogeneous Treatment Effects: Re‐Evaluating The Effect Of Progresa On School Enrollment," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(4), pages 612-626, June.
    74. Kevin Croke & Joan Hamory Hicks & Eric Hsu & Michael Kremer & Ricardo Maertens & Edward Miguel & Witold Więcek, 2016. "Meta-Analysis and Public Policy: Reconciling the Evidence on Deworming," NBER Working Papers 22382, National Bureau of Economic Research, Inc.
    75. Christophe Bontemps & Grayham E. Mizon, 2008. "Encompassing: Concepts and Implementation," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 70(s1), pages 721-750, December.
    76. B.D. McCullough & Kerry Anne McGeary & Teresa D. Harrison, 2008. "Do economics journal archives promote replicable research?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 41(4), pages 1406-1420, November.
    77. Berger, Ulrich, 2009. "The convergence of fictitious play in games with strategic complementarities: A Comment," MPRA Paper 20241, University Library of Munich, Germany.
    78. Orley Ashenfelter & Colm Harmon & Hessel Oosterbeek, 1999. "A Review of Estimates of the Schooling/Earnings Relationship, with Tests for Publication Bias," Working Papers 804, Princeton University, Department of Economics, Industrial Relations Section..
    79. Hashem Dezhbakhsh & Paul H. Rubin & Joanna M. Shepherd, 2003. "Does Capital Punishment Have a Deterrent Effect? New Evidence from Postmoratorium Panel Data," American Law and Economics Review, American Law and Economics Association, vol. 5(2), pages 344-376, August.
    80. John Ifcher & Homa Zarghamee, 2011. "Happiness and Time Preference: The Effect of Positive Affect in a Random-Assignment Experiment," American Economic Review, American Economic Association, vol. 101(7), pages 3109-3129, December.
    81. Walter Enders & Gary Hoover, 2006. "Plagiarism in the Economics Profession: A Survey," Challenge, Taylor & Francis Journals, vol. 49(5), pages 92-107.
    82. Maren Duvendack & Richard Palmer-Jones, 2013. "Replication of quantitative work in development studies: Experiences and suggestions," Progress in Development Studies, , vol. 13(4), pages 307-322, October.
    83. Welch, Finis, 1974. "Minimum Wage Legislation in the United States," Economic Inquiry, Western Economic Association International, vol. 12(3), pages 285-318, September.
    84. Feldstein, Martin S, 1974. "Social Security, Induced Retirement, and Aggregate Capital Accumulation," Journal of Political Economy, University of Chicago Press, vol. 82(5), pages 905-926, Sept./Oct.
    85. Markus Knell & Helmut Stix, 2005. "The Income Elasticity of Money Demand: A Meta‐Analysis of Empirical Results," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 513-533, July.
    86. McManus, Walter S, 1985. "Estimates of the Deterrent Effect of Capital Punishment: The Importance of the Researcher's Prior Beliefs," Journal of Political Economy, University of Chicago Press, vol. 93(2), pages 417-425, April.
    87. repec:fth:prinin:425 is not listed on IDEAS
    88. Larry V. Hedges & Jack L. Vevea, 1996. "Estimating Effect Size Under Publication Bias: Small Sample Properties and Robustness of a Random Effects Selection Model," Journal of Educational and Behavioral Statistics, , vol. 21(4), pages 299-332, December.
    89. Daniele Fanelli, 2009. "How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data," PLOS ONE, Public Library of Science, vol. 4(5), pages 1-11, May.
    90. repec:hal:spmain:info:hdl:2441/eu4vqp9ompqllr09iatr74eao is not listed on IDEAS
    91. Gerber, Alan & Arceneaux, Kevin & Boudreau, Cheryl & Dowling, Conor & Hillygus, Sunshine & Palfrey, Thomas & Biggers, Daniel R. & Hendry, David J., 2014. "Reporting Guidelines for Experimental Research: A Report from the Experimental Research Section Standards Committee," Journal of Experimental Political Science, Cambridge University Press, vol. 1(1), pages 81-98, April.
    92. Leimer, Dean R & Lesnoy, Selig D, 1982. "Social Security and Private Saving: New Time-Series Evidence," Journal of Political Economy, University of Chicago Press, vol. 90(3), pages 606-629, June.
    93. Doucouliagos, Chris & Stanley, T.D. & Giles, Margaret, 2012. "Are estimates of the value of a statistical life exaggerated?," Journal of Health Economics, Elsevier, vol. 31(1), pages 197-206.
    94. B. D. McCullough & H. D. Vinod, 2004. "Verifying the Solution from a Nonlinear Solver: A Case Study: Reply," American Economic Review, American Economic Association, vol. 94(1), pages 400-406, March.
    95. Martin Feldstein, 1982. "The Welfare Cost of Social Security's Impact on Private Saving," NBER Working Papers 0969, National Bureau of Economic Research, Inc.
    96. Leamer, Edward E & Leonard, Herman B, 1983. "Reporting the Fragility of Regression Estimates," The Review of Economics and Statistics, MIT Press, vol. 65(2), pages 306-317, May.
    97. Xavier Sala-I-Martin & Gernot Doppelhofer & Ronald I. Miller, 2004. "Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach," American Economic Review, American Economic Association, vol. 94(4), pages 813-835, September.
    98. Jesse Chandler & et. al, 2016. "Response to Comment on "Estimating the Reproducibility of Psychological Science"," Mathematica Policy Research Reports cff9c2f16bb544c4bcca530c0, Mathematica Policy Research.
    99. Orley Ashenfelter & Michael Greenstone, 2004. "Estimating the Value of a Statistical Life: The Importance of Omitted Variables and Publication Bias," Working Papers 858, Princeton University, Department of Economics, Industrial Relations Section..
    100. Emeric Henry, 2009. "Disclosure of research results: the cost of proving your honesty," Post-Print hal-01023670, HAL.
    101. Edward Miguel & Shanker Satyanath & Ernest Sergenti, 2004. "Economic Shocks and Civil Conflict: An Instrumental Variables Approach," Journal of Political Economy, University of Chicago Press, vol. 112(4), pages 725-753, August.
    102. Katherine Baicker & Amy Finkelstein & Jae Song & Sarah Taubman, 2014. "The Impact of Medicaid on Labor Market Activity and Program Participation: Evidence from the Oregon Health Insurance Experiment," American Economic Review, American Economic Association, vol. 104(5), pages 322-328, May.
    103. Larry A. Chenault, 1984. "A Note on the Stability Limitations in "A Stable Price Adjustment Process"," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 99(2), pages 385-386.
    104. Chris Doucouliagos & T.D. Stanley, 2013. "Are All Economic Facts Greatly Exaggerated? Theory Competition And Selectivity," Journal of Economic Surveys, Wiley Blackwell, vol. 27(2), pages 316-339, April.
    105. Shelby Gerking & William E. Morgan, 2007. "Effects of Environmental and Land Use Regulation in the Oil and Gas Industry Using the Wyoming Checkerboard as a Natural Experiment: Retraction," American Economic Review, American Economic Association, vol. 97(3), pages 1032-1032, June.
    106. Hunter, John E, 2001. "The Desperate Need for Replications," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 28(1), pages 149-158, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Opoku-Agyemang, Kweku A., 2017. "A Human-Computer Interaction Approach for Integrity in Economics," SocArXiv ra3cs, Center for Open Science.
    2. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    3. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    4. Cristina Blanco-Perez & Abel Brodeur, 2020. "Publication Bias and Editorial Statement on Negative Findings," The Economic Journal, Royal Economic Society, vol. 130(629), pages 1226-1247.
    5. Michael A. Clemens, 2017. "The Meaning Of Failed Replications: A Review And Proposal," Journal of Economic Surveys, Wiley Blackwell, vol. 31(1), pages 326-342, February.
    6. Chris Doucouliagos & T.D. Stanley, 2013. "Are All Economic Facts Greatly Exaggerated? Theory Competition And Selectivity," Journal of Economic Surveys, Wiley Blackwell, vol. 27(2), pages 316-339, April.
    7. Maren Duvendack & Richard Palmer-Jones, 2013. "Replication of quantitative work in development studies: Experiences and suggestions," Progress in Development Studies, , vol. 13(4), pages 307-322, October.
    8. Burlig, Fiona, 2018. "Improving transparency in observational social science research: A pre-analysis plan approach," Economics Letters, Elsevier, vol. 168(C), pages 56-60.
    9. Jarle Møen & Helge Sandvig Thorsen, 2017. "Publication Bias in the Returns to R&D Literature," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 8(3), pages 987-1013, September.
    10. Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
    11. Stanley, T.D. & Doucouliagos, Chris & Jarrell, Stephen B., 2008. "Meta-regression analysis as the socio-economics of economics research," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 37(1), pages 276-292, February.
    12. Annette N. Brown & Drew B. Cameron & Benjamin D. K. Wood, 2014. "Quality evidence for policymaking: I'll believe it when I see the replication," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(3), pages 215-235, September.
    13. Marek Rusnak & Tomas Havranek & Roman Horvath, 2013. "How to Solve the Price Puzzle? A Meta‐Analysis," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 45(1), pages 37-70, February.
    14. Roman Horvath & Ali Elminejad & Tomas Havranek, 2020. "Publication and Identification Biases in Measuring the Intertemporal Substitution of Labor Supply," Working Papers IES 2020/32, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Sep 2020.
    15. Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2018. "Methods Matter: P-Hacking and Causal Inference in Economics," IZA Discussion Papers 11796, Institute of Labor Economics (IZA).
    16. Havranek, Tomas & Irsova, Zuzana & Janda, Karel, 2012. "Demand for gasoline is more price-inelastic than commonly thought," Energy Economics, Elsevier, vol. 34(1), pages 201-207.
    17. Dominika Ehrenbergerova & Josef Bajzik & Tomas Havranek, 2023. "When Does Monetary Policy Sway House Prices? A Meta-Analysis," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(2), pages 538-573, June.
    18. Alex Eble & Peter Boone & Diana Elbourne, 2017. "On Minimizing the Risk of Bias in Randomized Controlled Trials in Economics," The World Bank Economic Review, World Bank, vol. 31(3), pages 687-707.
    19. Maren Duvendack & Richard W. Palmer-Jones & W. Robert Reed, 2015. "Replications in Economics: A Progress Report," Econ Journal Watch, Econ Journal Watch, vol. 12(2), pages 164–191-1, May.
    20. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," Research Policy, Elsevier, vol. 48(1), pages 62-83.

    More about this item

    JEL classification:

    • A12 - General Economics and Teaching - - General Economics - - - Relation of Economics to Other Disciplines
    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology
    • C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:22989. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.