Detecting p‐Hacking
Author
Abstract
Suggested Citation
Download full text from publisher
Other versions of this item:
- Graham Elliott & Nikolay Kudrin & Kaspar Wüthrich, 2022. "Detecting p‐Hacking," Econometrica, Econometric Society, vol. 90(2), pages 887-906, March.
- Graham Elliott & Nikolay Kudrin & Kaspar Wuthrich, 2019. "Detecting p-hacking," Papers 1906.06711, arXiv.org, revised May 2021.
References listed on IDEAS
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016.
"Star Wars: The Empirics Strike Back,"
American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2012. "Star wars: The empirics strike back," Working Papers halshs-00710122, HAL.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2015. "Star Wars: The Empirics Strike Back," PSE Working Papers halshs-01158500, HAL.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," PSE-Ecole d'économie de Paris (Postprint) hal-01447851, HAL.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2015. "Star Wars: The Empirics Strike Back," Working Papers halshs-01158500, HAL.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2015. "Star Wars: The Empirics Strike Back," AMSE Working Papers 1523, Aix-Marseille School of Economics, France, revised May 2015.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2012. "Star wars: The empirics strike back," PSE Working Papers halshs-00710122, HAL.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2012. "Star wars: The empirics strike back," SciencePo Working papers Main halshs-00710122, HAL.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2015. "Star Wars: The Empirics Strike Back," Working Papers 1505E, University of Ottawa, Department of Economics.
- Brodeur, Abel & Lé, Mathias & Sangnier, Marc & Zylberberg, Yanos, 2013. "Star Wars: The Empirics Strike Back," IZA Discussion Papers 7268, Institute of Labor Economics (IZA).
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," Post-Print hal-01447851, HAL.
- Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2018.
"Methods Matter: P-Hacking and Causal Inference in Economics,"
IZA Discussion Papers
11796, Institute of Labor Economics (IZA).
- Abel Brodeur & Nikolai Cook & Anthony Heyes, 2018. "Methods Matter: P-Hacking and Causal Inference in Economics," Working Papers 1809E, University of Ottawa, Department of Economics.
- Garret Christensen & Edward Miguel, 2018.
"Transparency, Reproducibility, and the Credibility of Economics Research,"
Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
- Garret S. Christensen & Edward Miguel, 2016. "Transparency, Reproducibility, and the Credibility of Economics Research," NBER Working Papers 22989, National Bureau of Economic Research, Inc.
- Christensen, Garret & Miguel, Edward & Sturdy, Jennifer, 2017. "Transparency, Reproducibility, and the Credibility of Economics Research," MetaArXiv 9a3rw, Center for Open Science.
- Christensen, Garret & Miguel, Edward, 2017. "Transparency, Reproducibility, and the Credibility of Economics Research," Department of Economics, Working Paper Series qt52h6x1cq, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
- Matias D. Cattaneo & Michael Jansson & Xinwei Ma, 2020.
"Simple Local Polynomial Density Estimators,"
Journal of the American Statistical Association, Taylor & Francis Journals, vol. 115(531), pages 1449-1455, July.
- Matias D. Cattaneo & Michael Jansson & Xinwei Ma, 2018. "Simple Local Polynomial Density Estimators," Papers 1811.11512, arXiv.org, revised Jun 2019.
- Cattaneo, Matias D & Jansson, Michael & Ma, Xinwei, 2020. "Simple Local Polynomial Density Estimators," Department of Economics, Working Paper Series qt9vt997qn, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
- Cattaneo, Matias D & Jansson, Michael & Ma, Xinwei, 2020. "Simple Local Polynomial Density Estimators," University of California at San Diego, Economics Working Paper Series qt9vt997qn, Department of Economics, UC San Diego.
- McCrary, Justin, 2008. "Manipulation of the running variable in the regression discontinuity design: A density test," Journal of Econometrics, Elsevier, vol. 142(2), pages 698-714, February.
- Eva Vivalt, 2019. "Specification Searching and Significance Inflation Across Time, Methods and Disciplines," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 81(4), pages 797-816, August.
- Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
- Isaiah Andrews & Maximilian Kasy, 2019.
"Identification of and Correction for Publication Bias,"
American Economic Review, American Economic Association, vol. 109(8), pages 2766-2794, August.
- Isaiah Andrews & Maximilian Kasy, 2017. "Identification of and correction for publication bias," Papers 1711.10527, arXiv.org.
- Kasy, Maximilian & Andrews, Isaiah, 2018. "Identification of and correction for publication bias," MetaArXiv 49yst, Center for Open Science.
- Isaiah Andrews & Maximilian Kasy, 2017. "Identification of and Correction for Publication Bias," NBER Working Papers 23298, National Bureau of Economic Research, Inc.
- Gerber, Alan & Malhotra, Neil, 2008. "Do Statistical Reporting Standards Affect What Is Published? Publication Bias in Two Leading Political Science Journals," Quarterly Journal of Political Science, now publishers, vol. 3(3), pages 313-326, October.
- Joseph P. Romano & Michael Wolf, 2011. "Testing for monotonicity in expected asset returns," ECON - Working Papers 017, Department of Economics - University of Zurich, revised Jan 2013.
- Megan L Head & Luke Holman & Rob Lanfear & Andrew T Kahn & Michael D Jennions, 2015. "The Extent and Consequences of P-Hacking in Science," PLOS Biology, Public Library of Science, vol. 13(3), pages 1-15, March.
- Bruns, Stephan B. & Asanov, Igor & Bode, Rasmus & Dunger, Melanie & Funk, Christoph & Hassan, Sherif M. & Hauschildt, Julia & Heinisch, Dominik & Kempa, Karol & König, Johannes & Lips, Johannes & Verb, 2019. "Reporting errors and biases in published empirical findings: Evidence from innovation research," Research Policy, Elsevier, vol. 48(9), pages 1-1.
- Romano, Joseph P. & Wolf, Michael, 2013. "Testing for monotonicity in expected asset returns," Journal of Empirical Finance, Elsevier, vol. 23(C), pages 93-116.
- Christopher A. Carolan & Joshua M. Tebbs, 2005. "Nonparametric tests for and against likelihood ratio ordering in the two-sample problem," Biometrika, Biometrika Trust, vol. 92(1), pages 159-171, March.
- Beare, Brendan K. & Moon, Jong-Myun, 2015. "Nonparametric Tests Of Density Ratio Ordering," Econometric Theory, Cambridge University Press, vol. 31(3), pages 471-492, June.
- Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
- Snyder, Christopher & Zhuo, Ran, 2018. "Sniff Tests in Economics: Aggregate Distribution of Their Probability Values and Implications for Publication Bias," MetaArXiv 8vdrh, Center for Open Science.
- P. M. Hartigan, 1985. "Computation of the Dip Statistic to Test for Unimodality," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 34(3), pages 320-325, November.
Citations
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
Cited by:
- Guido W. Imbens, 2021. "Statistical Significance, p-Values, and the Reporting of Uncertainty," Journal of Economic Perspectives, American Economic Association, vol. 35(3), pages 157-174, Summer.
- Josef Bajzik & Jan Janku & Simona Malovana & Klara Moravcova & Ngoc Anh Ngo, 2023. "Monetary Policy Has a Long-Lasting Impact on Credit: Evidence from 91 VAR Studies," Working Papers 2023/19, Czech National Bank.
- Simona Malovana & Martin Hodula & Zuzana Gric & Josef Bajzik, 2022. "Borrower-Based Macroprudential Measures and Credit Growth: How Biased is the Existing Literature?," Working Papers 2022/8, Czech National Bank.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Graham Elliott & Nikolay Kudrin & Kaspar Wuthrich, 2022. "The Power of Tests for Detecting $p$-Hacking," Papers 2205.07950, arXiv.org, revised Apr 2024.
- Abel Brodeur & Nikolai Cook & Carina Neisser, 2024.
"p-Hacking, Data type and Data-Sharing Policy,"
The Economic Journal, Royal Economic Society, vol. 134(659), pages 985-1018.
- Brodeur, Abel & Cook, Nikolai & Neisser, Carina, 2022. "P-Hacking, Data Type and Data-Sharing Policy," IZA Discussion Papers 15586, Institute of Labor Economics (IZA).
- Abel Brodeur & Nikolai Cook & Carina Neisser, 2022. "P-Hacking, Data Type and Data-Sharing Policy," ECONtribute Discussion Papers Series 200, University of Bonn and University of Cologne, Germany.
- Brodeur, Abel & Cook, Nikolai & Hartley, Jonathan & Heyes, Anthony, 2022.
"Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?,"
MetaArXiv
uxf39, Center for Open Science.
- Brodeur, Abel & Cook, Nikolai M. & Hartley, Jonathan S. & Heyes, Anthony, 2022. "Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?," GLO Discussion Paper Series 1147, Global Labor Organization (GLO).
- Brodeur, Abel & Cook, Nikolai & Hartley, Jonathan S. & Heyes, Anthony, 2022. "Do Pre-registration and Pre-analysis Plans Reduce P-Hacking and Publication Bias?," IZA Discussion Papers 15476, Institute of Labor Economics (IZA).
- Abel Brodeur, Nikolai M. Cook, Jonathan S. Hartley, Anthony Heyes, 2022. "Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?," LCERPA Working Papers am0132, Laurier Centre for Economic Research and Policy Analysis.
- Abel Brodeur & Scott Carrell & David Figlio & Lester Lusher, 2023.
"Unpacking P-hacking and Publication Bias,"
American Economic Review, American Economic Association, vol. 113(11), pages 2974-3002, November.
- Brodeur, Abel & Carrell, Scott & Figlio, David & Lusher, Lester, 2023. "Unpacking P-Hacking and Publication Bias," I4R Discussion Paper Series 52, The Institute for Replication (I4R).
- Abel Brodeur & Scott E. Carrell & David N. Figlio & Lester R. Lusher, 2023. "Unpacking P-Hacking and Publication Bias," NBER Working Papers 31548, National Bureau of Economic Research, Inc.
- Brodeur, Abel & Carrell, Scott E. & Figlio, David N. & Lusher, Lester, 2023. "Unpacking P-Hacking and Publication Bias," IZA Discussion Papers 16369, Institute of Labor Economics (IZA).
- Anna Dreber & Magnus Johannesson & Yifan Yang, 2024.
"Selective reporting of placebo tests in top economics journals,"
Economic Inquiry, Western Economic Association International, vol. 62(3), pages 921-932, July.
- Dreber, Anna & Johannesson, Magnus & Yang, Yifan, 2023. "Selective Reporting of Placebo Tests in Top Economics Journals," I4R Discussion Paper Series 31, The Institute for Replication (I4R).
- Doucouliagos, Hristos & Hinz, Thomas & Zigova, Katarina, 2022.
"Bias and careers: Evidence from the aid effectiveness literature,"
European Journal of Political Economy, Elsevier, vol. 71(C).
- Doucouliagos, Chris & Hinz, Thomas & Zigova, Katarina, 2020. "Bias and Careers: Evidence from the Aid Effectiveness Literature," IZA Discussion Papers 13287, Institute of Labor Economics (IZA).
- Abel Brodeur, Nikolai M. Cook, Anthony Heyes, 2022.
"We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments,"
LCERPA Working Papers
am0133, Laurier Centre for Economic Research and Policy Analysis.
- Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2022. "We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments," MetaArXiv a9vhr, Center for Open Science.
- Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2022. "We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments," IZA Discussion Papers 15478, Institute of Labor Economics (IZA).
- Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2022.
"We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments,"
GLO Discussion Paper Series
1157, Global Labor Organization (GLO).
- Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2022. "We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments," I4R Discussion Paper Series 8, The Institute for Replication (I4R).
- Dominika Ehrenbergerova & Josef Bajzik & Tomas Havranek, 2023.
"When Does Monetary Policy Sway House Prices? A Meta-Analysis,"
IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(2), pages 538-573, June.
- Havranek, Tomas & Kolcunova, Dominika & Bajzik, Josef, 2021. "When Does Monetary Policy Sway House Prices? A Meta-Analysis," CEPR Discussion Papers 16196, C.E.P.R. Discussion Papers.
- Josef Bajzik & Dominika Ehrenbergerova & Tomas Havranek, 2021. "When Does Monetary Policy Sway House Prices? A Meta-Analysis," Working Papers IES 2021/17, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised May 2021.
- Ehrenbergerova, Dominika & Bajzik, Josef & Havranek, Tomas, 2021. "When Does Monetary Policy Sway House Prices? A Meta-Analysis," EconStor Preprints 234126, ZBW - Leibniz Information Centre for Economics.
- Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
- Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
- Jindrich Matousek & Tomas Havranek & Zuzana Irsova, 2022.
"Individual discount rates: a meta-analysis of experimental evidence,"
Experimental Economics, Springer;Economic Science Association, vol. 25(1), pages 318-358, February.
- Jindrich Matousek, 2018. "Individual Discount Rates: A Meta-Analysis of the Experimental Evidence," Working Papers IES 2018/40, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Dec 2018.
- Havranek, Tomas & Matousek, Jindrich & Irsova, Zuzana, 2021. "Individual Discount Rates: A Meta-Analysis of Experimental Evidence," CEPR Discussion Papers 15688, C.E.P.R. Discussion Papers.
- Matousek, Jindrich & Havranek, Tomas & Irsova, Zuzana, 2019. "Individual Discount Rates: A Meta-Analysis of Experimental Evidence," EconStor Preprints 194617, ZBW - Leibniz Information Centre for Economics.
- Tomas Havranek & Zuzana Irsova & Lubica Laslopova & Olesia Zeynalova, 2020.
"Skilled and Unskilled Labor Are Less Substitutable than Commonly Thought,"
Working Papers IES
2020/29, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Sep 2020.
- Havranek, Tomas & Irsova, Zuzana & Laslopova, Lubica & Zeynalova, Olesia, 2021. "Skilled and Unskilled Labor Are Less Substitutable than Commonly Thought," CEPR Discussion Papers 15724, C.E.P.R. Discussion Papers.
- Havranek, Tomas & Irsova, Zuzana & Laslopova, Lubica & Zeynalova, Olesia, 2020. "Skilled and Unskilled Labor Are Less Substitutable than Commonly Thought," EconStor Preprints 223060, ZBW - Leibniz Information Centre for Economics.
- Abel Brodeur & Nikolai M. Cook & Jonathan S. Hartley & Anthony Heyes, 2024.
"Do Preregistration and Preanalysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement,"
Journal of Political Economy Microeconomics, University of Chicago Press, vol. 2(3), pages 527-561.
- Brodeur, Abel & Cook, Nikolai M. & Hartley, Jonathan S. & Heyes, Anthony, 2023. "Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias?: Evidence from 15,992 Test Statistics and Suggestions for Improvement," GLO Discussion Paper Series 1147 [pre.], Global Labor Organization (GLO).
- Brodeur, Abel & Cook, Nikolai M. & Hartley, Jonathan S. & Heyes, Anthony, 2024. "Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement," I4R Discussion Paper Series 101, The Institute for Replication (I4R).
- Cristina Blanco-Perez & Abel Brodeur, 2019. "Transparency in empirical economic research," IZA World of Labor, Institute of Labor Economics (IZA), pages 467-467, November.
- Alexander Frankel & Maximilian Kasy, 2022.
"Which Findings Should Be Published?,"
American Economic Journal: Microeconomics, American Economic Association, vol. 14(1), pages 1-38, February.
- Kasy, Maximilian & Frankel, Alexander, 2018. "Which findings should be published?," MetaArXiv mbvz3, Center for Open Science.
- Stefano DellaVigna & Elizabeth Linos, 2022.
"RCTs to Scale: Comprehensive Evidence From Two Nudge Units,"
Econometrica, Econometric Society, vol. 90(1), pages 81-116, January.
- Stefano DellaVigna & Elizabeth Linos, 2020. "RCTs to Scale: Comprehensive Evidence from Two Nudge Units," NBER Working Papers 27594, National Bureau of Economic Research, Inc.
- Bruns, Stephan & Herwartz, Helmut & Ioannidis, John P.A. & Islam, Chris-Gabriel & Raters, Fabian H. C., 2023. "Statistical reporting errors in economics," MetaArXiv mbx62, Center for Open Science.
- Roman Horvath & Ali Elminejad & Tomas Havranek, 2020.
"Publication and Identification Biases in Measuring the Intertemporal Substitution of Labor Supply,"
Working Papers IES
2020/32, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Sep 2020.
- Havranek, Tomas & Horváth, Roman & Elminejad , Ali, 2021. "Publication and Identification Biases in Measuring the Intertemporal Substitution of Labor Supply," CEPR Discussion Papers 16032, C.E.P.R. Discussion Papers.
- Elminejad, Ali & Havranek, Tomas & Horvath, Roman, 2021. "Publication and Identification Biases in Measuring the Intertemporal Substitution of Labor Supply," EconStor Preprints 232534, ZBW - Leibniz Information Centre for Economics.
- Havranek, Tomas & Horvath, Roman & Elminejad, Ali, 2021. "Publication and Identification Biases in Measuring the Intertemporal Substitution of Labor Supply," MetaArXiv nshqx, Center for Open Science.
- Dreber, Anna & Johannesson, Magnus, 2023.
"A framework for evaluating reproducibility and replicability in economics,"
Ruhr Economic Papers
1055, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
- Dreber, Anna & Johannesson, Magnus, 2023. "A framework for evaluating reproducibility and replicability in economics," I4R Discussion Paper Series 38, The Institute for Replication (I4R).
More about this item
Keywords
p-values; p-curve; complete monotonicity; publication bias; Economic Theory; Applied Economics; Econometrics;All these keywords.
NEP fields
This paper has been announced in the following NEP Reports:- NEP-SOG-2022-05-16 (Sociology of Economics)
Statistics
Access and download statisticsCorrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cdl:ucsdec:qt2p04s3dr. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Lisa Schiff (email available below). General contact details of provider: https://edirc.repec.org/data/deucsus.html .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.