The Limits of p‐Hacking: Some Thought Experiments
Author
Abstract
Suggested Citation
DOI: 10.1111/jofi.13036
Download full text from publisher
References listed on IDEAS
- Kelly, Bryan T. & Pruitt, Seth & Su, Yinan, 2019.
"Characteristics are covariances: A unified model of risk and return,"
Journal of Financial Economics, Elsevier, vol. 134(3), pages 501-524.
- Bryan Kelly & Seth Pruitt & Yinan Su, 2018. "Characteristics Are Covariances: A Unified Model of Risk and Return," NBER Working Papers 24540, National Bureau of Economic Research, Inc.
- Weitzman, Martin L, 1979.
"Optimal Search for the Best Alternative,"
Econometrica, Econometric Society, vol. 47(3), pages 641-654, May.
- M. L. Weitzman, 1978. "Optimal Search for the Best Alternative," Working papers 214, Massachusetts Institute of Technology (MIT), Department of Economics.
- Tarun Chordia & Amit Goyal & Alessio Saretto & Andrew KarolyiEditor, 2020. "Anomalies and False Rejections," Review of Finance, European Finance Association, vol. 33(5), pages 2134-2179.
- Andrew Y. Chen & Tom Zimmermann, 2022.
"Open Source Cross-Sectional Asset Pricing,"
Critical Finance Review, now publishers, vol. 11(2), pages 207-264, May.
- Chen, Andrew Y. & Zimmermann, Tom, 2020. "Open source cross-sectional asset pricing," CFR Working Papers 20-04, University of Cologne, Centre for Financial Research (CFR).
- Andrew Y. Chen & Tom Zimmermann, 2021. "Open Source Cross-Sectional Asset Pricing," Finance and Economics Discussion Series 2021-037, Board of Governors of the Federal Reserve System (U.S.).
- R. David Mclean & Jeffrey Pontiff, 2016. "Does Academic Research Destroy Stock Return Predictability?," Journal of Finance, American Finance Association, vol. 71(1), pages 5-32, February.
- Lewellen, Jonathan & Nagel, Stefan & Shanken, Jay, 2010.
"A skeptical appraisal of asset pricing tests,"
Journal of Financial Economics, Elsevier, vol. 96(2), pages 175-194, May.
- Jonathan Lewellen & Stefan Nagel & Jay Shanken, 2006. "A Skeptical Appraisal of Asset-Pricing Tests," NBER Working Papers 12360, National Bureau of Economic Research, Inc.
- Kewei Hou & Chen Xue & Lu Zhang, 2015. "Editor's Choice Digesting Anomalies: An Investment Approach," The Review of Financial Studies, Society for Financial Studies, vol. 28(3), pages 650-705.
- Guanhao Feng & Stefano Giglio & Dacheng Xiu, 2020.
"Taming the Factor Zoo: A Test of New Factors,"
Journal of Finance, American Finance Association, vol. 75(3), pages 1327-1370, June.
- Guanhao Feng & Stefano Giglio & Dacheng Xiu, 2019. "Taming the Factor Zoo: A Test of New Factors," NBER Working Papers 25481, National Bureau of Economic Research, Inc.
- Giglio, Stefano & Feng, Guanhao & Xiu, Dacheng, 2020. "Taming the Factor Zoo: A Test of New Factors," CEPR Discussion Papers 14266, C.E.P.R. Discussion Papers.
- Joel E. Cohen, 2019. "Sum of a Random Number of Correlated Random Variables that Depend on the Number of Summands," The American Statistician, Taylor & Francis Journals, vol. 73(1), pages 56-60, January.
- Raymond Kan & Chu Zhang, 1999. "Two‐Pass Tests of Asset Pricing Models with Useless Factors," Journal of Finance, American Finance Association, vol. 54(1), pages 203-235, February.
- Train,Kenneth E., 2009.
"Discrete Choice Methods with Simulation,"
Cambridge Books,
Cambridge University Press, number 9780521766555, September.
- Train,Kenneth E., 2009. "Discrete Choice Methods with Simulation," Cambridge Books, Cambridge University Press, number 9780521747387, October.
- Kenneth Train, 2003. "Discrete Choice Methods with Simulation," Online economics textbooks, SUNY-Oswego, Department of Economics, number emetr2.
- Fama, Eugene F. & French, Kenneth R., 2015. "A five-factor asset pricing model," Journal of Financial Economics, Elsevier, vol. 116(1), pages 1-22.
- Ryan Sullivan & Allan Timmermann & Halbert White, 1999.
"Data‐Snooping, Technical Trading Rule Performance, and the Bootstrap,"
Journal of Finance, American Finance Association, vol. 54(5), pages 1647-1691, October.
- Sullivan, Ryan & Timmermann, Allan & White, Halbert, 1998. "Data snooping, technical trading, rule performance, and the bootstrap," LSE Research Online Documents on Economics 119144, London School of Economics and Political Science, LSE Library.
- Sullivan, Ryan & Timmermann, Allan G & White, Halbert, 1998. "Data-Snooping, Technical Trading Rule Performance and the Bootstrap," CEPR Discussion Papers 1976, C.E.P.R. Discussion Papers.
- Allan Timmermann & Halbert White & Ryan Sullivan, 1998. "Data-Snooping, Technical Trading, Rule Performance and the Bootstrap," FMG Discussion Papers dp303, Financial Markets Group.
- Kenneth Burdett & Shouyong Shi & Randall Wright, 2001. "Pricing and Matching with Frictions," Journal of Political Economy, University of Chicago Press, vol. 109(5), pages 1060-1085, October.
- Martin Lettau & Markus Pelger & Stijn Van Nieuwerburgh, 2020.
"Factors That Fit the Time Series and Cross-Section of Stock Returns,"
The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2274-2325.
- Lettau, Martin & Pelger, Markus, 2018. "Factors that Fit the Time Series and Cross-Section of Stock Returns," CEPR Discussion Papers 13049, C.E.P.R. Discussion Papers.
- Martin Lettau & Markus Pelger, 2018. "Factors that Fit the Time Series and Cross-Section of Stock Returns," NBER Working Papers 24858, National Bureau of Economic Research, Inc.
- Lo, Andrew W & MacKinlay, A Craig, 1990.
"Data-Snooping Biases in Tests of Financial Asset Pricing Models,"
The Review of Financial Studies, Society for Financial Studies, vol. 3(3), pages 431-467.
- Lo, Andrew W. (Andrew Wen-Chuan) & MacKinlay, Archie Craig, 1955-, 1989. "Data-snooping biases in tests of financial asset pricing models," Working papers 3020-89., Massachusetts Institute of Technology (MIT), Sloan School of Management.
- Andrew W. Lo & A. Craig MacKinlay, 1989. "Data-Snooping Biases in Tests of Financial Asset Pricing Models," NBER Working Papers 3001, National Bureau of Economic Research, Inc.
- Campbell R. Harvey & Yan Liu, 2020. "False (and Missed) Discoveries in Financial Economics," Journal of Finance, American Finance Association, vol. 75(5), pages 2503-2553, October.
- Campbell R. Harvey & Yan Liu, 2020. "False (and Missed) Discoveries in Financial Economics," Papers 2006.04269, arXiv.org.
- Andrew Y Chen & Tom Zimmermann & Jeffrey Pontiff, 2020.
"Publication Bias and the Cross-Section of Stock Returns,"
The Review of Asset Pricing Studies, Society for Financial Studies, vol. 10(2), pages 249-289.
- Andrew Y. Chen & Tom Zimmermann, 2018. "Publication Bias and the Cross-Section of Stock Returns," Finance and Economics Discussion Series 2018-033, Board of Governors of the Federal Reserve System (U.S.).
- Nicholas C. Barberis, 2018. "Psychology-based Models of Asset Prices and Trading Volume," NBER Working Papers 24723, National Bureau of Economic Research, Inc.
- Glenn Ellison, 2002.
"Evolving Standards for Academic Publishing: A q-r Theory,"
Journal of Political Economy, University of Chicago Press, vol. 110(5), pages 994-1034, October.
- Glenn Ellison, 2000. "Evolving Standards for Academic Publishing: A q-r Theory," NBER Working Papers 7805, National Bureau of Economic Research, Inc.
- Isaiah Andrews & Maximilian Kasy, 2019.
"Identification of and Correction for Publication Bias,"
American Economic Review, American Economic Association, vol. 109(8), pages 2766-2794, August.
- Isaiah Andrews & Maximilian Kasy, 2017. "Identification of and correction for publication bias," Papers 1711.10527, arXiv.org.
- Kasy, Maximilian & Andrews, Isaiah, 2018. "Identification of and correction for publication bias," MetaArXiv 49yst, Center for Open Science.
- Isaiah Andrews & Maximilian Kasy, 2017. "Identification of and Correction for Publication Bias," NBER Working Papers 23298, National Bureau of Economic Research, Inc.
- Jacobs, Heiko & Müller, Sebastian, 2020. "Anomalies across the globe: Once public, no longer existent?," Journal of Financial Economics, Elsevier, vol. 135(1), pages 213-230.
- Serhiy Kozak & Stefan Nagel & Shrihari Santosh, 2018. "Interpreting Factor Models," Journal of Finance, American Finance Association, vol. 73(3), pages 1183-1223, June.
- Adam, Klaus, 2001.
"Learning While Searching for the Best Alternative,"
Journal of Economic Theory, Elsevier, vol. 101(1), pages 252-280, November.
- Adam, K., 1999. "Learning While Searching for the Beat Alternative," Economics Working Papers eco99/4, European University Institute.
- John D. Storey & Jonathan E. Taylor & David Siegmund, 2004. "Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(1), pages 187-205, February.
- Efron B. & Tibshirani R. & Storey J.D. & Tusher V., 2001. "Empirical Bayes Analysis of a Microarray Experiment," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1151-1160, December.
- Tarun Chordia & Amit Goyal & Alessio Saretto, 2020. "Anomalies and False Rejections," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2134-2179.
- Campbell R. Harvey & Yan Liu & Heqing Zhu, 2016. "Editor's Choice … and the Cross-Section of Expected Returns," The Review of Financial Studies, Society for Financial Studies, vol. 29(1), pages 5-68.
- James D. Montgomery, 1991. "Equilibrium Wage Dispersion and Interindustry Wage Differentials," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 106(1), pages 163-179.
- Jeremiah Green & John R. M. Hand & X. Frank Zhang, 2017. "The Characteristics that Provide Independent Information about Average U.S. Monthly Stock Returns," The Review of Financial Studies, Society for Financial Studies, vol. 30(12), pages 4389-4436.
- Yoav Benjamini, 2010. "Discovering the false discovery rate," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(4), pages 405-416, September.
Citations
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
Cited by:
- Andrew Y. Chen, 2022. "Most claimed statistical findings in cross-sectional return predictability are likely true," Papers 2206.15365, arXiv.org, revised Sep 2024.
- Andrew Y. Chen, 2022. "Do t-Statistic Hurdles Need to be Raised?," Papers 2204.10275, arXiv.org, revised Apr 2024.
- Andrew Y. Chen & Tom Zimmermann, 2022. "Publication Bias in Asset Pricing Research," Papers 2209.13623, arXiv.org, revised Sep 2023.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Harvey, Campbell R. & Liu, Yan, 2021. "Lucky factors," Journal of Financial Economics, Elsevier, vol. 141(2), pages 413-435.
- Chinco, Alex & Neuhierl, Andreas & Weber, Michael, 2021.
"Estimating the anomaly base rate,"
Journal of Financial Economics, Elsevier, vol. 140(1), pages 101-126.
- Alexander M. Chinco & Andreas Neuhierl & Michael Weber, 2019. "Estimating The Anomaly Base Rate," NBER Working Papers 26493, National Bureau of Economic Research, Inc.
- Söhnke M. Bartram & Harald Lohre & Peter F. Pope & Ananthalakshmi Ranganathan, 2021. "Navigating the factor zoo around the world: an institutional investor perspective," Journal of Business Economics, Springer, vol. 91(5), pages 655-703, July.
- Andrew Y. Chen & Tom Zimmermann, 2022. "Publication Bias in Asset Pricing Research," Papers 2209.13623, arXiv.org, revised Sep 2023.
- Andrew Y. Chen, 2022. "Most claimed statistical findings in cross-sectional return predictability are likely true," Papers 2206.15365, arXiv.org, revised Sep 2024.
- Cakici, Nusret & Zaremba, Adam & Bianchi, Robert J. & Pham, Nga, 2021. "False discoveries in the anomaly research: New insights from the Stock Exchange of Melbourne (1927–1987)," Pacific-Basin Finance Journal, Elsevier, vol. 70(C).
- Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
- Kozak, Serhiy & Nagel, Stefan & Santosh, Shrihari, 2020.
"Shrinking the cross-section,"
Journal of Financial Economics, Elsevier, vol. 135(2), pages 271-292.
- Serhiy Kozak & Stefan Nagel & Shrihari Santosh, 2017. "Shrinking the Cross Section," NBER Working Papers 24070, National Bureau of Economic Research, Inc.
- Nagel, Stefan & Santosh, Shrihari & Kozak, Serhiy, 2017. "Shrinking the Cross Section," CEPR Discussion Papers 12463, C.E.P.R. Discussion Papers.
- Andrew Y. Chen & Alejandro Lopez-Lira & Tom Zimmermann, 2022.
"Does Peer-Reviewed Research Help Predict Stock Returns?,"
Papers
2212.10317, arXiv.org, revised Jun 2024.
- Chen, Andrew Y. & Lopez-Lira, Alejandro & Zimmermann, Tom, 2024. "Does peer-reviewed research help predict stock returns?," CFR Working Papers 24-02, University of Cologne, Centre for Financial Research (CFR).
- Cujean, Julien & Andrei, Daniel & Fournier, Mathieu, 2019. "The Low-Minus-High Portfolio and the Factor Zoo," CEPR Discussion Papers 14153, C.E.P.R. Discussion Papers.
- Hoang, Khoa & Huang, Ronghong & Truong, Helen, 2023. "Resurrecting the market factor: A case of data mining across international markets," Pacific-Basin Finance Journal, Elsevier, vol. 82(C).
- Andrew Y. Chen, 2022. "Do t-Statistic Hurdles Need to be Raised?," Papers 2204.10275, arXiv.org, revised Apr 2024.
- Alex Chinco & Samuel M. Hartzmark & Abigail B. Sussman, 2022. "A New Test of Risk Factor Relevance," Journal of Finance, American Finance Association, vol. 77(4), pages 2183-2238, August.
- Vincent, Kendro & Hsu, Yu-Chin & Lin, Hsiou-Wei, 2021. "Investment styles and the multiple testing of cross-sectional stock return predictability," Journal of Financial Markets, Elsevier, vol. 56(C).
- Chen, Andrew Y. & McCoy, Jack, 2024. "Missing values handling for machine learning portfolios," Journal of Financial Economics, Elsevier, vol. 155(C).
- Gagliardini, Patrick & Ossola, Elisa & Scaillet, Olivier, 2019.
"A diagnostic criterion for approximate factor structure,"
Journal of Econometrics, Elsevier, vol. 212(2), pages 503-521.
- Patrick Gagliardini & Elisa Ossola & Olivier Scaillet, 2016. "A diagnostic criterion for approximate factor structure," Papers 1612.04990, arXiv.org, revised Aug 2017.
- Patrick Gagliardini & Elisa Ossola & O. Scaillet, 2016. "A Diagnostic Criterion for Approximate Factor Structure," Swiss Finance Institute Research Paper Series 16-51, Swiss Finance Institute, revised Dec 2016.
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüß & Michael Razen & Utz Weitzel & David Abad‐Díaz & Menachem (Meni) Abudy , 2024.
"Nonstandard Errors,"
Journal of Finance, American Finance Association, vol. 79(3), pages 2339-2390, June.
- Albert J. Menkveld & Anna Dreber & Félix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard, 2021. "Non-Standard Errors," Documents de travail du Centre d'Economie de la Sorbonne 21033, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Mena, 2021. "Non-Standard Errors," Working Papers 2021:17, Lund University, Department of Economics.
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian Brownlees & Javier Gil-Bazo, 2021. "Non-Standard Errors," Working Papers 1303, Barcelona School of Economics.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," IWH Discussion Papers 11/2021, Halle Institute for Economic Research (IWH).
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian T. Brownlees & Javier Gil-Baz, 2021. "Non-standard errors," Economics Working Papers 1807, Department of Economics and Business, Universitat Pompeu Fabra.
- Albert J. et al. Menkveld, 2021. "Non-Standard Errors," CESifo Working Paper Series 9453, CESifo.
- Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Post-Print halshs-03500882, HAL.
- Francesco Franzoni & Roxana Mihet & Markus Leippold & Per Ostberg & Olivier Scaillet & Norman Schürhoff & Oksana Bashchenko & Nicola Mano & Michele Pelli, 2022. "Non-Standard Errors," Swiss Finance Institute Research Paper Series 22-09, Swiss Finance Institute.
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Edwin Baidoo & Michael Frömmel & et al, 2021. "Non-Standard Errors," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 21/1032, Ghent University, Faculty of Economics and Business Administration.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Hasse, Jean-Baptiste & e.a.,, 2023. "Non-Standard Errors," LIDAM Reprints LFIN 2023002, Université catholique de Louvain, Louvain Finance (LFIN).
- Moinas, Sophie & Declerck, Fany & Menkveld, Albert J. & Dreber, Anna, 2023. "Non-Standard Errors," TSE Working Papers 23-1451, Toulouse School of Economics (TSE).
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüß, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Menac, 2024. "Nonstandard errors," LSE Research Online Documents on Economics 123002, London School of Economics and Political Science, LSE Library.
- Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Cambridge Working Papers in Economics 2182, Faculty of Economics, University of Cambridge.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," SAFE Working Paper Series 327, Leibniz Institute for Financial Research SAFE.
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-Díaz & Menachem Abudy & Tobi, 2021. "Non-Standard Errors," Working Papers 2021-31, Faculty of Economics and Statistics, Universität Innsbruck.
- Ferrara, Gerardo & Jurkatis, Simon, 2021. "Non-standard errors," Bank of England working papers 955, Bank of England.
- Ciril Bosch-Rosa & Bernhard Kassner, 2023. "Non-Standard Errors," Rationality and Competition Discussion Paper Series 385, CRC TRR 190 Rationality and Competition.
- Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-03500882, HAL.
- Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Janeway Institute Working Papers 2112, Faculty of Economics, University of Cambridge.
- Wolff, Christian & Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüess, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-Standard Errors," CEPR Discussion Papers 16751, C.E.P.R. Discussion Papers.
- Clarke, Charles, 2022. "The level, slope, and curve factor model for stocks," Journal of Financial Economics, Elsevier, vol. 143(1), pages 159-187.
- Vitor Azevedo & Christopher Hoegner, 2023. "Enhancing stock market anomalies with machine learning," Review of Quantitative Finance and Accounting, Springer, vol. 60(1), pages 195-230, January.
- Borup, Daniel, 2019. "Asset pricing model uncertainty," Journal of Empirical Finance, Elsevier, vol. 54(C), pages 166-189.
Corrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jfinan:v:76:y:2021:i:5:p:2447-2480. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/afaaaea.html .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.