IDEAS home Printed from https://ideas.repec.org/a/eee/finlet/v41y2021ics1544612320316494.html
   My bibliography  Save this article

Large sample size bias in empirical finance

Author

Listed:
  • Michaelides, Michael

Abstract

The vast majority of empirical studies in finance employ large enough sample sizes and use the conventional thresholds for statistical significance. This routine practice can potentially lead to spurious statistically significant results. The primary aim of this paper is to present a rule of thumb that can be used to determine the appropriate thresholds for statistical significance for a given sample size. The paper argues that the list of statistically significant findings in the broader finance literature is likely to be much shorter after accounting for large sample size bias.

Suggested Citation

  • Michaelides, Michael, 2021. "Large sample size bias in empirical finance," Finance Research Letters, Elsevier, vol. 41(C).
    • Handle: RePEc:eee:finlet:v:41:y:2021:i:c:s1544612320316494
    DOI: 10.1016/j.frl.2020.101835
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1544612320316494
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.frl.2020.101835?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Deirdre N. McCloskey & Stephen T. Ziliak, 1996. "The Standard Error of Regressions," Journal of Economic Literature, American Economic Association, vol. 34(1), pages 97-114, March.
    2. Ivo Welch & Amit Goyal, 2008. "A Comprehensive Look at The Empirical Performance of Equity Premium Prediction," Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1455-1508, July.
    3. Hui Chen & Michael Michaux & Nikolai Roussanov, 2020. "Houses as ATMs: Mortgage Refinancing and Macroeconomic Uncertainty," Journal of Finance, American Finance Association, vol. 75(1), pages 323-375, February.
    4. John P A Ioannidis, 2005. "Why Most Published Research Findings Are False," PLOS Medicine, Public Library of Science, vol. 2(8), pages 1-1, August.
    5. Juhani T Linnainmaa & Michael R Roberts, 2018. "The History of the Cross-Section of Stock Returns," Review of Financial Studies, Society for Financial Studies, vol. 31(7), pages 2606-2649.
    6. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    7. Keuzenkamp, Hugo A. & Magnus, Jan R., 1995. "On tests and significance in econometrics," Journal of Econometrics, Elsevier, vol. 67(1), pages 5-24, May.
    8. Kim, Jae H. & Ji, Philip Inyeob, 2015. "Significance testing in empirical finance: A critical review and assessment," Journal of Empirical Finance, Elsevier, vol. 34(C), pages 1-14.
    9. Wayne E. Ferson & Sergei Sarkissian & Timothy T. Simin, 2003. "Spurious Regressions in Financial Economics?," Journal of Finance, American Finance Association, vol. 58(4), pages 1393-1413, August.
    10. Campbell R. Harvey, 2017. "Presidential Address: The Scientific Outlook in Financial Economics," Journal of Finance, American Finance Association, vol. 72(4), pages 1399-1440, August.
    11. Wayne E. Ferson & Sergei Sarkissian & Timothy T. Simin, 2003. "Spurious Regressions in Financial Economics?," Journal of Finance, American Finance Association, vol. 58(4), pages 1393-1414, August.
    12. Ronald L. Wasserstein & Allen L. Schirm & Nicole A. Lazar, 2019. "Moving to a World Beyond “p," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 1-19, March.
    13. Leonard Kostovetsky & Jerold B. Warner, 2020. "Measuring Innovation and Product Differentiation: Evidence from Mutual Funds," Journal of Finance, American Finance Association, vol. 75(2), pages 779-823, April.
    14. Kewei Hou & Chen Xue & Lu Zhang, 2020. "Replicating Anomalies," Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2019-2133.
    15. Ronald L. Wasserstein & Nicole A. Lazar, 2016. "The ASA's Statement on p -Values: Context, Process, and Purpose," The American Statistician, Taylor & Francis Journals, vol. 70(2), pages 129-133, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jie-Cao He & Hsing-Hua Chang & Ting-Fu Chen & Shih-Kuei Lin, 2023. "Upside and downside correlated jump risk premia of currency options and expected returns," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 9(1), pages 1-58, December.
    2. Shafiqul Islam & Mohammad Fakhrul Islam & Noor-E- Zannat, 2023. "Behavioral Intention to Use Online for Shopping in Bangladesh: A Technology Acceptance Model Analysis," SAGE Open, , vol. 13(3), pages 21582440231, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Todd Mitton, 2022. "Methodological Variation in Empirical Corporate Finance," Review of Financial Studies, Society for Financial Studies, vol. 35(2), pages 527-575.
    2. Andrew Y. Chen & Tom Zimmermann, 2022. "Publication Bias in Asset Pricing Research," Papers 2209.13623, arXiv.org, revised Sep 2023.
    3. Oleg Rytchkov & Xun Zhong, 2020. "Information Aggregation and P-Hacking," Management Science, INFORMS, vol. 66(4), pages 1605-1626, April.
    4. Stephen A. Gorman & Frank J. Fabozzi, 2021. "The ABC’s of the alternative risk premium: academic roots," Journal of Asset Management, Palgrave Macmillan, vol. 22(6), pages 405-436, October.
    5. Jae H. Kim & Kamran Ahmed & Philip Inyeob Ji, 2018. "Significance Testing in Accounting Research: A Critical Evaluation Based on Evidence," Abacus, Accounting Foundation, University of Sydney, vol. 54(4), pages 524-546, December.
    6. Jae H. Kim, 2022. "Moving to a world beyond p-value," Review of Managerial Science, Springer, vol. 16(8), pages 2467-2493, November.
    7. Stephan B. Bruns & David I. Stern, 2019. "Lag length selection and p-hacking in Granger causality testing: prevalence and performance of meta-regression models," Empirical Economics, Springer, vol. 56(3), pages 797-830, March.
    8. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    9. Andrew Y. Chen, 2022. "Most claimed statistical findings in cross-sectional return predictability are likely true," Papers 2206.15365, arXiv.org, revised Mar 2024.
    10. Jae H. Kim & Andrew P. Robinson, 2019. "Interval-Based Hypothesis Testing and Its Applications to Economics and Finance," Econometrics, MDPI, vol. 7(2), pages 1-22, May.
    11. Jeremy Arkes, 2020. "Teaching Graduate (and Undergraduate) Econometrics: Some Sensible Shifts to Improve Efficiency, Effectiveness, and Usefulness," Econometrics, MDPI, vol. 8(3), pages 1-23, September.
    12. Papapostolou, Nikos C. & Pouliasis, Panos K. & Nomikos, Nikos K. & Kyriakou, Ioannis, 2016. "Shipping investor sentiment and international stock return predictability," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 96(C), pages 81-94.
    13. David Haab & Dr. Thomas Nitschka, 2017. "Predicting returns on asset markets of a small, open economy and the influence of global risks," Working Papers 2017-14, Swiss National Bank.
    14. Andersen, Torben G. & Varneskov, Rasmus T., 2021. "Consistent inference for predictive regressions in persistent economic systems," Journal of Econometrics, Elsevier, vol. 224(1), pages 215-244.
    15. Christopher J. Neely & David E. Rapach & Jun Tu & Guofu Zhou, 2014. "Forecasting the Equity Risk Premium: The Role of Technical Indicators," Management Science, INFORMS, vol. 60(7), pages 1772-1791, July.
    16. Guo, Hui & Savickas, Robert & Wang, Zijun & Yang, Jian, 2009. "Is the Value Premium a Proxy for Time-Varying Investment Opportunities? Some Time-Series Evidence," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 44(1), pages 133-154, February.
    17. Yu, Deshui & Huang, Difang & Chen, Li & Li, Luyang, 2023. "Forecasting dividend growth: The role of adjusted earnings yield," Economic Modelling, Elsevier, vol. 120(C).
    18. Doron Avramov & Si Cheng & Lior Metzker, 2023. "Machine Learning vs. Economic Restrictions: Evidence from Stock Return Predictability," Management Science, INFORMS, vol. 69(5), pages 2587-2619, May.
    19. Daniel Mantilla-García & Vijay Vaidyanathan, 2017. "Predicting stock returns in the presence of uncertain structural changes and sample noise," Financial Markets and Portfolio Management, Springer;Swiss Society for Financial Market Research, vol. 31(3), pages 357-391, August.
    20. João M. Sousa & Ricardo M. Sousa, 2019. "Asset Returns Under Model Uncertainty: Evidence from the Euro Area, the US and the UK," Computational Economics, Springer;Society for Computational Economics, vol. 54(1), pages 139-176, June.

    More about this item

    Keywords

    Large sample size; High statistical power; Spurious statistical significance; Appropriate significance thresholds; Methodological crisis; Publication bias;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • G0 - Financial Economics - - General
    • G12 - Financial Economics - - General Financial Markets - - - Asset Pricing; Trading Volume; Bond Interest Rates

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:finlet:v:41:y:2021:i:c:s1544612320316494. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/frl .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.