IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2608.30490.html

Two Kinds of Nothing: What Insignificant Results in Finance Actually Show

Author

Listed:
  • David Tan

Abstract

Claims of the form "we find no evidence that X affects Y" appear throughout the applied finance literature, yet whether such a claim contains evidence of absence or absence of evidence depends entirely on its confidence interval. The term "statistically insignificant" is routinely read to mean zero economic effect. However, a more honest description is that zero could not be rejected along with a range of other coefficient effect sizes. The crucial question is whether effect sizes in that range are consequential. This note distinguishes two kinds of insignificant results that are indistinguishable in a standard regression table: bounded null claims where the intervals reject effect sizes of consequence and thus represent a genuine finding, and vacuous null claims where even consequential effects remain unrejected and therefore establish nothing. I propose a minimal reporting standard for regression results in applied finance, where the smallest consequential effect size is stated (in the units of the decision, per a named increment of the regressor) alongside the descriptive statistics and compared with the relevant edges of the confidence intervals of null claims. Using only the reported coefficient and standard error, authors can distinguish bounded (informative) null claims that reject consequential effect sizes from vacuous null claims that establish no information, perhaps due to deficiencies in data or the identification strategy. The symmetric phrase "no effect" conceals, in particular, the frequent split verdict: bounded in one direction, vacuous in the other. In ongoing work, I apply this framework to published null claims in leading finance journals, beginning with my own.

Suggested Citation

  • David Tan, 2026. "Two Kinds of Nothing: What Insignificant Results in Finance Actually Show," Papers 2608.30490, arXiv.org, revised Sep 2026.
  • Handle: RePEc:arx:papers:2608.30490
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2608.30490
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Deirdre N. McCloskey & Stephen T. Ziliak, 1996. "The Standard Error of Regressions," Journal of Economic Literature, American Economic Association, vol. 34(1), pages 97-114, March.
    2. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    3. Jae H. Kim & Andrew P. Robinson, 2019. "Interval-Based Hypothesis Testing and Its Applications to Economics and Finance," Econometrics, MDPI, vol. 7(2), pages 1-22, May.
    4. Ronald L. Wasserstein & Nicole A. Lazar, 2016. "The ASA's Statement on p -Values: Context, Process, and Purpose," The American Statistician, Taylor & Francis Journals, vol. 70(2), pages 129-133, May.
    5. Campbell R. Harvey, 2017. "Presidential Address: The Scientific Outlook in Financial Economics," Journal of Finance, American Finance Association, vol. 72(4), pages 1399-1440, August.
    6. Kim, Jae H. & Ji, Philip Inyeob, 2015. "Significance testing in empirical finance: A critical review and assessment," Journal of Empirical Finance, Elsevier, vol. 34(C), pages 1-14.
    7. Foster, F Douglas & Smith, Tom & Whaley, Robert E, 1997. "Assessing Goodness-of-Fit of Asset Pricing Models: The Distribution of the Maximal R-Squared," Journal of Finance, American Finance Association, vol. 52(2), pages 591-607, June.
    8. Todd Mitton, 2024. "Economic Significance in Corporate Finance," The Review of Corporate Finance Studies, Society for Financial Studies, vol. 13(1), pages 38-79.
    9. Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
    10. Valentin Amrhein & Sander Greenland & Blake McShane, 2019. "Scientists rise up against statistical significance," Nature, Nature, vol. 567(7748), pages 305-307, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Heckelei, Thomas & Huettel, Silke & Odening, Martin & Rommel, Jens, "undated". "The replicability crisis and the p-value debate – what are the consequences for the agricultural and food economics community?," Discussion Papers 316369, University of Bonn, Institute for Food and Resource Economics.
    2. Jae H. Kim, 2022. "Moving to a world beyond p-value," Review of Managerial Science, Springer, vol. 16(8), pages 2467-2493, November.
    3. Michaelides, Michael, 2021. "Large sample size bias in empirical finance," Finance Research Letters, Elsevier, vol. 41(C).
    4. Todd Mitton, 2022. "Methodological Variation in Empirical Corporate Finance," The Review of Financial Studies, Society for Financial Studies, vol. 35(2), pages 527-575.
    5. Stephan B. Bruns & David I. Stern, 2019. "Lag length selection and p-hacking in Granger causality testing: prevalence and performance of meta-regression models," Empirical Economics, Springer, vol. 56(3), pages 797-830, March.
    6. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    7. Johnstone, David, 2022. "Accounting research and the significance test crisis," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 89(C).
    8. Jae H. Kim & Philip I. Ji, 2024. "Testing for signal-to-noise ratio in linear regression: a test under large or massive sample," Review of Managerial Science, Springer, vol. 18(10), pages 3007-3024, October.
    9. Christophe Hurlin & Jean-Edouard Colliard & Christophe Pérignon, 2024. "The Economics of Computational Reproducibility," Working Papers hal-04759240, HAL.
    10. Jae H. Kim & Kamran Ahmed & Philip Inyeob Ji, 2018. "Significance Testing in Accounting Research: A Critical Evaluation Based on Evidence," Abacus, Accounting Foundation, University of Sydney, vol. 54(4), pages 524-546, December.
    11. Thibaut Arpinon & Romain Espinosa, 2023. "A practical guide to Registered Reports for economists," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 9(1), pages 90-122, June.
    12. Graham Elliott & Nikolay Kudrin & Kaspar Wuthrich, 2022. "The Power of Tests for Detecting $p$-Hacking," Papers 2205.07950, arXiv.org, revised Aug 2025.
    13. Kim, Jae H., 2017. "Stock returns and investors' mood: Good day sunshine or spurious correlation?," International Review of Financial Analysis, Elsevier, vol. 52(C), pages 94-103.
    14. Jae H. Kim & Andrew P. Robinson, 2019. "Interval-Based Hypothesis Testing and Its Applications to Economics and Finance," Econometrics, MDPI, vol. 7(2), pages 1-22, May.
    15. Thibaut Arpinon & Romain Espinosa, 2023. "A Practical Guide to Registered Reports for Economists," Post-Print halshs-03897719, HAL.
    16. Jens Rommel & Meike Weltin, 2021. "Is There a Cult of Statistical Significance in Agricultural Economics?," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 43(3), pages 1176-1191, September.
    17. Kim, Jae H. & Shamsuddin, Abul, 2023. "Stock market anomalies: An extreme bounds analysis," International Review of Financial Analysis, Elsevier, vol. 90(C).
    18. Geyer-Klingeberg, Jerome & Hang, Markus & Rathgeber, Andreas, 2020. "Meta-analysis in finance research: Opportunities, challenges, and contemporary applications," International Review of Financial Analysis, Elsevier, vol. 71(C).
    19. Fan Yang & Tomas Havranek & Zuzana Irsova & Jiri Novak, 2024. "Is research on hedge fund performance published selectively? A quantitative survey," Journal of Economic Surveys, Wiley Blackwell, vol. 38(4), pages 1085-1131, September.
    20. Jeremy Arkes, 2020. "Teaching Graduate (and Undergraduate) Econometrics: Some Sensible Shifts to Improve Efficiency, Effectiveness, and Usefulness," Econometrics, MDPI, vol. 8(3), pages 1-23, September.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2608.30490. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.