IDEAS home Printed from https://ideas.repec.org/a/eee/bushor/v65y2022i3p379-388.html
   My bibliography  Save this article

What executives get wrong about statistics: Moving from statistical significance to effect sizes and practical impact

Author

Listed:
  • Anderson, Brian S.

Abstract

Statistical significance functions as an arbiter of sorts for data analysis purporting to show a relationship between two or more variables. Unfortunately, in far too many situations, statistical significance may lead decision-makers relying on data and analytics to improve business decisions astray, particularly in the context of big data. In this article, I outline reasons why executives should develop a healthy discernment when they see the phrase “statistically significant” in media outlets, internal analyses, consulting reports, and other sources. To overcome the limitations of focusing on statistical significance, I propose executives shift their attention toward the effect size reported from a statistical model. While not without limitation, effect sizes are more useful to decision-makers, highlight the practical implication of analyses, and help in quantifying the uncertainty inherent to working with data.

Suggested Citation

  • Anderson, Brian S., 2022. "What executives get wrong about statistics: Moving from statistical significance to effect sizes and practical impact," Business Horizons, Elsevier, vol. 65(3), pages 379-388.
    • Handle: RePEc:eee:bushor:v:65:y:2022:i:3:p:379-388
    DOI: 10.1016/j.bushor.2021.05.001
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S000768132100094X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.bushor.2021.05.001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Blakeley B. McShane & David Gal, 2017. "Rejoinder: Statistical Significance and the Dichotomization of Evidence," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 904-908, July.
    2. Anja Lambrecht & Catherine Tucker, 2019. "Algorithmic Bias? An Empirical Study of Apparent Gender-Based Discrimination in the Display of STEM Career Ads," Management Science, INFORMS, vol. 65(7), pages 2966-2981, July.
    3. Lee, In & Shin, Yong Jae, 2020. "Machine learning for enterprises: Applications, algorithm selection, and challenges," Business Horizons, Elsevier, vol. 63(2), pages 157-170.
    4. Blakeley B. McShane & David Gal, 2017. "Statistical Significance and the Dichotomization of Evidence," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 885-895, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gustavo da Silva Motta, 2022. "What Is a Technological Article?," RAC - Revista de Administração Contemporânea (Journal of Contemporary Administration), ANPAD - Associação Nacional de Pós-Graduação e Pesquisa em Administração, vol. 26(sup2022), pages 220208-2202.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alina Köchling & Marius Claus Wehner, 2020. "Discriminated by an algorithm: a systematic review of discrimination and fairness by algorithmic decision-making in the context of HR recruitment and HR development," Business Research, Springer;German Academic Association for Business Research, vol. 13(3), pages 795-848, November.
    2. Bertoldi, Paolo & Mosconi, Rocco, 2020. "Do energy efficiency policies save energy? A new approach based on energy policy indicators (in the EU Member States)," Energy Policy, Elsevier, vol. 139(C).
    3. Maier, Maximilian & VanderWeele, Tyler & Mathur, Maya B, 2021. "Using Selection Models to Assess Sensitivity to Publication Bias: A Tutorial and Call for More Routine Use," MetaArXiv tp45u, Center for Open Science.
    4. Anderson, Brian S. & Wennberg, Karl & McMullen, Jeffery S., 2019. "Editorial: Enhancing quantitative theory-testing entrepreneurship research," Journal of Business Venturing, Elsevier, vol. 34(5), pages 1-1.
    5. Wennberg, Karl & Anderson, Brian S. & McMullen, Jeffrey, 2019. "2 Editorial: Enhancing Quantitative Theory-Testing Entrepreneurship Research," Ratio Working Papers 323, The Ratio Institute.
    6. Maya B. Mathur & Tyler J. VanderWeele, 2020. "Sensitivity analysis for publication bias in meta‐analyses," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 69(5), pages 1091-1119, November.
    7. David J. Hand, 2022. "Trustworthiness of statistical inference," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(1), pages 329-347, January.
    8. J. M. Bauer & L. A. Reisch, 2019. "Behavioural Insights and (Un)healthy Dietary Choices: a Review of Current Evidence," Journal of Consumer Policy, Springer, vol. 42(1), pages 3-45, March.
    9. Jeffrey A. Mills & Gary Cornwall & Beau A. Sauley & Jeffrey R. Strawn, 2018. "Improving the Analysis of Randomized Controlled Trials: a Posterior Simulation Approach," BEA Working Papers 0157, Bureau of Economic Analysis.
    10. Han Wang & Sieglinde S Snapp & Monica Fisher & Frederi Viens, 2019. "A Bayesian analysis of longitudinal farm surveys in Central Malawi reveals yield determinants and site-specific management strategies," PLOS ONE, Public Library of Science, vol. 14(8), pages 1-17, August.
    11. Tom Engsted, 2024. "What Is the False Discovery Rate in Empirical Research?," Econ Journal Watch, Econ Journal Watch, vol. 21(1), pages 1-92–112, March.
    12. Luigi Pace & Alessandra Salvan, 2020. "Likelihood, Replicability and Robbins' Confidence Sequences," International Statistical Review, International Statistical Institute, vol. 88(3), pages 599-615, December.
    13. Glenn Shafer, 2021. "Testing by betting: A strategy for statistical and scientific communication," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(2), pages 407-431, April.
    14. Maximilian Maier & Tyler J. VanderWeele & Maya B. Mathur, 2022. "Using selection models to assess sensitivity to publication bias: A tutorial and call for more routine use," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(3), September.
    15. Hirschauer Norbert & Grüner Sven & Mußhoff Oliver & Becker Claudia, 2019. "Twenty Steps Towards an Adequate Inferential Interpretation of p-Values in Econometrics," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 239(4), pages 703-721, August.
    16. Sadri, Arash, 2022. "The Ultimate Cause of the “Reproducibility Crisis”: Reductionist Statistics," MetaArXiv yxba5, Center for Open Science.
    17. Strømland, Eirik, 2019. "Preregistration and reproducibility," Journal of Economic Psychology, Elsevier, vol. 75(PA).
    18. Furukawa, Chishio, 2019. "Publication Bias under Aggregation Frictions: Theory, Evidence, and a New Correction Method," EconStor Preprints 194798, ZBW - Leibniz Information Centre for Economics.
    19. Martinovici, A., 2019. "Revealing attention - how eye movements predict brand choice and moment of choice," Other publications TiSEM 7dca38a5-9f78-4aee-bd81-c, Tilburg University, School of Economics and Management.
    20. Bavaresco, Rodrigo Simon & Nesi, Luan Carlos & Victória Barbosa, Jorge Luis & Antunes, Rodolfo Stoffel & da Rosa Righi, Rodrigo & da Costa, Cristiano André & Vanzin, Mariangela & Dornelles, Daniel & J, 2023. "Machine learning-based automation of accounting services: An exploratory case study," International Journal of Accounting Information Systems, Elsevier, vol. 49(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:bushor:v:65:y:2022:i:3:p:379-388. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/bushor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.