IDEAS home Printed from https://ideas.repec.org/a/gam/jecnmx/v7y2019i2p18-d229157.html
   My bibliography  Save this article

Important Issues in Statistical Testing and Recommended Improvements in Accounting Research

Author

Listed:
  • Thomas R. Dyckman

    (Accounting Department, Cornell University, Ithaca, NY 14850, USA)

  • Stephen A. Zeff

    (Accounting Department, Rice University, Houston, TX 77005, USA)

Abstract

A great deal of the accounting research published in recent years has involved statistical tests. Our paper proposes improvements to both the quality and execution of such research. We address the following limitations in current research that appear to us to be ignored or used inappropriately: (1) unaddressed situational effects resulting from model limitations and what has been referred to as “data carpentry,” (2) limitations and alternatives to winsorizing, (3) necessary improvements to relying on a study’s calculated “ p -values” instead of on the economic or behavioral importance of the results, and (4) the information loss incurred by under-valuing what can and cannot be learned from replications.

Suggested Citation

  • Thomas R. Dyckman & Stephen A. Zeff, 2019. "Important Issues in Statistical Testing and Recommended Improvements in Accounting Research," Econometrics, MDPI, vol. 7(2), pages 1-11, May.
    • Handle: RePEc:gam:jecnmx:v:7:y:2019:i:2:p:18-:d:229157
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2225-1146/7/2/18/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2225-1146/7/2/18/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2018. "Methods Matter: P-Hacking and Causal Inference in Economics," IZA Discussion Papers 11796, Institute of Labor Economics (IZA).
    2. Ian D. Gow & David F. Larcker & Peter C. Reiss, 2016. "Causal Inference in Accounting Research," Journal of Accounting Research, Wiley Blackwell, vol. 54(2), pages 477-523, May.
    3. Ziliak, Stephen T. & McCloskey, Deirdre N., 2004. "Size matters: the standard error of regressions in the American Economic Review," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 33(5), pages 527-546, November.
    4. Dunning,Thad, 2012. "Natural Experiments in the Social Sciences," Cambridge Books, Cambridge University Press, number 9781107017665, November.
    5. Gow, Ian D. & Larcker, David F. & Reiss, Peter C., 2016. "Causal Inference in Accounting Research," Research Papers 3393, Stanford University, Graduate School of Business.
    6. Eric Floyd & John A. List, 2016. "Using Field Experiments in Accounting and Finance," Journal of Accounting Research, Wiley Blackwell, vol. 54(2), pages 437-475, May.
    7. Beaver, Wh, 1968. "Information Content Of Annual Earnings Announcements," Journal of Accounting Research, Wiley Blackwell, vol. 6, pages 67-92.
    8. R. Murray Lindsay, 1994. "Publication System Biases Associated with the Statistical Testing Paradigm," Contemporary Accounting Research, John Wiley & Sons, vol. 11(1), pages 33-57, June.
    9. Lindsay, R. Murray, 1995. "Reconsidering the status of tests of significance: An alternative criterion of adequacy," Accounting, Organizations and Society, Elsevier, vol. 20(1), pages 35-53, January.
    10. Robert Bloomfield & Kristina Rennekamp & Blake Steenhoven, 2018. "No System Is Perfect: Understanding How Registration‐Based Editorial Processes Affect Reproducibility and Investment in Research Quality," Journal of Accounting Research, Wiley Blackwell, vol. 56(2), pages 313-362, May.
    11. Thomas R. Dyckman & Stephen A. Zeff, 2015. "Accounting Research: Past, Present, and Future," Abacus, Accounting Foundation, University of Sydney, vol. 51(4), pages 511-524, December.
    12. James A. Ohlson, 2015. "Accounting Research and Common Sense," Abacus, Accounting Foundation, University of Sydney, vol. 51(4), pages 525-535, December.
    13. Jae H. Kim & Kamran Ahmed & Philip Inyeob Ji, 2018. "Significance Testing in Accounting Research: A Critical Evaluation Based on Evidence," Abacus, Accounting Foundation, University of Sydney, vol. 54(4), pages 524-546, December.
    14. Ronald L. Wasserstein & Allen L. Schirm & Nicole A. Lazar, 2019. "Moving to a World Beyond “p," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 1-19, March.
    15. Thomas R. Dyckman, 2016. "Significance Testing: We Can Do Better," Abacus, Accounting Foundation, University of Sydney, vol. 52(2), pages 319-342, June.
    16. Dunning,Thad, 2012. "Natural Experiments in the Social Sciences," Cambridge Books, Cambridge University Press, number 9781107698000, November.
    17. Smith Bamber, Linda & Christensen, Theodore E. & Gaver, Kenneth M., 2000. "Do we really 'know' what we think we know? A case study of seminal research and its subsequent overgeneralization," Accounting, Organizations and Society, Elsevier, vol. 25(2), pages 103-129, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sylwia Lewicka & Barbara Piotrowska & Aneta Łukaszek-Chmielewska & Tomasz Drzymała, 2022. "Assessment of Natural Radioactivity in Cements Used as Building Materials in Poland," IJERPH, MDPI, vol. 19(18), pages 1-26, September.
    2. Johnstone, David, 2022. "Accounting research and the significance test crisis," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 89(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Johnstone, David, 2022. "Accounting research and the significance test crisis," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 89(C).
    2. Stewart Jones & Nurul Alam, 2019. "A machine learning analysis of citation impact among selected Pacific Basin journals," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 59(4), pages 2509-2552, December.
    3. Ormazabal, Gaizka, 2018. "The Role of Stakeholders in Corporate Governance: A View from Accounting Research," CEPR Discussion Papers 12775, C.E.P.R. Discussion Papers.
    4. Jae H. Kim & Kamran Ahmed & Philip Inyeob Ji, 2018. "Significance Testing in Accounting Research: A Critical Evaluation Based on Evidence," Abacus, Accounting Foundation, University of Sydney, vol. 54(4), pages 524-546, December.
    5. James A. Ohlson, 2022. "Researchers’ data analysis choices: an excess of false positives?," Review of Accounting Studies, Springer, vol. 27(2), pages 649-667, June.
    6. Chen, Xiaomeng Charlene & Jones, Stewart & Hasan, Mostafa Monzur & Zhao, Ruoyun & Alam, Nurul, 2023. "Does strategic deviation influence firms’ use of supplier finance?," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 85(C).
    7. Christian Leuz, 2018. "Evidence-based policymaking: promise, challenges and opportunities for accounting and financial markets research," Accounting and Business Research, Taylor & Francis Journals, vol. 48(5), pages 582-608, July.
    8. Jones, Stewart & Wang, Tim, 2019. "Predicting private company failure: A multi-class analysis," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 61(C), pages 161-188.
    9. Grüner Sven, 2020. "Sample Size Calculation in Economic Experiments," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 240(6), pages 791-823, December.
    10. Joachim Gassen & Hollis A. Skaife & David Veenman, 2020. "Illiquidity and the Measurement of Stock Price Synchronicity," Contemporary Accounting Research, John Wiley & Sons, vol. 37(1), pages 419-456, March.
    11. Ian D. Gow & David F. Larcker & Peter C. Reiss, 2016. "Causal Inference in Accounting Research," Journal of Accounting Research, Wiley Blackwell, vol. 54(2), pages 477-523, May.
    12. Robert Bloomfield & Mark W. Nelson & Eugene Soltes, 2016. "Gathering Data for Archival, Field, Survey, and Experimental Accounting Research," Journal of Accounting Research, Wiley Blackwell, vol. 54(2), pages 341-395, May.
    13. Armstrong, Christopher & Kepler, John D. & Samuels, Delphine & Taylor, Daniel, 2022. "Causality redux: The evolution of empirical methods in accounting research and the growth of quasi-experiments," Journal of Accounting and Economics, Elsevier, vol. 74(2).
    14. Christian Leuz & Peter D. Wysocki, 2016. "The Economics of Disclosure and Financial Reporting Regulation: Evidence and Suggestions for Future Research," Journal of Accounting Research, Wiley Blackwell, vol. 54(2), pages 525-622, May.
    15. Thomas R. Dyckman, 2016. "Significance Testing: We Can Do Better," Abacus, Accounting Foundation, University of Sydney, vol. 52(2), pages 319-342, June.
    16. Suman Banerjee & Saul Estrin & Sarmistha Pal, 2022. "Corporate disclosure, compliance and consequences: evidence from Russia," The European Journal of Finance, Taylor & Francis Journals, vol. 28(17), pages 1770-1802, November.
    17. Rolf Uwe Fülbier & Thorsten Sellhorn, 2023. "Understanding and improving the language of business: How accounting and corporate reporting research can better serve business and society," Journal of Business Economics, Springer, vol. 93(6), pages 1089-1124, August.
    18. Parker Hevron, 2018. "Judicialization and Its Effects: Experiments as a Way Forward," Laws, MDPI, vol. 7(2), pages 1-21, May.
    19. Baldwin, Kate & Bhavnani, Rikhil R., 2013. "Ancillary Experiments: Opportunities and Challenges," WIDER Working Paper Series 024, World Institute for Development Economic Research (UNU-WIDER).
    20. Laurion, Henry, 2020. "Implications of Non-GAAP earnings for real activities and accounting choices," Journal of Accounting and Economics, Elsevier, vol. 70(1).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jecnmx:v:7:y:2019:i:2:p:18-:d:229157. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.