IDEAS home Printed from https://ideas.repec.org/a/wly/coacre/v11y1994i1p33-57.html
   My bibliography  Save this article

Publication System Biases Associated with the Statistical Testing Paradigm

Author

Listed:
  • R. MURRAY LINDSAY

Abstract

. Running a test of significance has become equated with scientific rigour and the touchstone for establishing knowledge. This article focuses on two related publication system biases that have emanated from holding this episteme: the bias against publishing studies failing to attain statistical significance (i.e., “negative results†) and the tendency to equate scientific significance with statistical significance. It is argued that the emphasis placed on obtaining statistical significance is misguided because statistical significance is not always a prerequisite, nor is it ever sufficient in establishing research findings as scientifically valuable or meaningful. In addition, negative result studies are capable of making a valuable contribution to science and their systematic omission impedes the advance of the discipline. Consequently, statistical significance is found to be a poor criterion on which to base publication decisions. The article also surveys the budgeting, participation, and performance evaluation areas of managerial accounting research to determine whether evidence of these biases exists in accounting. The findings are consistent with the operation of both biases. Some approaches for counteracting or eliminating these biases are examined; as a first step, it is recommended that statistical significance be replaced with the criterion of reasonable precision. Résumé. Le test de signification est devenu synonyme de rigueur scientifique, la pierre de touche de la légitime connaissance. L'auteur se penche sur deux déformations connexes du système de publication qui découlent de cet épistémè: la tendance qui consiste à ne pas publier les études ne parvenant pas à une signification statistique (c'est†à †dire dont les résultats sont négatifs) et la tendance à associer signification scientifique et signification statistique. Selon l'auteur, on fait fausse route en accordant une telle importance à la signification statistique puisqu'elle n'est pas toujours indispensable et qu'elle n'est jamais suffisante pour garantir la valeur scientifique ou la signification des résultats d'une étude. En outre, les études dont les résultats sont négatifs peuvent apporter une contribution précieuse à la science, et leur omission systématique entrave les progrès d'une discipline. La signification statistique est donc un critère de piètre qualité sur lequel fonder les décisions relatives à la publication. L'auteur s'intéresse également aux domaines de la recherche en comptabilité de management qui touchent l'établissement des budgets, la participation et l'évaluation du rendement, dans le but de déterminer si l'existence de ces déformations s'avère en comptabilité. Les résultats corroborent la prégnane des deux déformations. L'auteur examine certaines méthodes visant à contrer ou éliminer ces déformations et recommande, dans une première étape, le remplacement du critère de la signification statistique par celui de la précision raisonnable.

Suggested Citation

  • R. Murray Lindsay, 1994. "Publication System Biases Associated with the Statistical Testing Paradigm," Contemporary Accounting Research, John Wiley & Sons, vol. 11(1), pages 33-57, June.
    • Handle: RePEc:wly:coacre:v:11:y:1994:i:1:p:33-57
    DOI: 10.1111/j.1911-3846.1994.tb00435.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1911-3846.1994.tb00435.x
    Download Restriction: no
    File URL: https://libkey.io/10.1111/j.1911-3846.1994.tb00435.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Seiler, Robert E. & Bartlett, Roger W., 1982. "Personality variables as predictors of budget system characteristics," Accounting, Organizations and Society, Elsevier, vol. 7(4), pages 381-403, October.
    2. Christie, Andrew A., 1990. "Aggregation of test statistics : An evaluation of the evidence on contracting and size hypotheses," Journal of Accounting and Economics, Elsevier, vol. 12(1-3), pages 15-36, January.
    3. Govindarajan, V. & Gupta, Anil K., 1985. "Linking control systems to business unit strategy: impact on performance," Accounting, Organizations and Society, Elsevier, vol. 10(1), pages 51-66, January.
    4. Brownell, P, 1985. "Budgetary Systems And The Control Of Functionally Differentiated Organizational Activities," Journal of Accounting Research, Wiley Blackwell, vol. 23(2), pages 502-512.
    5. Merchant, Kenneth A., 1985. "Budgeting and the propensity to create budgetary slack," Accounting, Organizations and Society, Elsevier, vol. 10(2), pages 201-210, April.
    6. Gordon, Lawrence A. & Narayanan, V. K., 1984. "Management accounting systems, perceived environmental uncertainty and organization structure: An empirical investigation," Accounting, Organizations and Society, Elsevier, vol. 9(1), pages 33-47, January.
    7. A. S. C. Ehrenberg & J. A. Bound, 1993. "Predictability and Prediction," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 156(2), pages 167-194, March.
    8. Ball, R & Foster, G, 1982. "Corporate Financial-Reporting - A Methodological Review Of Empirical-Research," Journal of Accounting Research, Wiley Blackwell, vol. 20, pages 161-234.
    9. Brownell, P & Hirst, M, 1986. "Reliance On Accounting Information, Budgetary Participation, And Task Uncertainty - Tests Of A 3-Way Interaction," Journal of Accounting Research, Wiley Blackwell, vol. 24(2), pages 241-249.
    10. Cherrington, Dj & Cherrington, Jo, 1973. "Appropriate Reinforcement Contingencies In Budgeting Process," Journal of Accounting Research, Wiley Blackwell, vol. 11, pages 225-253.
    11. Govindarajan, V., 1984. "Appropriateness of accounting data in performance evaluation: An empirical examination of environmental uncertainty as an intervening variable," Accounting, Organizations and Society, Elsevier, vol. 9(2), pages 125-135, June.
    12. Young, Sm, 1985. "Participative Budgeting - The Effects Of Risk-Aversion And Asymmetric Information On Budgetary Slack," Journal of Accounting Research, Wiley Blackwell, vol. 23(2), pages 829-842.
    13. Hopwood, Ag, 1972. "Empirical Study Of Role Of Accounting Data In Performance Evaluation," Journal of Accounting Research, Wiley Blackwell, vol. 10, pages 156-182.
    14. Louis Guttman, 1985. "The illogic of statistical inference for cumulative science," Applied Stochastic Models and Data Analysis, John Wiley & Sons, vol. 1(1), pages 3-9.
    15. Leftwich, Richard, 1990. "Aggregation of test statistics: Statistics vs. economics," Journal of Accounting and Economics, Elsevier, vol. 12(1-3), pages 37-44, January.
    16. Hirst, Mk, 1983. "Reliance On Accounting Performance-Measures, Task Uncertainty, And Dysfunctional Behavior - Some Extensions," Journal of Accounting Research, Wiley Blackwell, vol. 21(2), pages 596-605.
    17. Christie, Andrew A., 1987. "On cross-sectional analysis in accounting research," Journal of Accounting and Economics, Elsevier, vol. 9(3), pages 231-258, December.
    18. Feige, Edgar L, 1975. "The Consequences of Journal Editorial Policies and a Suggestion for Revision," Journal of Political Economy, University of Chicago Press, vol. 83(6), pages 1291-1295, December.
    19. Tomkins, Cyril & Groves, Roger, 1983. "The everyday accountant and researching his reality," Accounting, Organizations and Society, Elsevier, vol. 8(4), pages 361-374, October.
    20. Otley, Dt, 1978. "Budget Use And Managerial Performance," Journal of Accounting Research, Wiley Blackwell, vol. 16(1), pages 122-149.
    21. Searfoss, D. Gerald, 1976. "Some behavioral aspects of budgeting for control: An empirical study," Accounting, Organizations and Society, Elsevier, vol. 1(4), pages 375-385, October.
    22. Rockness, Howard O. & Shields, Michael D., 1984. "Organizational control systems in research and development," Accounting, Organizations and Society, Elsevier, vol. 9(2), pages 165-177, June.
    23. Dewald, William G & Thursby, Jerry G & Anderson, Richard G, 1986. "Replication in Empirical Economics: The Journal of Money, Credit and Banking Project," American Economic Review, American Economic Association, vol. 76(4), pages 587-603, September.
    24. Hopwood, Anthony G., 1983. "On trying to study accounting in the contexts in which it operates," Accounting, Organizations and Society, Elsevier, vol. 8(2-3), pages 287-305, May.
    25. Daroca, Frank P., 1984. "Informational influences on group decision making in a participative budgeting context," Accounting, Organizations and Society, Elsevier, vol. 9(1), pages 13-32, January.
    26. Ball, R & Foster, G, 1982. "Corporate Financial-Reporting - A Methodological Review Of Empirical-Research - Reply," Journal of Accounting Research, Wiley Blackwell, vol. 20, pages 245-248.
    27. Kaplan, Robert S., 1986. "The role for empirical research in management accounting," Accounting, Organizations and Society, Elsevier, vol. 11(4-5), pages 429-452, July.
    28. Lev, B, 1989. "On The Usefulness Of Earnings And Earnings Research - Lessons And Directions From 2 Decades Of Empirical-Research," Journal of Accounting Research, Wiley Blackwell, vol. 27, pages 153-192.
    29. Macintosh, N. B. & Daft, R. L., 1987. "Management control systems and departmental interdependencies: An empirical study," Accounting, Organizations and Society, Elsevier, vol. 12(1), pages 49-61, January.
    30. Trotman, Kt & Wood, R, 1991. "A Metaanalysis Of Studies On Internal Control Judgments," Journal of Accounting Research, Wiley Blackwell, vol. 29(1), pages 180-192.
    31. Otley, David T., 1980. "The contingency theory of management accounting: Achievement and prognosis," Accounting, Organizations and Society, Elsevier, vol. 5(4), pages 413-428, October.
    32. Lev, B & Ohlson, Ja, 1982. "Market-Based Empirical-Research In Accounting - A Review, Interpretation, And Extension," Journal of Accounting Research, Wiley Blackwell, vol. 20, pages 249-322.
    33. Tiller, Mg, 1983. "The Dissonance Model Of Participative Budgeting - An Empirical Exploration," Journal of Accounting Research, Wiley Blackwell, vol. 21(2), pages 581-595.
    34. Merchant, Kenneth A., 1984. "Influences on departmental budgeting: an empirical examination of a contingency model," Accounting, Organizations and Society, Elsevier, vol. 9(3-4), pages 291-307, October.
    35. Holstrum, Gl, 1971. "Effect Of Budget Adaptiveness And Tightness On Managerial Decision Behavior," Journal of Accounting Research, Wiley Blackwell, vol. 9(2), pages 268-277.
    36. Hopwood, Ag, 1972. "Empirical Study Of Role Of Accounting Data In Performance Evaluation - Reply," Journal of Accounting Research, Wiley Blackwell, vol. 10, pages 189-193.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Beverley R. Lord & Yvonne P. Shanahan & Benjamin M. Nolan, 2007. "The Use and Perceived Merit of Customer Accounting in New Zealand," Accounting Research Journal, Emerald Group Publishing Limited, vol. 20(1), pages 47-59, July.
    2. Stefania Veltri & Antonella Silvestri, 2020. "The value relevance of corporate financial and nonfinancial information provided by the integrated report: A systematic review," Business Strategy and the Environment, Wiley Blackwell, vol. 29(8), pages 3038-3054, December.
    3. Smith Bamber, Linda & Christensen, Theodore E. & Gaver, Kenneth M., 2000. "Do we really 'know' what we think we know? A case study of seminal research and its subsequent overgeneralization," Accounting, Organizations and Society, Elsevier, vol. 25(2), pages 103-129, February.
    4. Jae H. Kim & Kamran Ahmed & Philip Inyeob Ji, 2018. "Significance Testing in Accounting Research: A Critical Evaluation Based on Evidence," Abacus, Accounting Foundation, University of Sydney, vol. 54(4), pages 524-546, December.
    5. Lindsay, R. Murray, 1995. "Reconsidering the status of tests of significance: An alternative criterion of adequacy," Accounting, Organizations and Society, Elsevier, vol. 20(1), pages 35-53, January.
    6. Thomas R. Dyckman & Stephen A. Zeff, 2019. "Important Issues in Statistical Testing and Recommended Improvements in Accounting Research," Econometrics, MDPI, vol. 7(2), pages 1-11, May.
    7. Raymond Hubbard & JS Armstrong, 2005. "Publication Bias Against Null Results," General Economics and Teaching 0502034, University Library of Munich, Germany.
    8. Hubbard, Raymond & Lindsay, R. Murray, 2013. "From significant difference to significant sameness: Proposing a paradigm shift in business research," Journal of Business Research, Elsevier, vol. 66(9), pages 1377-1388.
    9. Thomas R. Dyckman, 2016. "Significance Testing: We Can Do Better," Abacus, Accounting Foundation, University of Sydney, vol. 52(2), pages 319-342, June.
    10. Ho, Carol-Anne & Williams, S. Mitchell, 2003. "Reply to "International comparative analysis of the association between board structure and the efficiency of value added by a firm from its physical capital and intellectual capital resources: A," The International Journal of Accounting, Elsevier, vol. 38(4), pages 499-502.
    11. Johnstone, David, 2022. "Accounting research and the significance test crisis," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 89(C).
    12. Klaus Derfuss, 2015. "Relating Context Variables to Participative Budgeting and Evaluative Use of Performance Measures: A Meta-analysis," Abacus, Accounting Foundation, University of Sydney, vol. 51(2), pages 238-278, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Luft, Joan & Shields, Michael D., 2003. "Mapping management accounting: graphics and guidelines for theory-consistent empirical research," Accounting, Organizations and Society, Elsevier, vol. 28(2-3), pages 169-249.
    2. Chenhall, Robert H., 2003. "Management control systems design within its organizational context: findings from contingency-based research and directions for the future," Accounting, Organizations and Society, Elsevier, vol. 28(2-3), pages 127-168.
    3. Klaus Derfuss, 2015. "Relating Context Variables to Participative Budgeting and Evaluative Use of Performance Measures: A Meta-analysis," Abacus, Accounting Foundation, University of Sydney, vol. 51(2), pages 238-278, June.
    4. Chapman, Christopher S., 1997. "Reflections on a contingent view of accounting," Accounting, Organizations and Society, Elsevier, vol. 22(2), pages 189-205, February.
    5. Hartmann, Frank G. H., 2000. "The appropriateness of RAPM: toward the further development of theory," Accounting, Organizations and Society, Elsevier, vol. 25(4-5), pages 451-482, May.
    6. Brown, Lawrence D., 1996. "Influential accounting articles, individuals, Ph.D. granting institutions and faculties: A citational analysis," Accounting, Organizations and Society, Elsevier, vol. 21(7-8), pages 723-754.
    7. Hartmann, Frank G. H. & Moers, Frank, 1999. "Testing contingency hypotheses in budgetary research: an evaluation of the use of moderated regression analysis," Accounting, Organizations and Society, Elsevier, vol. 24(4), pages 291-315, May.
    8. Staci A. Kenno & Michelle C. Lau & Barbara J. Sainty, 2018. "In Search of a Theory of Budgeting: A Literature Review," Accounting Perspectives, John Wiley & Sons, vol. 17(4), pages 507-553, December.
    9. Pascal Langevin & Gérald Naro, 2003. "Controle Et Comportements : Une Revue De La Litterature Anglo-Saxonne," Post-Print halshs-00582794, HAL.
    10. Vagneur, K. & Peiperl, M., 2000. "Reconsidering performance evaluative style," Accounting, Organizations and Society, Elsevier, vol. 25(4-5), pages 511-525, May.
    11. Otley, David & Fakiolas, Alexander, 2000. "Reliance on accounting performance measures: dead end or new beginning?," Accounting, Organizations and Society, Elsevier, vol. 25(4-5), pages 497-510, May.
    12. Granlund, Markus & Lukka, Kari, 2017. "Investigating highly established research paradigms: Reviving contextuality in contingency theory based management accounting research," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 45(C), pages 63-80.
    13. Van der Stede, Wim A. & Young, S. Mark & Chen, Clara Xiaoling, 2005. "Assessing the quality of evidence in empirical management accounting research: The case of survey studies," Accounting, Organizations and Society, Elsevier, vol. 30(7-8), pages 655-684.
    14. Shields, J. F. & Shields, M. D., 1998. "Antecedents of participative budgeting," Accounting, Organizations and Society, Elsevier, vol. 23(1), pages 49-76, January.
    15. Chong Lau & Christen Buckland, 2000. "Budget emphasis, participation, task difficulty and performance: the effect of diversity within culture," Accounting and Business Research, Taylor & Francis Journals, vol. 31(1), pages 37-55.
    16. Bhimani, Alnoor & Sivabalan, Prabhu & Soonawalla, Kazbi, 2018. "A study of the linkages between rolling budget forms, uncertainty and strategy," The British Accounting Review, Elsevier, vol. 50(3), pages 306-323.
    17. Ewelina ZARZYCKA & Justyna DOBROSZEK & Cristina CIRCA & Alina ALMASAN, 2017. "The perceived suitability of management accounting information: a contingency based investigation," The Audit Financiar journal, Chamber of Financial Auditors of Romania, vol. 15(147), pages 395-395.
    18. Littkemann, Jörn, 1997. "Erfolgreiche Gestaltung unternehmensinterner Abrechnungssysteme," Manuskripte aus den Instituten für Betriebswirtschaftslehre der Universität Kiel 436, Christian-Albrechts-Universität zu Kiel, Institut für Betriebswirtschaftslehre.
    19. Sarah A. Hinchliffe, 2019. "A Focus on ‘Control’: Reconciling Contemporary Transaction Cost Economics with Behavioural Contingency Accounting Perspectives," Accounting and Finance Research, Sciedu Press, vol. 8(2), pages 189-189, May.
    20. Matthias Meyer & Utz Schäffer & Markus Gmür, 2008. "Transfer und Austausch von Wissen in der Accounting-Forschung: Eine Zitations- und Kozitationsanalyse englischsprachiger Accounting-Journals 1990–2004," Schmalenbach Journal of Business Research, Springer, vol. 60(2), pages 153-181, March.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:coacre:v:11:y:1994:i:1:p:33-57. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1911-3846 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.