IDEAS home Printed from https://ideas.repec.org/a/wly/riskan/v39y2019i7p1520-1532.html
   My bibliography  Save this article

Null Hypothesis Testing ≠ Scientific Inference: A Critique of the Shaky Premise at the Heart of the Science and Values Debate, and a Defense of Value‐Neutral Risk Assessment

Author

Listed:
  • Brian H. MacGillivray

Abstract

Many philosophers and statisticians argue that risk assessors are morally obligated to evaluate the probabilities and consequences of methodological error, and to base their decisions of whether to adopt a given parameter value, model, or hypothesis on those considerations. This argument is couched within the rubric of null hypothesis testing, which I suggest is a poor descriptive and normative model for risk assessment. Risk regulation is not primarily concerned with evaluating the probability of data conditional upon the null hypothesis, but rather with measuring risks, estimating the consequences of available courses of action and inaction, formally characterizing uncertainty, and deciding what to do based upon explicit values and decision criteria. In turn, I defend an ideal of value‐neutrality, whereby the core inferential tasks of risk assessment—such as weighing evidence, estimating parameters, and model selection—should be guided by the aim of correspondence to reality. This is not to say that value judgments be damned, but rather that they should be accounted for within a structured approach to decision analysis, rather than embedded within risk assessment in an informal manner.

Suggested Citation

  • Brian H. MacGillivray, 2019. "Null Hypothesis Testing ≠ Scientific Inference: A Critique of the Shaky Premise at the Heart of the Science and Values Debate, and a Defense of Value‐Neutral Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 39(7), pages 1520-1532, July.
  • Handle: RePEc:wly:riskan:v:39:y:2019:i:7:p:1520-1532
    DOI: 10.1111/risa.13284
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/risa.13284
    Download Restriction: no

    File URL: https://libkey.io/10.1111/risa.13284?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sander Greenland, 2005. "Multiple‐bias modelling for analysis of observational data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(2), pages 267-306, March.
    2. Oecd, 1996. "The Role of Information in Telecommunications Regulations," OECD Digital Economy Papers 22, OECD Publishing.
    3. Robert J. Lempert & Myles T. Collins, 2007. "Managing the Risk of Uncertain Threshold Responses: Comparison of Robust, Optimum, and Precautionary Approaches," Risk Analysis, John Wiley & Sons, vol. 27(4), pages 1009-1026, August.
    4. Andrew Gelman & Christian Hennig, 2017. "Beyond subjective and objective in statistics," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(4), pages 967-1033, October.
    5. Willy Aspinall, 2010. "A route to more tractable expert advice," Nature, Nature, vol. 463(7279), pages 294-295, January.
    6. Paul Slovic, 1999. "Trust, Emotion, Sex, Politics, and Science: Surveying the Risk‐Assessment Battlefield," Risk Analysis, John Wiley & Sons, vol. 19(4), pages 689-701, August.
    7. Stanley Kaplan & B. John Garrick, 1981. "On The Quantitative Definition of Risk," Risk Analysis, John Wiley & Sons, vol. 1(1), pages 11-27, March.
    8. J.S. Busby & R.E. Alcock & B.H. MacGillivray, 2012. "Types of risk transformation: a case study," Journal of Risk Research, Taylor & Francis Journals, vol. 15(1), pages 67-84, January.
    9. Brian H. MacGillivray, 2014. "Heuristics Structure and Pervade Formal Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 34(4), pages 771-787, April.
    10. Aven, Terje, 2013. "Practical implications of the new risk perspectives," Reliability Engineering and System Safety, Elsevier, vol. 115(C), pages 136-145.
    11. Coglianese, Cary & Marchant, Gary E., 2004. "The EPA's Risky Reasoning," Working Paper Series rwp04-046, Harvard University, John F. Kennedy School of Government.
    12. Robert T. Clemen & Robert L. Winkler, 1999. "Combining Probability Distributions From Experts in Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 19(2), pages 187-203, April.
    13. Louis Anthony (Tony) Cox, 2012. "Confronting Deep Uncertainties in Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 32(10), pages 1607-1629, October.
    14. Borgonovo, E. & Cappelli, V. & Maccheroni, F. & Marinacci, M., 2018. "Risk analysis and decision theory: A bridge," European Journal of Operational Research, Elsevier, vol. 264(1), pages 280-293.
    15. Lash, T.L. & Fox, M.P. & Cooney, D. & Lu, Y. & Forshee, R.A., 2016. "Quantitative bias analysis in regulatory settings," American Journal of Public Health, American Public Health Association, vol. 106(7), pages 1227-1230.
    16. Sven Ove Hansson & Terje Aven, 2014. "Is Risk Analysis Scientific?," Risk Analysis, John Wiley & Sons, vol. 34(7), pages 1173-1183, July.
    17. Andy Stirling, 2010. "Keep it complex," Nature, Nature, vol. 468(7327), pages 1029-1031, December.
    18. W. Kip Viscusi, 1996. "Economic Foundations of the Current Regulatory Reform Efforts," Journal of Economic Perspectives, American Economic Association, vol. 10(3), pages 119-134, Summer.
    19. Michael Mastrandrea & Katharine Mach & Gian-Kasper Plattner & Ottmar Edenhofer & Thomas Stocker & Christopher Field & Kristie Ebi & Patrick Matschoss, 2011. "The IPCC AR5 guidance note on consistent treatment of uncertainties: a common approach across the working groups," Climatic Change, Springer, vol. 108(4), pages 675-691, October.
    20. Aven, Terje, 2016. "Risk assessment and risk management: Review of recent advances on their foundation," European Journal of Operational Research, Elsevier, vol. 253(1), pages 1-13.
    21. Sheila Jasanoff, 1989. "Norms for Evaluating Regulatory Science," Risk Analysis, John Wiley & Sons, vol. 9(3), pages 271-273, September.
    22. Zweifel, Peter & Crivelli, Luca, 1996. "Price Regulation of Drugs: Lessons from Germany," Journal of Regulatory Economics, Springer, vol. 10(3), pages 257-273, November.
    23. Michael Oppenheimer & Christopher M. Little & Roger M. Cooke, 2016. "Expert judgement and uncertainty quantification for climate change," Nature Climate Change, Nature, vol. 6(5), pages 445-451, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. David G Jenkins & Pedro F Quintana-Ascencio, 2020. "A solution to minimum sample size for regressions," PLOS ONE, Public Library of Science, vol. 15(2), pages 1-15, February.
    2. Daniel J. Hicks & P. D. Magnus & Jessey Wright, 2020. "Inductive Risk, Science, and Values: A Reply to MacGillivray," Risk Analysis, John Wiley & Sons, vol. 40(4), pages 667-673, April.
    3. Vicki Bier, 2020. "The Role of Decision Analysis in Risk Analysis: A Retrospective," Risk Analysis, John Wiley & Sons, vol. 40(S1), pages 2207-2217, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aven, Terje, 2020. "Three influential risk foundation papers from the 80s and 90s: Are they still state-of-the-art?," Reliability Engineering and System Safety, Elsevier, vol. 193(C).
    2. Amro Nasr & Oskar Larsson Ivanov & Ivar Björnsson & Jonas Johansson & Dániel Honfi, 2021. "Towards a Conceptual Framework for Built Infrastructure Design in an Uncertain Climate: Challenges and Research Needs," Sustainability, MDPI, vol. 13(21), pages 1-19, October.
    3. Henrik Hassel & Alexander Cedergren, 2019. "Exploring the Conceptual Foundation of Continuity Management in the Context of Societal Safety," Risk Analysis, John Wiley & Sons, vol. 39(7), pages 1503-1519, July.
    4. Tommi Ekholm & Erin Baker, 2022. "Multiple Beliefs, Dominance and Dynamic Consistency," Management Science, INFORMS, vol. 68(1), pages 529-540, January.
    5. Baker, Erin & Bosetti, Valentina & Salo, Ahti, 2020. "Robust portfolio decision analysis: An application to the energy research and development portfolio problem," European Journal of Operational Research, Elsevier, vol. 284(3), pages 1107-1120.
    6. Hamed Taherdoost, 2021. "A Review on Risk Management in Information Systems: Risk Policy, Control and Fraud Detection," Post-Print hal-03741848, HAL.
    7. Moshe Yanovskiy & Yehoshua Socol, 2022. "Are Lockdowns Effective in Managing Pandemics?," IJERPH, MDPI, vol. 19(15), pages 1-12, July.
    8. Torbjørn Bjerga & Terje Aven, 2016. "Some perspectives on risk management: A security case study from the oil and gas industry," Journal of Risk and Reliability, , vol. 230(5), pages 512-520, October.
    9. Jacob Taarup‐Esbensen, 2019. "Making Sense of Risk—A Sociological Perspective on the Management of Risk," Risk Analysis, John Wiley & Sons, vol. 39(4), pages 749-760, April.
    10. Payuna Uday & Karen Marais, 2015. "Designing Resilient Systems‐of‐Systems: A Survey of Metrics, Methods, and Challenges," Systems Engineering, John Wiley & Sons, vol. 18(5), pages 491-510, October.
    11. Zio, E., 2018. "The future of risk assessment," Reliability Engineering and System Safety, Elsevier, vol. 177(C), pages 176-190.
    12. Tasneem Bani-Mustafa & Nicola Pedroni & Enrico Zio & Dominique Vasseur & Francois Beaudouin, 2020. "A hierarchical tree-based decision-making approach for assessing the relative trustworthiness of risk assessment models," Journal of Risk and Reliability, , vol. 234(6), pages 748-763, December.
    13. Julie E. Shortridge & Benjamin F. Zaitchik, 2018. "Characterizing climate change risks by linking robust decision frameworks and uncertain probabilistic projections," Climatic Change, Springer, vol. 151(3), pages 525-539, December.
    14. Scholz, Roland W. & Czichos, Reiner & Parycek, Peter & Lampoltshammer, Thomas J., 2020. "Organizational vulnerability of digital threats: A first validation of an assessment method," European Journal of Operational Research, Elsevier, vol. 282(2), pages 627-643.
    15. Ram, Camelia, 2020. "Scenario presentation and scenario generation in multi-criteria assessments: An exploratory study," Technological Forecasting and Social Change, Elsevier, vol. 151(C).
    16. Giuseppe Di Vita, 2023. "The economic impact of legislative complexity and corruption: A cross‐country analysis," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 28(2), pages 1801-1825, April.
    17. Aven, Terje, 2018. "How the integration of System 1-System 2 thinking and recent risk perspectives can improve risk assessment and management," Reliability Engineering and System Safety, Elsevier, vol. 180(C), pages 237-244.
    18. Kjell Hausken, 2019. "Principal–Agent Theory, Game Theory, and the Precautionary Principle," Decision Analysis, INFORMS, vol. 16(2), pages 105-127, June.
    19. Anca M. Hanea & Marissa F. McBride & Mark A. Burgman & Bonnie C. Wintle, 2018. "The Value of Performance Weights and Discussion in Aggregated Expert Judgments," Risk Analysis, John Wiley & Sons, vol. 38(9), pages 1781-1794, September.
    20. Inger Lise Johansen & Marvin Rausand, 2014. "Defining complexity for risk assessment of sociotechnical systems: A conceptual framework," Journal of Risk and Reliability, , vol. 228(3), pages 272-290, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:riskan:v:39:y:2019:i:7:p:1520-1532. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1539-6924 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.