IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/27029.html
   My bibliography  Save this paper

Variation in Performance of Commonly Used Statistical Methods for Estimating Effectiveness of State-Level Opioid Policies on Opioid-Related Mortality

Author

Listed:
  • Beth Ann Griffin
  • Megan S. Schuler
  • Elizabeth A. Stuart
  • Stephen Patrick
  • Elizabeth McNeer
  • Rosanna Smart
  • David Powell
  • Bradley Stein
  • Terry Schell
  • Rosalie Liccardo Pacula

Abstract

Over the last two decades, there has been a surge of opioid-related overdose deaths resulting in a myriad of state policy responses. Researchers have evaluated the effectiveness of such policies using a wide-range of statistical models, each of which requires multiple design choices that can influence the accuracy and precision of the estimated policy effects. This simulation study used real-world data to compare model performance across a range of important statistical constructs to better understand which methods are appropriate for measuring the impacts of state-level opioid policies on opioid-related mortality. Our findings show that many commonly-used methods have very low statistical power to detect a significant policy effect (

Suggested Citation

  • Beth Ann Griffin & Megan S. Schuler & Elizabeth A. Stuart & Stephen Patrick & Elizabeth McNeer & Rosanna Smart & David Powell & Bradley Stein & Terry Schell & Rosalie Liccardo Pacula, 2020. "Variation in Performance of Commonly Used Statistical Methods for Estimating Effectiveness of State-Level Opioid Policies on Opioid-Related Mortality," NBER Working Papers 27029, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:27029
    Note: EH PE
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w27029.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Mark McInerney, 2017. "The Affordable Care Act, Public Insurance Expansion and Opioid Overdose Mortality," Working papers 2017-23, University of Connecticut, Department of Economics.
    2. Stephen G. Donald & Kevin Lang, 2007. "Inference with Difference-in-Differences and Other Panel Data," The Review of Economics and Statistics, MIT Press, vol. 89(2), pages 221-233, May.
    3. Black, Bernard & Hollingsworth, Alex & Nunes, Letícia & Simon, Kosali, 2022. "Simulated power analyses for observational studies: An application to the Affordable Care Act Medicaid expansion," Journal of Public Economics, Elsevier, vol. 213(C).
    4. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    5. Zeileis, Achim, 2004. "Econometric Computing with HC and HAC Covariance Matrix Estimators," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 11(i10).
    6. Zeileis, Achim, 2006. "Object-oriented Computation of Sandwich Estimators," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 16(i09).
    7. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    8. Nathan W. Chan & Jesse Burkhardt & Matthew Flyr, 2020. "The Effects Of Recreational Marijuana Legalization And Dispensing On Opioid Mortality," Economic Inquiry, Western Economic Association International, vol. 58(2), pages 589-606, April.
    9. Thomas C. Buchmueller & Colleen Carey, 2018. "The Effect of Prescription Drug Monitoring Programs on Opioid Utilization in Medicare," American Economic Journal: Economic Policy, American Economic Association, vol. 10(1), pages 77-112, February.
    10. Helland, Eric & Tabarrok, Alexander, 2004. "The Fugitive: Evidence on Public versus Private Law Enforcement from Bail Jumping," Journal of Law and Economics, University of Chicago Press, vol. 47(1), pages 93-122, April.
    11. Ioana Popovici & Johanna Catherine Maclean & Bushra Hijazi & Sharmini Radakrishnan, 2018. "The effect of state laws designed to prevent nonmedical prescription opioid use on overdose deaths and treatment," Health Economics, John Wiley & Sons, Ltd., vol. 27(2), pages 294-305, February.
    12. Birk, Erica G. & Waddell, Glen R., 2017. "The Mitigating Role of Prescription Drug Monitoring Programs in the Abuse of Prescription Drugs," IZA Discussion Papers 10990, Institute of Labor Economics (IZA).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Simone Balestra & Helge Liebert & Nicole Maestas & Tisamarie B. Sherry, 2021. "Behavioral Responses to Supply-Side Drug Policy During the Opioid Epidemic," NBER Working Papers 29596, National Bureau of Economic Research, Inc.
    2. Carolina Arteaga Cabrales & Victoria Barone, 2021. "The Opioid Epidemic: Causes and Consequences," Working Papers tecipa-698, University of Toronto, Department of Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shishir Shakya & Jane E. Ruseski, 2023. "The effect of Prescription Drug Monitoring Programs on county‐level opioid prescribing practices and spillovers," Contemporary Economic Policy, Western Economic Association International, vol. 41(3), pages 435-454, July.
    2. Sviták, Jan & Tichem, Jan & Haasbeek, Stefan, 2021. "Price effects of search advertising restrictions," International Journal of Industrial Organization, Elsevier, vol. 77(C).
    3. A. Colin Cameron & Douglas L. Miller, 2010. "Robust Inference with Clustered Data," Working Papers 106, University of California, Davis, Department of Economics.
    4. Hansen, Bruce E. & Lee, Seojeong, 2019. "Asymptotic theory for clustered samples," Journal of Econometrics, Elsevier, vol. 210(2), pages 268-290.
    5. Johanna Catherine Maclean & Justine Mallatt & Christopher J. Ruhm & Kosali Simon, 2022. "The Opioid Crisis, Health, Healthcare, and Crime: A Review of Quasi-Experimental Economic Studies," The ANNALS of the American Academy of Political and Social Science, , vol. 703(1), pages 15-49, September.
    6. Deiana, Claudio & Giua, Ludovica & Nistico, Roberto, 2019. "The Economics behind the Epidemic: Afghan Opium Price and Prescription Opioids in the US," IZA Discussion Papers 12872, Institute of Labor Economics (IZA).
    7. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2008. "Bootstrap-Based Improvements for Inference with Clustered Errors," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 414-427, August.
    8. Cohen-Goldner, Sarit & Paserman, M. Daniele, 2011. "The dynamic impact of immigration on natives' labor market outcomes: Evidence from Israel," European Economic Review, Elsevier, vol. 55(8), pages 1027-1045.
    9. Deiana, Claudio & Giua, Ludovica, 2018. "The US Opidemic: Prescription Opioids, Labour Market Conditions and Crime," MPRA Paper 85712, University Library of Munich, Germany.
    10. Brewer, Mike & Crossley, Thomas F. & Zilio, Federico, 2019. "What Do We Really Know about the Employment Effects of the UK's National Minimum Wage?," IZA Discussion Papers 12369, Institute of Labor Economics (IZA).
    11. Bruno Ferman & Cristine Pinto, 2019. "Inference in Differences-in-Differences with Few Treated Groups and Heteroskedasticity," The Review of Economics and Statistics, MIT Press, vol. 101(3), pages 452-467, July.
    12. Sakari Lähdemäki, 2017. "Traditional convergence tests with Penn World Table 9.0," Working Papers 309, Työn ja talouden tutkimus LABORE, The Labour Institute for Economic Research LABORE.
    13. Andrew Dillon & Valerie Mueller & Sheu Salau, 2011. "Migratory Responses to Agricultural Risk in Northern Nigeria," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 93(4), pages 1048-1061.
    14. Thomas S. Dee & Brian A. Jacob, 2012. "Rational Ignorance in Education: A Field Experiment in Student Plagiarism," Journal of Human Resources, University of Wisconsin Press, vol. 47(2), pages 397-434.
    15. Thomas C. Buchmueller & Colleen M. Carey & Giacomo Meille, 2020. "How well do doctors know their patients? Evidence from a mandatory access prescription drug monitoring program," Health Economics, John Wiley & Sons, Ltd., vol. 29(9), pages 957-974, September.
    16. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2008. "Bootstrap-Based Improvements for Inference with Clustered Errors," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 414-427, August.
    17. A. Colin Cameron & Douglas L. Miller, 2010. "Robust Inference with Clustered Data," Working Papers 318, University of California, Davis, Department of Economics.
    18. Slawa Rokicki & Jessica Cohen & Gunther Fink & Joshua Salomon & Mary Beth Landrum, 2018. "Inference with difference-in-differences with a small number of groups: a review, simulation study and empirical application using SHARE data," CHaRMS Working Papers 18-01, Centre for HeAlth Research at the Management School (CHaRMS).
    19. Kim, Bokyung, 2021. "Must-access prescription drug monitoring programs and the opioid overdose epidemic: The unintended consequences," Journal of Health Economics, Elsevier, vol. 75(C).
    20. Jochen Heberle & Cristina Sattarhoff, 2017. "A Fast Algorithm for the Computation of HAC Covariance Matrix Estimators," Econometrics, MDPI, vol. 5(1), pages 1-16, January.

    More about this item

    JEL classification:

    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General
    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models
    • C4 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C54 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Quantitative Policy Modeling
    • H0 - Public Economics - - General
    • H51 - Public Economics - - National Government Expenditures and Related Policies - - - Government Expenditures and Health
    • H7 - Public Economics - - State and Local Government; Intergovernmental Relations
    • I18 - Health, Education, and Welfare - - Health - - - Government Policy; Regulation; Public Health
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy
    • K32 - Law and Economics - - Other Substantive Areas of Law - - - Energy, Environmental, Health, and Safety Law

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:27029. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.