IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/25320.html
   My bibliography  Save this paper

Shackling the Identification Police?

Author

Listed:
  • Christopher J. Ruhm

Abstract

This paper examines potential tradeoffs between research methods in answering important questions versus providing more cleanly identified estimates on problems that are potentially of lesser interest. The strengths and limitations of experimental and quasi-experimental methods are discussed and it is postulated that confidence in the results obtained may sometimes be overvalued compared to the importance of the topics addressed. The consequences of this are modeled and several suggestions are provided regarding possible steps to encourage greater focus on questions of fundamental importance.

Suggested Citation

  • Christopher J. Ruhm, 2018. "Shackling the Identification Police?," NBER Working Papers 25320, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:25320
    Note: AG CH ED EEE EH LS PE
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w25320.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Keane, Michael P., 2010. "Structural vs. atheoretic approaches to econometrics," Journal of Econometrics, Elsevier, vol. 156(1), pages 3-20, May.
    2. Matthew T. Panhans & John D. Singleton, 2015. "The Empirical Economist's Toolkit: From Models to Methods," Center for the History of Political Economy Working Paper Series 2015-3, Center for the History of Political Economy.
    3. George J. Borjas, 2021. "The Wage Impact Of The Marielitos: A Reappraisal," World Scientific Book Chapters, in: Foundational Essays in Immigration Economics, chapter 12, pages 375-408, World Scientific Publishing Co. Pte. Ltd..
    4. David Card, 1990. "The Impact of the Mariel Boatlift on the Miami Labor Market," ILR Review, Cornell University, ILR School, vol. 43(2), pages 245-257, January.
    5. Kasey S. Buckles & Daniel M. Hungerman, 2013. "Season of Birth and Later Outcomes: Old Questions, New Answers," The Review of Economics and Statistics, MIT Press, vol. 95(3), pages 711-724, July.
    6. Michael A. Clemens & Jennifer Hunt, 2019. "The Labor Market Effects of Refugee Waves: Reconciling Conflicting Results," ILR Review, Cornell University, ILR School, vol. 72(4), pages 818-857, August.
    7. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    8. Anne Case & Angus Deaton, 2017. "Mortality and Morbidity in the 21st Century," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 48(1 (Spring), pages 397-476.
    9. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    10. David A. Jaeger & Theodore J. Joyce & Robert Kaestner, 2020. "A Cautionary Tale of Evaluating Identifying Assumptions: Did Reality TV Really Cause a Decline in Teenage Childbearing?," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(2), pages 317-326, April.
    11. William N. Evans & Ethan M. J. Lieber & Patrick Power, 2019. "How the Reformulation of OxyContin Ignited the Heroin Epidemic," The Review of Economics and Statistics, MIT Press, vol. 101(1), pages 1-15, March.
    12. Yu-Wei Luke Chu, 2015. "Do Medical Marijuana Laws Increase Hard-Drug Use?," Journal of Law and Economics, University of Chicago Press, vol. 58(2), pages 481-517.
    13. Powell, David & Pacula, Rosalie Liccardo & Jacobson, Mireille, 2018. "Do medical marijuana laws reduce addictions and deaths related to pain killers?," Journal of Health Economics, Elsevier, vol. 58(C), pages 29-42.
    14. Aviva Aron-Dine & Liran Einav & Amy Finkelstein, 2013. "The RAND Health Insurance Experiment, Three Decades Later," Journal of Economic Perspectives, American Economic Association, vol. 27(1), pages 197-222, Winter.
    15. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    16. Thomas C. Buchmueller & Colleen Carey, 2018. "The Effect of Prescription Drug Monitoring Programs on Opioid Utilization in Medicare," American Economic Journal: Economic Policy, American Economic Association, vol. 10(1), pages 77-112, February.
    17. Jill Horwitz & Corey S. Davis & Lynn S. McClelland & Rebecca S. Fordon & Ellen Meara, 2018. "The Problem of Data Quality in Analyses of Opioid Regulation: The Case of Prescription Drug Monitoring Programs," NBER Working Papers 24947, National Bureau of Economic Research, Inc.
    18. Michael Clemens & Jennifer Hunt, 2017. "The Labor Market Effects of Refugee Waves: Reconciling Conflicting Results - Working Paper 455," Working Papers 455, Center for Global Development, revised 19 Jul 2017.
    19. Manning, Willard G, et al, 1987. "Health Insurance and the Demand for Medical Care: Evidence from a Randomized Experiment," American Economic Review, American Economic Association, vol. 77(3), pages 251-277, June.
    20. Abby Alpert & David Powell & Rosalie Liccardo Pacula, 2018. "Supply-Side Drug Policy in the Presence of Substitutes: Evidence from the Introduction of Abuse-Deterrent Opioids," American Economic Journal: Economic Policy, American Economic Association, vol. 10(4), pages 1-35, November.
    21. Leamer, Edward E, 1983. "Let's Take the Con Out of Econometrics," American Economic Review, American Economic Association, vol. 73(1), pages 31-43, March.
    22. Ariella Kahn-Lang & Kevin Lang, 2020. "The Promise and Pitfalls of Differences-in-Differences: Reflections on 16 and Pregnant and Other Applications," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(3), pages 613-620, July.
    23. Amy Finkelstein, 2007. "The Aggregate Effects of Health Insurance: Evidence from the Introduction of Medicare," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(1), pages 1-37.
    24. Joshua D. Angrist & Alan B. Keueger, 1991. "Does Compulsory School Attendance Affect Schooling and Earnings?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 106(4), pages 979-1014.
    25. Drew B. Cameron & Anjini Mishra & Annette N. Brown, 2016. "The growth of impact evaluation for international development: how much have we learned?," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 8(1), pages 1-21, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jay Bhattacharya & Mikko Packalen, 2020. "Stagnation and Scientific Incentives," NBER Working Papers 26752, National Bureau of Economic Research, Inc.
    2. Eva Forslund & Magnus Henrekson, 2022. "The Virtues of Native Discourse: Striking a Balance Between English and the Native Language," Econ Journal Watch, Econ Journal Watch, vol. 19(2), pages 258–282-2, September.
    3. Ryan H Murphy, 2020. "Does democracy die in recessions? A descriptive analysis of aggregate demand shortfalls and regime transition," Economic Affairs, Wiley Blackwell, vol. 40(1), pages 63-76, February.
    4. Magnus Henrekson & Lars Jonung & Mats Lundahl, 2025. "Not just the top five journals: A recipe for European economists," Economic Affairs, Wiley Blackwell, vol. 45(1), pages 123-131, February.
    5. Skarbek, David, 2020. "Qualitative research methods for institutional analysis," Journal of Institutional Economics, Cambridge University Press, vol. 16(4), pages 409-422, August.
    6. Felix Pretis, 2022. "Does a Carbon Tax Reduce CO2 Emissions? Evidence from British Columbia," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 83(1), pages 115-144, September.
    7. John Mullahy, 2019. "Health and evidence in health economics," Health Economics, John Wiley & Sons, Ltd., vol. 28(10), pages 1163-1165, October.
    8. Richard J. Arend & Pankaj C. Patel, 2020. "The American Dream, Melting Pot And Regional Knowledge Stock As Drivers Of Entrepreneurial Activity," Journal of Developmental Entrepreneurship (JDE), World Scientific Publishing Co. Pte. Ltd., vol. 25(01), pages 1-35, March.
    9. Finigan, Duncan & Mills, Brian M. & Stone, Daniel F., 2020. "Pulling starters," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 89(C).
    10. Janna Bergsvik & Agnes Fauske & Rannveig Kaldager Hart, 2021. "Can Policies Stall the Fertility Fall? A Systematic Review of the (Quasi‐) Experimental Literature," Population and Development Review, The Population Council, Inc., vol. 47(4), pages 913-964, December.
    11. Preetika Joshi & Edmund Outslay & Anh Persson & Terry Shevlin & Aruhn Venkat, 2020. "Does Public Country‐by‐Country Reporting Deter Tax Avoidance and Income Shifting? Evidence from the European Banking Industry," Contemporary Accounting Research, John Wiley & Sons, vol. 37(4), pages 2357-2397, December.
    12. Robert A. Lawson & Ryan Murphy & Benjamin Powell, 2020. "The Determinants Of Economic Freedom: A Survey," Contemporary Economic Policy, Western Economic Association International, vol. 38(4), pages 622-642, October.
    13. Richardson, Eugene T. & Malik, Momin M. & Darity, William A. & Mullen, A. Kirsten & Morse, Michelle E. & Malik, Maya & Maybank, Aletha & Bassett, Mary T. & Farmer, Paul E. & Worden, Lee & Jones, James, 2021. "Reparations for Black American descendants of persons enslaved in the U.S. and their potential impact on SARS-CoV-2 transmission," Social Science & Medicine, Elsevier, vol. 276(C).
    14. Ali Sina Önder & Sergey V. Popov & Sascha Schweitzer, 2021. "Leadership in Scholarship: Editors’ Appointments and the Profession’s Narrative," Working Papers in Economics & Finance 2021-05, University of Portsmouth, Portsmouth Business School, Economics and Finance Subject Group.
    15. Reilly Barry & Sam Hannah, 2022. "The distributional impact of the Sierra Leone conflict on household welfare," IZA Journal of Development and Migration, Sciendo & Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 13(1), pages 1-41, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David Beheshti, 2019. "Adverse health effects of abuse‐deterrent opioids: Evidence from the reformulation of OxyContin," Health Economics, John Wiley & Sons, Ltd., vol. 28(12), pages 1449-1461, December.
    2. Lucy Xiaolu Wang, 2021. "The complementarity of drug monitoring programs and health IT for reducing opioid‐related mortality and morbidity," Health Economics, John Wiley & Sons, Ltd., vol. 30(9), pages 2026-2046, September.
    3. Ruhm, Christopher J., 2019. "Drivers of the fatal drug epidemic," Journal of Health Economics, Elsevier, vol. 64(C), pages 25-42.
    4. Aliprantis, Dionissi & Fee, Kyle & Schweitzer, Mark E., 2023. "Opioids and the labor market," Labour Economics, Elsevier, vol. 85(C).
    5. Nicolas R. Ziebarth, 2018. "Social Insurance and Health," Contributions to Economic Analysis, in: Health Econometrics, volume 127, pages 57-84, Emerald Group Publishing Limited.
    6. Powell, David & Pacula, Rosalie Liccardo & Taylor, Erin, 2020. "How increasing medical access to opioids contributes to the opioid epidemic: Evidence from Medicare Part D," Journal of Health Economics, Elsevier, vol. 71(C).
    7. Gal Wettstein, 2019. "Health insurance and opioid deaths: Evidence from the Affordable Care Act young adult provision," Health Economics, John Wiley & Sons, Ltd., vol. 28(5), pages 666-677, May.
    8. Christopher J. Ruhm, 2018. "Deaths of Despair or Drug Problems?," NBER Working Papers 24188, National Bureau of Economic Research, Inc.
    9. Kim, Bokyung, 2021. "Must-access prescription drug monitoring programs and the opioid overdose epidemic: The unintended consequences," Journal of Health Economics, Elsevier, vol. 75(C).
    10. Pohl, R. Vincent, 2018. "Time Trends Matter: The Case of Medical Cannabis Laws and Opioid Overdose Mortality," MPRA Paper 88219, University Library of Munich, Germany.
    11. Dhaval Dave & Monica Deza & Brady Horn, 2021. "Prescription drug monitoring programs, opioid abuse, and crime," Southern Economic Journal, John Wiley & Sons, vol. 87(3), pages 808-848, January.
    12. David Cho & Daniel I. García & Joshua Montes & Alison E. Weingarden, 2021. "Labor Market Effects of the Oxycodone-Heroin Epidemic," Finance and Economics Discussion Series 2021-025, Board of Governors of the Federal Reserve System (U.S.).
    13. Gihleb, Rania & Giuntella, Osea & Zhang, Ning, 2020. "Prescription drug monitoring programs and neonatal outcomes," Regional Science and Urban Economics, Elsevier, vol. 81(C).
    14. David Powell & Rosalie Liccardo Pacula, 2021. "The Evolving Consequences of OxyContin Reformulation on Drug Overdoses," American Journal of Health Economics, University of Chicago Press, vol. 7(1), pages 41-67.
    15. Anda David & Charbel Nahas & Björn Nilsson & Mohamed Ali Marouani, 2018. "The economics of the Syrian refugee crisis in neighboring countries," Working Papers hal-04000224, HAL.
    16. Liran Einav & Amy Finkelstein, 2018. "Moral Hazard in Health Insurance: What We Know and How We Know It," Journal of the European Economic Association, European Economic Association, vol. 16(4), pages 957-982.
    17. Simone Balestra & Helge Liebert & Nicole Maestas & Tisamarie B. Sherry, 2021. "Behavioral Responses to Supply-Side Drug Policy During the Opioid Epidemic," NBER Working Papers 29596, National Bureau of Economic Research, Inc.
    18. Buckles, Kasey & Evans, William N. & Lieber, Ethan M.J., 2023. "The drug crisis and the living arrangements of children," Journal of Health Economics, Elsevier, vol. 87(C).
    19. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    20. Jason Anastasopoulos & George J. Borjas & Gavin G. Cook & Michael Lachanski, 2018. "Job Vacancies, the Beveridge Curve, and Supply Shocks: The Frequency and Content of Help-Wanted Ads in Pre- and Post-Mariel Miami," NBER Working Papers 24580, National Bureau of Economic Research, Inc.

    More about this item

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • B4 - Schools of Economic Thought and Methodology - - Economic Methodology
    • C50 - Mathematical and Quantitative Methods - - Econometric Modeling - - - General
    • H0 - Public Economics - - General
    • I0 - Health, Education, and Welfare - - General
    • J0 - Labor and Demographic Economics - - General
    • O1 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:25320. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.