IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/25320.html
   My bibliography  Save this paper

Shackling the Identification Police?

Author

Listed:
  • Christopher J. Ruhm

Abstract

This paper examines potential tradeoffs between research methods in answering important questions versus providing more cleanly identified estimates on problems that are potentially of lesser interest. The strengths and limitations of experimental and quasi-experimental methods are discussed and it is postulated that confidence in the results obtained may sometimes be overvalued compared to the importance of the topics addressed. The consequences of this are modeled and several suggestions are provided regarding possible steps to encourage greater focus on questions of fundamental importance.

Suggested Citation

  • Christopher J. Ruhm, 2018. "Shackling the Identification Police?," NBER Working Papers 25320, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:25320
    Note: AG CH ED EEE EH LS PE
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w25320.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Keane, Michael P., 2010. "Structural vs. atheoretic approaches to econometrics," Journal of Econometrics, Elsevier, vol. 156(1), pages 3-20, May.
    2. Matthew T. Panhans & John D. Singleton, 2015. "The Empirical Economist's Toolkit: From Models to Methods," Center for the History of Political Economy Working Paper Series 2015-3, Center for the History of Political Economy.
    3. George J. Borjas, 2021. "The Wage Impact Of The Marielitos: A Reappraisal," World Scientific Book Chapters, in: Foundational Essays in Immigration Economics, chapter 12, pages 375-408, World Scientific Publishing Co. Pte. Ltd..
    4. David Card, 1989. "The Impact of the Mariel Boatlift on the Miami Labor Market," Working Papers 633, Princeton University, Department of Economics, Industrial Relations Section..
    5. Kasey S. Buckles & Daniel M. Hungerman, 2013. "Season of Birth and Later Outcomes: Old Questions, New Answers," The Review of Economics and Statistics, MIT Press, vol. 95(3), pages 711-724, July.
    6. Michael A. Clemens & Jennifer Hunt, 2019. "The Labor Market Effects of Refugee Waves: Reconciling Conflicting Results," ILR Review, Cornell University, ILR School, vol. 72(4), pages 818-857, August.
    7. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    8. Anne Case & Angus Deaton, 2017. "Mortality and Morbidity in the 21st Century," Working Papers 2017-spring, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
    9. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    10. David A. Jaeger & Theodore J. Joyce & Robert Kaestner, 2020. "A Cautionary Tale of Evaluating Identifying Assumptions: Did Reality TV Really Cause a Decline in Teenage Childbearing?," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(2), pages 317-326, April.
    11. David Card, 1990. "The Impact of the Mariel Boatlift on the Miami Labor Market," ILR Review, Cornell University, ILR School, vol. 43(2), pages 245-257, January.
    12. William N. Evans & Ethan M. J. Lieber & Patrick Power, 2019. "How the Reformulation of OxyContin Ignited the Heroin Epidemic," The Review of Economics and Statistics, MIT Press, vol. 101(1), pages 1-15, March.
    13. Yu-Wei Luke Chu, 2015. "Do Medical Marijuana Laws Increase Hard-Drug Use?," Journal of Law and Economics, University of Chicago Press, vol. 58(2), pages 481-517.
    14. Powell, David & Pacula, Rosalie Liccardo & Jacobson, Mireille, 2018. "Do medical marijuana laws reduce addictions and deaths related to pain killers?," Journal of Health Economics, Elsevier, vol. 58(C), pages 29-42.
    15. Aviva Aron-Dine & Liran Einav & Amy Finkelstein, 2013. "The RAND Health Insurance Experiment, Three Decades Later," Journal of Economic Perspectives, American Economic Association, vol. 27(1), pages 197-222, Winter.
    16. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    17. Thomas C. Buchmueller & Colleen Carey, 2018. "The Effect of Prescription Drug Monitoring Programs on Opioid Utilization in Medicare," American Economic Journal: Economic Policy, American Economic Association, vol. 10(1), pages 77-112, February.
    18. Jill Horwitz & Corey S. Davis & Lynn S. McClelland & Rebecca S. Fordon & Ellen Meara, 2018. "The Problem of Data Quality in Analyses of Opioid Regulation: The Case of Prescription Drug Monitoring Programs," NBER Working Papers 24947, National Bureau of Economic Research, Inc.
    19. Michael Clemens & Jennifer Hunt, 2017. "The Labor Market Effects of Refugee Waves: Reconciling Conflicting Results - Working Paper 455," Working Papers 455, Center for Global Development, revised 19 Jul 2017.
    20. Manning, Willard G, et al, 1987. "Health Insurance and the Demand for Medical Care: Evidence from a Randomized Experiment," American Economic Review, American Economic Association, vol. 77(3), pages 251-277, June.
    21. Abby Alpert & David Powell & Rosalie Liccardo Pacula, 2018. "Supply-Side Drug Policy in the Presence of Substitutes: Evidence from the Introduction of Abuse-Deterrent Opioids," American Economic Journal: Economic Policy, American Economic Association, vol. 10(4), pages 1-35, November.
    22. Leamer, Edward E, 1983. "Let's Take the Con Out of Econometrics," American Economic Review, American Economic Association, vol. 73(1), pages 31-43, March.
    23. Anne Case & Angus Deaton, 2017. "Mortality and Morbidity in the 21st Century," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 48(1 (Spring), pages 397-476.
    24. Ariella Kahn-Lang & Kevin Lang, 2020. "The Promise and Pitfalls of Differences-in-Differences: Reflections on 16 and Pregnant and Other Applications," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(3), pages 613-620, July.
    25. Amy Finkelstein, 2007. "The Aggregate Effects of Health Insurance: Evidence from the Introduction of Medicare," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(1), pages 1-37.
    26. Joshua D. Angrist & Alan B. Keueger, 1991. "Does Compulsory School Attendance Affect Schooling and Earnings?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 106(4), pages 979-1014.
    27. Drew B. Cameron & Anjini Mishra & Annette N. Brown, 2016. "The growth of impact evaluation for international development: how much have we learned?," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 8(1), pages 1-21, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jay Bhattacharya & Mikko Packalen, 2020. "Stagnation and Scientific Incentives," NBER Working Papers 26752, National Bureau of Economic Research, Inc.
    2. Eva Forslund & Magnus Henrekson, 2022. "The Virtues of Native Discourse: Striking a Balance Between English and the Native Language," Econ Journal Watch, Econ Journal Watch, vol. 19(2), pages 258–282-2, September.
    3. Ryan H Murphy, 2020. "Does democracy die in recessions? A descriptive analysis of aggregate demand shortfalls and regime transition," Economic Affairs, Wiley Blackwell, vol. 40(1), pages 63-76, February.
    4. Magnus Henrekson & Lars Jonung & Mats Lundahl, 2025. "Not just the top five journals: A recipe for European economists," Economic Affairs, Wiley Blackwell, vol. 45(1), pages 123-131, February.
    5. Skarbek, David, 2020. "Qualitative research methods for institutional analysis," Journal of Institutional Economics, Cambridge University Press, vol. 16(4), pages 409-422, August.
    6. Felix Pretis, 2022. "Does a Carbon Tax Reduce CO2 Emissions? Evidence from British Columbia," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 83(1), pages 115-144, September.
    7. John Mullahy, 2019. "Health and evidence in health economics," Health Economics, John Wiley & Sons, Ltd., vol. 28(10), pages 1163-1165, October.
    8. Richard J. Arend & Pankaj C. Patel, 2020. "The American Dream, Melting Pot And Regional Knowledge Stock As Drivers Of Entrepreneurial Activity," Journal of Developmental Entrepreneurship (JDE), World Scientific Publishing Co. Pte. Ltd., vol. 25(01), pages 1-35, March.
    9. Finigan, Duncan & Mills, Brian M. & Stone, Daniel F., 2020. "Pulling starters," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 89(C).
    10. Janna Bergsvik & Agnes Fauske & Rannveig Kaldager Hart, 2021. "Can Policies Stall the Fertility Fall? A Systematic Review of the (Quasi‐) Experimental Literature," Population and Development Review, The Population Council, Inc., vol. 47(4), pages 913-964, December.
    11. Preetika Joshi & Edmund Outslay & Anh Persson & Terry Shevlin & Aruhn Venkat, 2020. "Does Public Country‐by‐Country Reporting Deter Tax Avoidance and Income Shifting? Evidence from the European Banking Industry," Contemporary Accounting Research, John Wiley & Sons, vol. 37(4), pages 2357-2397, December.
    12. Robert A. Lawson & Ryan Murphy & Benjamin Powell, 2020. "The Determinants Of Economic Freedom: A Survey," Contemporary Economic Policy, Western Economic Association International, vol. 38(4), pages 622-642, October.
    13. Richardson, Eugene T. & Malik, Momin M. & Darity, William A. & Mullen, A. Kirsten & Morse, Michelle E. & Malik, Maya & Maybank, Aletha & Bassett, Mary T. & Farmer, Paul E. & Worden, Lee & Jones, James, 2021. "Reparations for Black American descendants of persons enslaved in the U.S. and their potential impact on SARS-CoV-2 transmission," Social Science & Medicine, Elsevier, vol. 276(C).
    14. Ali Sina Önder & Sergey V. Popov & Sascha Schweitzer, 2021. "Leadership in Scholarship: Editors’ Appointments and the Profession’s Narrative," Working Papers in Economics & Finance 2021-05, University of Portsmouth, Portsmouth Business School, Economics and Finance Subject Group.
    15. Reilly Barry & Sam Hannah, 2022. "The distributional impact of the Sierra Leone conflict on household welfare," IZA Journal of Development and Migration, Sciendo & Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 13(1), pages 1-41, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ruhm, Christopher J., 2019. "Drivers of the fatal drug epidemic," Journal of Health Economics, Elsevier, vol. 64(C), pages 25-42.
    2. David Beheshti, 2019. "Adverse health effects of abuse‐deterrent opioids: Evidence from the reformulation of OxyContin," Health Economics, John Wiley & Sons, Ltd., vol. 28(12), pages 1449-1461, December.
    3. Lucy Xiaolu Wang, 2021. "The complementarity of drug monitoring programs and health IT for reducing opioid‐related mortality and morbidity," Health Economics, John Wiley & Sons, Ltd., vol. 30(9), pages 2026-2046, September.
    4. Aliprantis, Dionissi & Fee, Kyle & Schweitzer, Mark E., 2023. "Opioids and the labor market," Labour Economics, Elsevier, vol. 85(C).
    5. Nicolas R. Ziebarth, 2018. "Social Insurance and Health," Contributions to Economic Analysis, in: Health Econometrics, volume 127, pages 57-84, Emerald Group Publishing Limited.
    6. Powell, David & Pacula, Rosalie Liccardo & Taylor, Erin, 2020. "How increasing medical access to opioids contributes to the opioid epidemic: Evidence from Medicare Part D," Journal of Health Economics, Elsevier, vol. 71(C).
    7. David Cho & Daniel I. García & Joshua Montes & Alison E. Weingarden, 2021. "Labor Market Effects of the Oxycodone-Heroin Epidemic," Finance and Economics Discussion Series 2021-025, Board of Governors of the Federal Reserve System (U.S.).
    8. Anda David & Charbel Nahas & Björn Nilsson & Mohamed Ali Marouani, 2018. "The economics of the Syrian refugee crisis in neighboring countries," Working Papers hal-04000224, HAL.
    9. Buckles, Kasey & Evans, William N. & Lieber, Ethan M.J., 2023. "The drug crisis and the living arrangements of children," Journal of Health Economics, Elsevier, vol. 87(C).
    10. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    11. Jason Anastasopoulos & George J. Borjas & Gavin G. Cook & Michael Lachanski, 2018. "Job Vacancies, the Beveridge Curve, and Supply Shocks: The Frequency and Content of Help-Wanted Ads in Pre- and Post-Mariel Miami," NBER Working Papers 24580, National Bureau of Economic Research, Inc.
    12. Guido W. Imbens, 2020. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," Journal of Economic Literature, American Economic Association, vol. 58(4), pages 1129-1179, December.
    13. Anda David & Mohamed Ali Marouani & Charbel Nahas & Björn Nilsson, 2020. "The economics of the Syrian refugee crisis in neighbouring countries: The case of Lebanon," Economics of Transition and Institutional Change, John Wiley & Sons, vol. 28(1), pages 89-109, January.
    14. Natali, Ilaria, 2024. "Economic Opportunity and Opioid Regulation: the Case of Codeine in France," TSE Working Papers 24-1563, Toulouse School of Economics (TSE).
    15. Mask Joshua, 2020. "Consequences of immigrating during a recession: Evidence from the US Refugee Resettlement program," IZA Journal of Development and Migration, Sciendo & Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 11(1), pages 1-31, January.
    16. Gal Wettstein, 2019. "Health insurance and opioid deaths: Evidence from the Affordable Care Act young adult provision," Health Economics, John Wiley & Sons, Ltd., vol. 28(5), pages 666-677, May.
    17. Boris Salazar-Trujillo & Daniel Otero Robles, 2019. "La revolución empírica en economía," Apuntes del Cenes, Universidad Pedagógica y Tecnológica de Colombia, vol. 38(68), pages 15-48, July.
    18. Casey B. Mulligan, 2020. "Prices and Federal Policies in Opioid Markets," Working Papers 2020-10, Becker Friedman Institute for Research In Economics.
    19. Billy, Alexander & Packard, Michael, 2022. "Crime and the Mariel Boatlift," International Review of Law and Economics, Elsevier, vol. 72(C).
    20. Pohl, R. Vincent, 2018. "Time Trends Matter: The Case of Medical Cannabis Laws and Opioid Overdose Mortality," MPRA Paper 88219, University Library of Munich, Germany.

    More about this item

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • B4 - Schools of Economic Thought and Methodology - - Economic Methodology
    • C50 - Mathematical and Quantitative Methods - - Econometric Modeling - - - General
    • H0 - Public Economics - - General
    • I0 - Health, Education, and Welfare - - General
    • J0 - Labor and Demographic Economics - - General
    • O1 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:25320. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.