IDEAS home Printed from
   My bibliography  Save this paper

The Promise and Pitfalls of Differences-in-Differences: Reflections on ‘16 and Pregnant’ and Other Applications


  • Ariella Kahn-Lang
  • Kevin Lang


We use the exchange between Kearney/Levine and Jaeger/Joyce/Kaestner on “16 and Pregnant” to reexamine the use of DiD as a response to the failure of nature to properly design an experiment for us. We argue that 1) any DiD paper should address why the original levels of the experimental and control groups differed, and why this would not impact trends, 2) the parallel trends argument requires a justification of the chosen functional form and that the use of the interaction coefficients in probit and logit may be justified in some cases, and 3) parallel trends in the period prior to treatment is suggestive of counterfactual parallel trends, but parallel pre-trends is neither necessary nor sufficient for the parallel counterfactual trends condition to hold. Importantly, the purely statistical approach uses pretesting and thus generates the wrong standard errors. Moreover, we underline the dangers of implicitly or explicitly accepting the null hypothesis when failing to reject the absence of a differential pre-trend.

Suggested Citation

  • Ariella Kahn-Lang & Kevin Lang, 2018. "The Promise and Pitfalls of Differences-in-Differences: Reflections on ‘16 and Pregnant’ and Other Applications," NBER Working Papers 24857, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:24857
    Note: CH LS

    Download full text from publisher

    File URL:
    Download Restriction: no

    Other versions of this item:


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Eli Ben-Michael & Avi Feller & Jesse Rothstein, 2019. "Synthetic Controls with Staggered Adoption," Papers 1912.03290,, revised Jan 2021.
    2. Marco Calaresu & Moris Triventi, 2019. "Governing by contract as a way to reduce crime? An impact evaluation of the large-scale policy of security pacts," Policy Sciences, Springer;Society of Policy Sciences, vol. 52(2), pages 255-279, June.
    3. Mari, Gabriele, 2020. "Working-time flexibility is (not the same) for all: Evidence from a right-to-request reform," SocArXiv bnp9r, Center for Open Science.
    4. Aksoy, Cevat G. & Carpenter, Christopher S. & De Haas, Ralph & Tran, Kevin D., 2020. "Do laws shape attitudes? Evidence from same-sex relationship recognition policies in Europe," European Economic Review, Elsevier, vol. 124(C).
    5. Christopher J. Ruhm, 2019. "Shackling the Identification Police?," Southern Economic Journal, John Wiley & Sons, vol. 85(4), pages 1016-1026, April.
    6. Gathmann, Christina & Huttunen, Kristiina & Jernström, Laura & Sääksvuori, Lauri & Stitzing, Robin, 2020. "In Sickness and in Health: Job Displacement and Health Spillovers in Couples," IZA Discussion Papers 13329, Institute of Labor Economics (IZA).
    7. Sansone, Dario, 2019. "Pink work: Same-sex marriage, employment and discrimination," Journal of Public Economics, Elsevier, vol. 180(C).
    8. Byard, Donal & Darrough, Masako & Suh, Jangwon, 2019. "There is No Evidence that Mandatory IFRS Adoption Significantly Decreased IPO Underpricing," SocArXiv b56u2, Center for Open Science.
    9. Grossman, Daniel & Khalil, Umair & Ray, Arijit, 2019. "Terrorism and early childhood health outcomes: Evidence from Pakistan," Social Science & Medicine, Elsevier, vol. 237(C), pages 1-1.
    10. Clarke, Damian & Schythe, Kathya Tapia, 2020. "Implementing the Panel Event Study," IZA Discussion Papers 13524, Institute of Labor Economics (IZA).
    11. Michał Myck & Mateusz Najsztub, 2020. "Implications of the Polish 1999 administrative reform for regional socio‐economic development," Economics of Transition and Institutional Change, John Wiley & Sons, vol. 28(4), pages 559-579, October.
    12. Sujeong Park & David Powell, 2020. "Is the Rise in Illicit Opioids Affecting Labor Supply and Disability Claiming Rates?," NBER Working Papers 27804, National Bureau of Economic Research, Inc.
    13. Leduc, Elisabeth & Tojerow, Ilan, 2020. "Subsidizing Domestic Services as a Tool to Fight Unemployment: Effectiveness and Hidden Costs," IZA Discussion Papers 13544, Institute of Labor Economics (IZA).
    14. Gianmarco Daniele & Gemma Dipoppa, 2018. "Doing Business Below the Line: Screening, Mafias and Public Funds," BAFFI CAREFIN Working Papers 1898, BAFFI CAREFIN, Centre for Applied Research on International Markets Banking Finance and Regulation, Universita' Bocconi, Milano, Italy.
    15. Ismael Sanz & J .D. Tenaa, 2020. "A Natural Experiment on the Effect of Instruction Time and Quality: Lessons for the Covid-19 Outbreak," Working Papers 202032, University of Liverpool, Department of Economics.
    16. Christopher Carpenter & Dario Sansone, 2020. "Cigarette Taxes and Smoking Among Sexual Minority Adults," NBER Working Papers 26692, National Bureau of Economic Research, Inc.
    17. Brian Beach & W. Walker Hanlon, 2019. "Censorship, Family Planning, and the Historical Fertility Transition," NBER Working Papers 25752, National Bureau of Economic Research, Inc.
    18. Deepak Varshney & Devesh Roy & J. V. Meenakshi, 2020. "Impact of COVID-19 on agricultural markets: assessing the roles of commodity characteristics, disease caseload and market reforms," Indian Economic Review, Springer, vol. 55(1), pages 83-103, November.
    19. Ben Brunckhorst, 2020. "Rural Mobility and Climate Vulnerability: Evidence from the 2015 Drought in Ethiopia," CSAE Working Paper Series 2020-17, Centre for the Study of African Economies, University of Oxford.
    20. Sonia Bhalotra & Damian Clarke & Joseph Gomes & Atheendar Venkataramani, 2018. "Maternal mortality and women's political participation," WIDER Working Paper Series wp-2018-58, World Institute for Development Economic Research (UNU-WIDER).
    21. Cook, Jason B., 2018. "Race-Blind Admissions, School Segregation, and Student Outcomes: Evidence from Race-Blind Magnet School Lotteries," IZA Discussion Papers 11909, Institute of Labor Economics (IZA).
    22. Cenjie Liu & Chunbo Ma & Rui Xie, 2020. "Structural, Innovation and Efficiency Effects of Environmental Regulation: Evidence from China’s Carbon Emissions Trading Pilot," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 75(4), pages 741-768, April.
    23. von Haaren, Paula & Klonner, Stefan, 2020. "Maternal cash for better child health? The impacts of India’s IGMSY/PMMVY maternity benefit scheme," Working Papers 0689, University of Heidelberg, Department of Economics.
    24. Yuval Mazar, 2018. "The effect of child allowances on the labor supply: Evidence from the early 2000s," Bank of Israel Working Papers 2018.07, Bank of Israel.
    25. Jason M. Lindo & Isaac D. Swensen & Glen R. Waddell, 2020. "Persistent Effects of Violent Media Content," NBER Working Papers 27240, National Bureau of Economic Research, Inc.

    More about this item

    JEL classification:

    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • J13 - Labor and Demographic Economics - - Demographic Economics - - - Fertility; Family Planning; Child Care; Children; Youth

    NEP fields

    This paper has been announced in the following NEP Reports:


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:24857. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.