IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp18274.html

Understanding IV Versus OLS Estimates of Treatment Effects and the Coefficient Difference Check

Author

Listed:
  • Bjerk, David J.

    (Claremont McKenna College)

Abstract

This article derives an equation characterizing the difference between OLS and IV coefficients under potentially heterogenous treatment effects. This leads to what I call the Coefficient Difference Check, which consists of checking that the difference between the estimated OLS and IV coefficients has the same sign as the expected selection effect. I show failures of this check can arise because: IV is invalid, the expected selection story is incorrect, or there are particular heterogenous treatment effects that imply the IV estimate is both “fragile” and that it provides a more biased estimate of the ATT than OLS. Failures of this check are relatively common in the literature. I describe best practices given such failures.

Suggested Citation

  • Bjerk, David J., 2025. "Understanding IV Versus OLS Estimates of Treatment Effects and the Coefficient Difference Check," IZA Discussion Papers 18274, Institute of Labor Economics (IZA).
    • Handle: RePEc:iza:izadps:dp18274
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp18274.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Joseph J. Doyle Jr., 2007. "Child Protection and Child Outcomes: Measuring the Effects of Foster Care," American Economic Review, American Economic Association, vol. 97(5), pages 1583-1610, December.
    2. Ozkan Eren & Naci Mocan, 2021. "Juvenile Punishment, High School Graduation, and Adult Crime: Evidence from Idiosyncratic Judge Harshness," The Review of Economics and Statistics, MIT Press, vol. 103(1), pages 34-47, March.
    3. Samuel Norris & Matthew Pecenco & Jeffrey Weaver, 2021. "The Effects of Parental and Sibling Incarceration: Evidence from Ohio," American Economic Review, American Economic Association, vol. 111(9), pages 2926-2963, September.
    4. Max Gross & E. Jason Baron, 2022. "Temporary Stays and Persistent Gains: The Causal Effects of Foster Care," American Economic Journal: Applied Economics, American Economic Association, vol. 14(2), pages 170-199, April.
    5. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects, and Econometric Policy Evaluation," Econometrica, Econometric Society, vol. 73(3), pages 669-738, May.
    6. Martin Huber & Giovanni Mellace, 2015. "Testing Instrument Validity for LATE Identification Based on Inequality Moment Constraints," The Review of Economics and Statistics, MIT Press, vol. 97(2), pages 398-411, May.
    7. Black, Dan A. & Joo, Joonhwi & LaLonde, Robert & Smith, Jeffrey A. & Taylor, Evan J., 2022. "Simple Tests for Selection: Learning More from Instrumental Variables," Labour Economics, Elsevier, vol. 79(C).
    8. Eric Chyn & Brigham Frandsen & Emily Leslie, 2025. "Examiner and Judge Designs in Economics: A Practitioner's Guide," Journal of Economic Literature, American Economic Association, vol. 63(2), pages 401-439, June.
    9. Ismael Mourifié & Yuanyuan Wan, 2017. "Testing Local Average Treatment Effect Assumptions," The Review of Economics and Statistics, MIT Press, vol. 99(2), pages 305-313, May.
    10. Magne Mogstad & Andres Santos & Alexander Torgovitsky, 2018. "Using Instrumental Variables for Inference About Policy Relevant Treatment Parameters," Econometrica, Econometric Society, vol. 86(5), pages 1589-1619, September.
    11. Shoya Ishimaru, 2024. "Empirical Decomposition of the IV-OLS Gap with Heterogeneous and Nonlinear Effects," The Review of Economics and Statistics, MIT Press, vol. 106(2), pages 505-520, March.
    12. Nicole Maestas & Kathleen J. Mullen & Alexander Strand, 2013. "Does Disability Insurance Receipt Discourage Work? Using Examiner Assignment to Estimate Causal Effects of SSDI Receipt," American Economic Review, American Economic Association, vol. 103(5), pages 1797-1829, August.
    13. Will Dobbie & Jacob Goldin & Crystal S. Yang, 2018. "The Effects of Pretrial Detention on Conviction, Future Crime, and Employment: Evidence from Randomly Assigned Judges," American Economic Review, American Economic Association, vol. 108(2), pages 201-240, February.
    14. Anthony Bald & Eric Chyn & Justine Hastings & Margarita Machelett, 2022. "The Causal Impact of Removing Children from Abusive and Neglectful Homes," Journal of Political Economy, University of Chicago Press, vol. 130(7), pages 1919-1962.
    15. Emily Leslie & Nolan G. Pope, 2017. "The Unintended Impact of Pretrial Detention on Case Outcomes: Evidence from New York City Arraignments," Journal of Law and Economics, University of Chicago Press, vol. 60(3), pages 529-557.
    16. Card, David, 2001. "Estimating the Return to Schooling: Progress on Some Persistent Econometric Problems," Econometrica, Econometric Society, vol. 69(5), pages 1127-1160, September.
    17. Rafael Di Tella & Ernesto Schargrodsky, 2013. "Criminal Recidivism after Prison and Electronic Monitoring," Journal of Political Economy, University of Chicago Press, vol. 121(1), pages 28-73.
    18. Joseph J. Doyle Jr., 2008. "Child Protection and Adult Crime: Using Investigator Assignment to Estimate Causal Effects of Foster Care," Journal of Political Economy, University of Chicago Press, vol. 116(4), pages 746-770, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Johannes W. Ligtenberg & Tiemen Woutersen, 2024. "Multidimensional clustering in judge designs," Papers 2406.09473, arXiv.org.
    2. Mogstad, Magne & Torgovitsky, Alexander, 2024. "Instrumental variables with unobserved heterogeneity in treatment effects," Handbook of Labor Economics,, Elsevier.
    3. Mourifié, Ismael & Wan, Yuanyuan, 2025. "Layered policy analysis in program evaluation using the marginal treatment effect," Journal of Econometrics, Elsevier, vol. 251(C).
    4. Agan, Amanda & Doleac, Jennifer & Harvey, Anna, 2021. "Misdemeanor Prosecution," IZA Discussion Papers 14234, Institute of Labor Economics (IZA).
    5. Eric Chyn & Brigham Frandsen & Emily Leslie, 2025. "Examiner and Judge Designs in Economics: A Practitioner's Guide," Journal of Economic Literature, American Economic Association, vol. 63(2), pages 401-439, June.
    6. Carolina Arteaga, 2021. "Parental Incarceration and Children's Educational Attainment," Working Papers tecipa-703, University of Toronto, Department of Economics.
    7. Brigham Frandsen & Lars Lefgren & Emily Leslie, 2023. "Judging Judge Fixed Effects," American Economic Review, American Economic Association, vol. 113(1), pages 253-277, January.
    8. Manudeep Bhuller & Gordon B. Dahl & Katrine V. Løken & Magne Mogstad, 2020. "Incarceration, Recidivism, and Employment," Journal of Political Economy, University of Chicago Press, vol. 128(4), pages 1269-1324.
    9. Ivan A Canay & Magne Mogstad & Jack Mount, 2024. "On the Use of Outcome Tests for Detecting Bias in Decision Making," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 91(4), pages 2135-2167.
    10. Anthony Bald & Eric Chyn & Justine Hastings & Margarita Machelett, 2022. "The Causal Impact of Removing Children from Abusive and Neglectful Homes," Journal of Political Economy, University of Chicago Press, vol. 130(7), pages 1919-1962.
    11. Moffitt, Robert A. & Zahn, Matthew V., 2022. "The Marginal Labor Supply Disincentives of Welfare: Evidence from Administrative Barriers to Participation," Economics Working Paper Archive 66674, The Johns Hopkins University,Department of Economics.
    12. Sofia Amaral & Gordon B. Dahl & Victoria Endl-Geyer & Timo Hener & Helmut Rainer, 2023. "Deterrence or Backlash? Arrests and the Dynamics of Domestic Violence," NBER Working Papers 30855, National Bureau of Economic Research, Inc.
    13. E. Jason Baron & Ezra G. Goldstein & Joseph Ryan, 2023. "The Push for Racial Equity in Child Welfare: Can Blind Removals Reduce Disproportionality?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(2), pages 456-487, March.
    14. Manudeep Bhuller & Gordon B. Dahl & Katrine V. Løken & Magne Mogstad, 2018. "Incarceration Spillovers in Criminal and Family Networks," NBER Working Papers 24878, National Bureau of Economic Research, Inc.
    15. Michelle Yin & Garima Siwach & Dajun Lin, 2023. "Vocational Rehabilitation Services and Labor Market Outcomes for Transition‐Age Youth with Disabilities in Maine," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(1), pages 166-197, January.
    16. Robert Collinson & John Eric Humphries & Nicholas Mader & Davin Reed & Daniel Tannenbaum & Winnie van Dijk, 2024. "Eviction and Poverty in American Cities," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 139(1), pages 57-120.
    17. Vitor Possebom, 2021. "Crime and Mismeasured Punishment: Marginal Treatment Effect with Misclassification," Papers 2106.00536, arXiv.org, revised Jul 2023.
    18. Gonzalez-Uribe, Juanita & Reyes, Santiago, 2021. "Identifying and boosting “gazelles”: evidence from business accelerators," LSE Research Online Documents on Economics 103145, London School of Economics and Political Science, LSE Library.
    19. Cornelissen, Thomas & Dustmann, Christian & Raute, Anna & Schönberg, Uta, 2016. "From LATE to MTE: Alternative methods for the evaluation of policy interventions," Labour Economics, Elsevier, vol. 41(C), pages 47-60.
    20. Honigsberg, Colleen & Jacob, Matthew, 2021. "Deleting misconduct: The expungement of BrokerCheck records," Journal of Financial Economics, Elsevier, vol. 139(3), pages 800-831.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp18274. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.