IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp18741.html

Misleading Estimates from Nonlinear Models with a Binary Outcome

Author

Listed:
  • Curran, Brian

    (University of Chicago)

  • Meyer, Bruce

    (University of Chicago)

  • Wu, Derek

    (University of Virginia)

Abstract

When estimating nonlinear models for binary outcomes, such as probit and logit models, researchers often rely on average partial effects (APEs) to summarize the effect of a regressor. Because the marginal effect of a variable in these models depends on the values of all other variables, the value of an APE hinges on the portion of the sample used for the calculations. When averaged over parts of the sample drawn from a subpopulation not used to define the object of interest, the APE may be misleading. This paper highlights common situations, such as differences-in-means with a secondary group and difference-in-differences designs, where APEs calculated for the full sample deviate from marginal effects for the appropriate part of the sample. We propose a simple and costless solution in specific cases and demonstrate through simulations that recalculating APEs over the appropriate subsample yields unbiased results. Reexamining published results from multiple papers, we find statistically significant discrepancies between the reported estimates and the appropriately calculated APEs.

Suggested Citation

  • Curran, Brian & Meyer, Bruce & Wu, Derek, 2026. "Misleading Estimates from Nonlinear Models with a Binary Outcome," IZA Discussion Papers 18741, IZA Network @ LISER.
  • Handle: RePEc:iza:izadps:dp18741
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp18741.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Apesteguia, Jose & Funk, Patricia & Iriberri, Nagore, 2013. "Promoting rule compliance in daily-life: Evidence from a randomized field experiment in the public libraries of Barcelona," European Economic Review, Elsevier, vol. 64(C), pages 266-284.
    2. Jacob Bastian, 2020. "The Rise of Working Mothers and the 1975 Earned Income Tax Credit," American Economic Journal: Economic Policy, American Economic Association, vol. 12(3), pages 44-75, August.
    3. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769, December.
    4. Sandra Sequeira, 2016. "Corruption, Trade Costs, and Gains from Tariff Liberalization: Evidence from Southern Africa," American Economic Review, American Economic Association, vol. 106(10), pages 3029-3063, October.
    5. Melissa Hidrobo & Amber Peterman & Lori Heise, 2016. "The Effect of Cash, Vouchers, and Food Transfers on Intimate Partner Violence: Evidence from a Randomized Experiment in Northern Ecuador," American Economic Journal: Applied Economics, American Economic Association, vol. 8(3), pages 284-303, July.
    6. Jeffrey M Wooldridge, 2023. "Simple approaches to nonlinear difference-in-differences with panel data," The Econometrics Journal, Royal Economic Society, vol. 26(3), pages 31-66.
    7. Elira Kuka & Na'ama Shenhav & Kevin Shih, 2020. "Do Human Capital Decisions Respond to the Returns to Education? Evidence from DACA," American Economic Journal: Economic Policy, American Economic Association, vol. 12(1), pages 293-324, February.
    8. Luoto, Jill & Levine, David & Albert, Jeff & Luby, Stephen, 2014. "Nudging to use: Achieving safe water behaviors in Kenya and Bangladesh," Journal of Development Economics, Elsevier, vol. 110(C), pages 13-21.
    9. Puhani, Patrick A., 2012. "The treatment effect, the cross difference, and the interaction term in nonlinear “difference-in-differences” models," Economics Letters, Elsevier, vol. 115(1), pages 85-87.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gambardella, Alfonso & Conti, Raffaele & Novelli, Elena, 2018. "Specializing in Generality: Firm Strategies When Intermediate Markets Work," CEPR Discussion Papers 12782, Centre for Economic Policy Research.
    2. Shimizu, Hiroshi & Wakutsu, Naohiko, 2026. "Stimulating spin-outs and stalled subsequent technological development: Laser diodes in the United States and Japan under the SBIR program," Technological Forecasting and Social Change, Elsevier, vol. 223(C).
    3. Piotr R. Paradowski & Joanna Wolszczak-Derlacz, 2025. "Inequality, poverty, and child benefits: evidence from a natural experiment," Post-Soviet Affairs, Taylor & Francis Journals, vol. 41(3), pages 242-262, May.
    4. Masserini, Lucio & Bini, Matilde & Lorenzoni, Valentina, 2024. "The effect of pricing policies on students’ use of university canteens," Socio-Economic Planning Sciences, Elsevier, vol. 94(C).
    5. Robayo, Monica & Rude, Britta Laurin, 2023. "Preparatory School Years and Maternal Employment in Romania," Policy Research Working Paper Series 10638, The World Bank.
    6. Barkowski, Scott, 2021. "Interpretation of nonlinear difference-in-differences: the role of the parallel trends assumption," MPRA Paper 108975, University Library of Munich, Germany.
    7. Arne Henningsen & Guy Low & David Wuepper & Tobias Dalhaus & Hugo Storm & Dagim Belay & Stefan Hirsch, 2026. "Estimating Causal Effects With Observational Data: Guidelines for Agricultural and Applied Economists," Journal of Agricultural Economics, Wiley Blackwell, vol. 77(2), pages 356-382, June.
    8. Arne Risa Hole & Anita Ratcliffe, 2025. "Difference‐in‐differences with ordinal data: analysing the impact of terror attacks on racial harassment fears of UK Muslims," Fiscal Studies, John Wiley & Sons, vol. 46(4), pages 513-527, December.
    9. Brown, Caitlin & Tommasi, Denni, 2025. "Quality Upgrading in the Street Food Market: Is Better Equipment and Training Sufficient?," IZA Discussion Papers 18328, IZA Network @ LISER.
    10. Ksoll, Christopher & Lilleør, Helene Bie & Lønborg, Jonas Helth & Rasmussen, Ole Dahl, 2016. "Impact of Village Savings and Loan Associations: Evidence from a cluster randomized trial," Journal of Development Economics, Elsevier, vol. 120(C), pages 70-85.
    11. Eicher, Theo S. & Eskimez, Reina Kawai & Newiak, Monique, 2025. "Effects of IMF-supported programs on gender inequality," European Journal of Political Economy, Elsevier, vol. 90(PB).
    12. Joshua B. Gilbert & Zachary Himmelsbach & James Soland & Mridul Joshi & Benjamin W. Domingue, 2024. "Estimating Heterogeneous Treatment Effects with Item-Level Outcome Data: Insights from Item Response Theory," Papers 2405.00161, arXiv.org, revised Jan 2025.
    13. Gerard, Andrew & Lopez, Maria Claudia & Mason, Nicole M. & Bizoza, Alfred R., 2022. "Do government zoning policies improve buyer-farmer relationships? Evidence from Rwanda’s coffee sector," Food Policy, Elsevier, vol. 107(C).
    14. Michael Berlemann & Erik Haustein & Max F. Steinhardt & Jascha Tutt, 2026. "Do Natural Disasters Affect Household Saving? Evidence From the August 2002 Flood in Germany," Economics of Disasters and Climate Change, Springer, vol. 10(1), pages 1-42, April.
    15. Artuc, Erhan & Falcone, Guillermo & Porto, Guido & Rijkers, Bob, 2026. "Protectionism, evasion and household welfare evidence from Nigeria’s import bans," Journal of Public Economics, Elsevier, vol. 253(C).
    16. Rafi Amir‐ud‐Din & Muhammad Khan & Zahra Murad & Irene Mussio, 2025. "Clear Waters, Bright Futures: Do Low‐Cost Information Interventions Increase Health Preventive Behaviors," Health Economics, John Wiley & Sons, Ltd., vol. 34(9), pages 1578-1594, September.
    17. Andreas Kucher, 2019. "Vaccination recommendations and timeliness - The German measles case," Discussion Paper Series 336, Universitaet Augsburg, Institute for Economics.
    18. Myoung-jae Lee & Sanghyeok Lee, 2021. "Difference in Differences and Ratio in Ratios for Limited Dependent Variables," Papers 2111.12948, arXiv.org, revised Aug 2023.
    19. Ciani Emanuele & Fisher Paul, 2019. "Dif-in-Dif Estimators of Multiplicative Treatment Effects," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-10, January.
    20. E. Van de Poel & O. O'Donnell & E. Van Doorslaer, 2012. "Is there a health penalty of China's rapid urbanization?," Health Economics, John Wiley & Sons, Ltd., vol. 21(4), pages 367-385, April.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C25 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Discrete Regression and Qualitative Choice Models; Discrete Regressors; Proportions; Probabilities
    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp18741. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Mark Fallak (email available below). General contact details of provider: https://edirc.repec.org/data/izaaalu.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.