IDEAS home Printed from https://ideas.repec.org/p/pad/wpaper/0305.html

Difference-In-Difference Design With Repeated Cross-Sections Under Compositional Changes: a Monte-Carlo Evaluation of Alternative Approaches

Author

Listed:
  • Tommaso Manfè

    (University of Chicago)

  • Luca Nunziata

    (University of Padova and IZA)

Abstract

We discuss the potentially severe bias in the commonly-used Difference-in-Difference estimators under compositional changes and propose a Double Inverse Probability Weighting estimator for repeated cross-sections based on both the probability of being treated and of belonging to the post-treatment period, deriving also its doubly-robust version. Through Monte Carlo simulations, we compare its performance with several methods suggested by the literature. Results show that our proposed estimator outperforms all alternatives in the most realistic scenarios. We provide an empirical application estimating the effect of tariff reduction on bribing behavior on trades data between South Africa and Mozambique during the period 2006–2014.

Suggested Citation

  • Tommaso Manfè & Luca Nunziata, 2023. "Difference-In-Difference Design With Repeated Cross-Sections Under Compositional Changes: a Monte-Carlo Evaluation of Alternative Approaches," "Marco Fanno" Working Papers 0305, Dipartimento di Scienze Economiche "Marco Fanno".
    • Handle: RePEc:pad:wpaper:0305
    as

    Download full text from publisher

    File URL: https://economia.unipd.it/sites/economia.unipd.it/files/20230305_rev1.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    2. Friedman, Jerome H. & Hastie, Trevor & Tibshirani, Rob, 2010. "Regularization Paths for Generalized Linear Models via Coordinate Descent," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 33(i01).
    3. Poterba, James M, 1987. "Tax Evasion and Capital Gains Taxation," American Economic Review, American Economic Association, vol. 77(2), pages 234-239, May.
    4. Sequeira, Sandra & Djankov, Simeon, 2014. "Corruption and firm behavior: Evidence from African ports," Journal of International Economics, Elsevier, vol. 94(2), pages 277-294.
    5. Sandra Sequeira, 2016. "Corruption, Trade Costs, and Gains from Tariff Liberalization: Evidence from Southern Africa," American Economic Review, American Economic Association, vol. 106(10), pages 3029-3063, October.
    6. Bryan S. Graham & Cristine Campos De Xavier Pinto & Daniel Egel, 2012. "Inverse Probability Tilting for Moment Condition Models with Missing Data," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 79(3), pages 1053-1079.
    7. Meyer, Bruce D, 1995. "Natural and Quasi-experiments in Economics," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(2), pages 151-161, April.
    8. Max H. Farrell & Tengyuan Liang & Sanjog Misra, 2021. "Deep Neural Networks for Estimation and Inference," Econometrica, Econometric Society, vol. 89(1), pages 181-213, January.
    9. Alberto Abadie, 2005. "Semiparametric Difference-in-Differences Estimators," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 72(1), pages 1-19.
    10. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2018. "Double/debiased machine learning for treatment and structural parameters," Econometrics Journal, Royal Economic Society, vol. 21(1), pages 1-68, February.
    11. Philipp Bach & Victor Chernozhukov & Malte S. Kurz & Martin Spindler, 2021. "DoubleML -- An Object-Oriented Implementation of Double Machine Learning in Python," Papers 2104.03220, arXiv.org, revised Dec 2021.
    12. Seung‐Hyun Hong, 2013. "Measuring The Effect Of Napster On Recorded Music Sales: Difference‐In‐Differences Estimates Under Compositional Changes," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(2), pages 297-324, March.
    13. Raymond Fisman & Shang-Jin Wei, 2004. "Tax Rates and Tax Evasion: Evidence from "Missing Imports" in China," Journal of Political Economy, University of Chicago Press, vol. 112(2), pages 471-500, April.
    14. Slemrod, Joel & Yitzhaki, Shlomo, 2002. "Tax avoidance, evasion, and administration," Handbook of Public Economics, in: A. J. Auerbach & M. Feldstein (ed.), Handbook of Public Economics, edition 1, volume 3, chapter 22, pages 1423-1470, Elsevier.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    2. Neng-Chieh Chang, 2018. "Semiparametric Difference-in-Differences with Potentially Many Control Variables," Papers 1812.10846, arXiv.org, revised Jan 2019.
    3. Pedro H. C. Sant'Anna & Qi Xu, 2023. "Difference-in-Differences with Compositional Changes," Papers 2304.13925, arXiv.org, revised Nov 2025.
    4. Kim, Bora & Lee, Myoung-jae, 2025. "Overlap-weighted difference-in-differences: A simple way to overcome poor propensity score overlap," Economics Letters, Elsevier, vol. 250(C).
    5. Martin Huber, 2019. "An introduction to flexible methods for policy evaluation," Papers 1910.00641, arXiv.org.
    6. Franziska Zimmert & Michael Zimmert, 2024. "Part‐time subsidies and maternal reemployment: Evidence from a difference‐in‐differences analysis," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 39(6), pages 1149-1171, September.
    7. Lin, Chin-Ho, 2018. "Tariff evasion in machinery production networks: Evidence from East Asia," Economic Modelling, Elsevier, vol. 70(C), pages 115-126.
    8. Heinisch, Katja & Scaramella, Fabio & Schult, Christoph, 2025. "Assumption errors and forecast accuracy: A partial linear instrumental variable and double machine learning approach," IWH Discussion Papers 6/2025, Halle Institute for Economic Research (IWH).
    9. Sina Akbari & Negar Kiyavash & AmirEmad Ghassami, 2025. "Semiparametric Triple Difference Estimators," Papers 2502.19788, arXiv.org, revised Sep 2025.
    10. Michael C Knaus & Michael Lechner & Anthony Strittmatter, 2021. "Machine learning estimation of heterogeneous causal effects: Empirical Monte Carlo evidence," The Econometrics Journal, Royal Economic Society, vol. 24(1), pages 134-161.
    11. Yang Ning & Sida Peng & Jing Tao, 2020. "Doubly Robust Semiparametric Difference-in-Differences Estimators with High-Dimensional Data," Papers 2009.03151, arXiv.org.
    12. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    13. Roth, Jonathan & Sant’Anna, Pedro H.C. & Bilinski, Alyssa & Poe, John, 2023. "What’s trending in difference-in-differences? A synthesis of the recent econometrics literature," Journal of Econometrics, Elsevier, vol. 235(2), pages 2218-2244.
    14. Christian Volpe Martincus & Jerónimo Carballo, 2012. "Export promotion activities in developing countries: What kind of trade do they promote?," The Journal of International Trade & Economic Development, Taylor & Francis Journals, vol. 21(4), pages 539-578, June.
    15. Zara Liaqat, Karrar Hussain, 2025. "Costly Imports: Exchange rate shocks and trade misreporting by firms," LCERPA Working Papers jc0154, Laurier Centre for Economic Research and Policy Analysis, revised 2025.
    16. Fang, Yan & Liu, Yinglin & Yang, Yi & Lucey, Brian & Abedin, Mohammad Zoynul, 2025. "How do Chinese urban investment bonds affect its economic resilience? Evidence from double machine learning," Research in International Business and Finance, Elsevier, vol. 74(C).
    17. Christian Volpe Martincus, 2010. "Odyssey in International Markets: An Assessment of the Effectiveness of Export Promotion in Latin America and the Caribbean," IDB Publications (Books), Inter-American Development Bank, number 16458, February.
    18. Mazzutti, Caio Cícero Toledo Piza da Costa, . "Three essays on the causal impacts of child labour laws in Brazil," Economics PhD Theses, Department of Economics, University of Sussex Business School, number 0616, December.
    19. Jing Kong, 2025. "Causal Inference in High-Dimensional Generalized Linear Models with Binary Outcomes," Papers 2510.16669, arXiv.org.
    20. Mattia Filomena & Matteo Picchio, 2025. "You’ll never walk alone: unemployment, social networks and leisure activities," Oxford Economic Papers, Oxford University Press, vol. 77(4), pages 1062-1079.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pad:wpaper:0305. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Raffaele Dei Campielisi (email available below). General contact details of provider: https://edirc.repec.org/data/dspadit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.