IDEAS home Printed from https://ideas.repec.org/a/eee/eecrev/v163y2024ics0014292123002726.html
   My bibliography  Save this article

Asymptotically robust permutation-based randomization confidence intervals for parametric OLS regression

Author

Listed:
  • Young, Alwyn

Abstract

Randomization inference provides exact finite sample tests of sharp null hypotheses which fully specify the distribution of outcomes under counterfactual realizations of treatment, but the sharp null is often considered restrictive as it rules out unspecified heterogeneity in treatment response. However, a growing literature shows that tests based upon permutations of regressors using pivotal statistics can remain asymptotically valid when the assumption regarding the permutation invariance of the data generating process used to motivate them is actually false. For experiments where potential outcomes involve the permutation of regressors, these results show that permutation-based randomization inference, while providing exact tests of sharp nulls, can also have the same asymptotic validity as conventional tests of average treatment effects with unspecified heterogeneity and other forms of specification error in treatment response. This paper extends this work to the consideration of interactions between treatment variables and covariates, a common feature of published regressions, as well as issues in the construction of confidence intervals and testing of subsets of treatment effects.

Suggested Citation

  • Young, Alwyn, 2024. "Asymptotically robust permutation-based randomization confidence intervals for parametric OLS regression," European Economic Review, Elsevier, vol. 163(C).
  • Handle: RePEc:eee:eecrev:v:163:y:2024:i:c:s0014292123002726
    DOI: 10.1016/j.euroecorev.2023.104644
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0014292123002726
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.euroecorev.2023.104644?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lihua Lei & Peter J Bickel, 2021. "An assumption-free exact test for fixed-design linear models with exchangeable errors [Rank tests of sub-hypotheses in the general linear regression]," Biometrika, Biometrika Trust, vol. 108(2), pages 397-412.
    2. Lori Beaman & Jeremy Magruder, 2012. "Who Gets the Job Referral? Evidence from a Social Networks Experiment," American Economic Review, American Economic Association, vol. 102(7), pages 3574-3593, December.
    3. Federico A. Bugni & Ivan A. Canay & Azeem M. Shaikh, 2018. "Inference Under Covariate-Adaptive Randomization," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(524), pages 1784-1796, October.
    4. Jesse Hemerik & Jelle J. Goeman, 2021. "Another Look at the Lady Tasting Tea and Differences Between Permutation Tests and Randomisation Tests," International Statistical Review, International Statistical Institute, vol. 89(2), pages 367-381, August.
    5. Esther Duflo & Pascaline Dupas & Michael Kremer, 2011. "Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya," American Economic Review, American Economic Association, vol. 101(5), pages 1739-1774, August.
    6. Jonathan Robinson, 2012. "Limited Insurance within the Household: Evidence from a Field Experiment in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 4(4), pages 140-164, October.
    7. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    8. Pascaline Dupas & Jonathan Robinson, 2013. "Savings Constraints and Microenterprise Development: Evidence from a Field Experiment in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 5(1), pages 163-192, January.
    9. Jessica Wisdom & Julie S. Downs & George Loewenstein, 2010. "Promoting Healthy Choices: Information versus Convenience," American Economic Journal: Applied Economics, American Economic Association, vol. 2(2), pages 164-178, April.
    10. Chung, EunYi & Romano, Joseph P., 2016. "Multivariate and multiple permutation tests," Journal of Econometrics, Elsevier, vol. 193(1), pages 76-91.
    11. Nava Ashraf & James Berry & Jesse M. Shapiro, 2010. "Can Higher Prices Stimulate Product Use? Evidence from a Field Experiment in Zambia," American Economic Review, American Economic Association, vol. 100(5), pages 2383-2413, December.
    12. Federico A. Bugni & Ivan A. Canay & Azeem M. Shaikh, 2019. "Inference under covariate‐adaptive randomization with multiple treatments," Quantitative Economics, Econometric Society, vol. 10(4), pages 1747-1785, November.
    13. Hongbin Cai & Yuyu Chen & Hanming Fang, 2009. "Observational Learning: Evidence from a Randomized Natural Field Experiment," American Economic Review, American Economic Association, vol. 99(3), pages 864-882, June.
    14. Shawn Cole & Xavier Gine & Jeremy Tobacman & Petia Topalova & Robert Townsend & James Vickery, 2013. "Barriers to Household Risk Management: Evidence from India," American Economic Journal: Applied Economics, American Economic Association, vol. 5(1), pages 104-135, January.
    15. Emily Oster & Rebecca Thornton, 2011. "Menstruation, Sanitary Products, and School Attendance: Evidence from a Randomized Evaluation," American Economic Journal: Applied Economics, American Economic Association, vol. 3(1), pages 91-100, January.
    16. Rebecca L. Thornton, 2008. "The Demand for, and Impact of, Learning HIV Status," American Economic Review, American Economic Association, vol. 98(5), pages 1829-1863, December.
    17. Janssen, Arnold, 1997. "Studentized permutation tests for non-i.i.d. hypotheses and the generalized Behrens-Fisher problem," Statistics & Probability Letters, Elsevier, vol. 36(1), pages 9-21, November.
    18. Christina M. Fong & Erzo F. P. Luttmer, 2009. "What Determines Giving to Hurricane Katrina Victims? Experimental Evidence on Racial Group Loyalty," American Economic Journal: Applied Economics, American Economic Association, vol. 1(2), pages 64-87, April.
    19. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2020. "Sampling‐Based versus Design‐Based Uncertainty in Regression Analysis," Econometrica, Econometric Society, vol. 88(1), pages 265-296, January.
    20. Jenny C. Aker & Christopher Ksoll & Travis J. Lybbert, 2012. "Can Mobile Phones Improve Learning? Evidence from a Field Experiment in Niger," American Economic Journal: Applied Economics, American Economic Association, vol. 4(4), pages 94-120, October.
    21. Cyrus J. DiCiccio & Joseph P. Romano, 2017. "Robust Permutation Tests For Correlation And Regression Coefficients," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 1211-1220, July.
    22. Sebastian Galiani & Martín A. Rossi & Ernesto Schargrodsky, 2011. "Conscription and Crime: Evidence from the Argentine Draft Lottery," American Economic Journal: Applied Economics, American Economic Association, vol. 3(2), pages 119-136, April.
    23. Chung, EunYi & Olivares, Mauricio, 2021. "Permutation test for heterogeneous treatment effects with a nuisance parameter," Journal of Econometrics, Elsevier, vol. 225(2), pages 148-174.
    24. Chesher, Andrew, 1989. "Hajek Inequalities, Measures of Leverage and the Size of Heteroskedasticity Robust Wald Tests," Econometrica, Econometric Society, vol. 57(4), pages 971-977, July.
    25. Joshua Angrist & Victor Lavy, 2009. "The Effects of High Stakes High School Achievement Awards: Evidence from a Randomized Trial," American Economic Review, American Economic Association, vol. 99(4), pages 1384-1414, September.
    26. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    27. Alwyn Young, 2019. "Channeling Fisher: Randomization Tests and the Statistical Insignificance of Seemingly Significant Experimental Results," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(2), pages 557-598.
    28. Ben O'Neill, 2009. "Exchangeability, Correlation, and Bayes' Effect," International Statistical Review, International Statistical Institute, vol. 77(2), pages 241-250, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Young, Alwyn, 2024. "Asymptotically robust permutation-based randomization confidence intervals for parametric OLS regression," LSE Research Online Documents on Economics 120933, London School of Economics and Political Science, LSE Library.
    2. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
    3. Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.
    4. Belot, Michèle & James, Jonathan, 2016. "Partner selection into policy relevant field experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 123(C), pages 31-56.
    5. Yuehao Bai & Joseph P. Romano & Azeem M. Shaikh, 2022. "Inference in Experiments With Matched Pairs," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 117(540), pages 1726-1737, October.
    6. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    7. Bugni, Federico A. & Gao, Mengsi, 2023. "Inference under covariate-adaptive randomization with imperfect compliance," Journal of Econometrics, Elsevier, vol. 237(1).
    8. Young, Alwyn, 2019. "Channeling Fisher: randomization tests and the statistical insignificance of seemingly significant experimental results," LSE Research Online Documents on Economics 101401, London School of Economics and Political Science, LSE Library.
    9. Ganesh Karapakula, 2023. "Stable Probability Weighting: Large-Sample and Finite-Sample Estimation and Inference Methods for Heterogeneous Causal Effects of Multivalued Treatments Under Limited Overlap," Papers 2301.05703, arXiv.org, revised Jan 2023.
    10. Yuehao Bai, 2022. "Optimality of Matched-Pair Designs in Randomized Controlled Trials," Papers 2206.07845, arXiv.org.
    11. Hidalgo, Diana & Onofa, Mercedes & Oosterbeek, Hessel & Ponce, Juan, 2013. "Can provision of free school uniforms harm attendance? Evidence from Ecuador," Journal of Development Economics, Elsevier, vol. 103(C), pages 43-51.
    12. Purevdorj Tuvaandorj, 2021. "Robust Permutation Tests in Linear Instrumental Variables Regression," Papers 2111.13774, arXiv.org, revised Jun 2023.
    13. G�nther Fink & Margaret McConnell & Sebastian Vollmer, 2014. "Testing for heterogeneous treatment effects in experimental data: false discovery risks and correction procedures," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(1), pages 44-57, January.
    14. Federico A. Bugni & Ivan A. Canay & Azeem M. Shaikh, 2018. "Inference Under Covariate-Adaptive Randomization," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(524), pages 1784-1796, October.
    15. Karthik Muralidharan & Mauricio Romero & Kaspar Wüthrich, 2019. "Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments," NBER Working Papers 26562, National Bureau of Economic Research, Inc.
    16. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2022. "Experimental evidence on learning using low-tech when school is out," Nature Human Behaviour, Nature, vol. 6(7), pages 941-950, July.
    17. Allen, James & Mahumane, Arlete & Riddell, James & Rosenblat, Tanya & Yang, Dean & Yu, Hang, 2022. "Teaching and incentives: Substitutes or complements?," Economics of Education Review, Elsevier, vol. 91(C).
    18. James J Heckman & Ganesh Karapakula, 2021. "Using a satisficing model of experimenter decision-making to guide finite-sample inference for compromised experiments," The Econometrics Journal, Royal Economic Society, vol. 24(2), pages 1-39.
    19. Liang Jiang & Xiaobin Liu & Peter C. B. Phillips & Yichong Zhang, 2024. "Bootstrap Inference for Quantile Treatment Effects in Randomized Experiments with Matched Pairs," The Review of Economics and Statistics, MIT Press, vol. 106(2), pages 542-556, March.
    20. Federico A. Bugni & Mengsi Gao, 2021. "Inference under Covariate-Adaptive Randomization with Imperfect Compliance," Papers 2102.03937, arXiv.org, revised Jul 2023.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:eecrev:v:163:y:2024:i:c:s0014292123002726. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eer .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.