IDEAS home Printed from https://ideas.repec.org/a/taf/jnlasa/v118y2023i544p2833-2846.html
   My bibliography  Save this article

Permutation Tests at Nonparametric Rates

Author

Listed:
  • Marinho Bertanha
  • Eunyi Chung

Abstract

Classical two-sample permutation tests for equality of distributions have exact size in finite samples, but they fail to control size for testing equality of parameters that summarize each distribution. This article proposes permutation tests for equality of parameters that are estimated at root-n or slower rates. Our general framework applies to both parametric and nonparametric models, with two samples or one sample split into two subsamples. Our tests have correct size asymptotically while preserving exact size in finite samples when distributions are equal. They have no loss in local asymptotic power compared to tests that use asymptotic critical values. We propose confidence sets with correct coverage in large samples that also have exact coverage in finite samples if distributions are equal up to a transformation. We apply our theory to four commonly-used hypothesis tests of nonparametric functions evaluated at a point. Lastly, simulations show good finite sample properties, and two empirical examples illustrate our tests in practice. Supplementary materials for this article are available online.

Suggested Citation

  • Marinho Bertanha & Eunyi Chung, 2023. "Permutation Tests at Nonparametric Rates," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 118(544), pages 2833-2846, October.
  • Handle: RePEc:taf:jnlasa:v:118:y:2023:i:544:p:2833-2846
    DOI: 10.1080/01621459.2022.2087660
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/01621459.2022.2087660
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1080/01621459.2022.2087660?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Kamat, Vishal, 2018. "On Nonparametric Inference In The Regression Discontinuity Design," Econometric Theory, Cambridge University Press, vol. 34(3), pages 694-703, June.
    2. Bugni, Federico A. & Canay, Ivan A., 2021. "Testing continuity of a density via g-order statistics in the regression discontinuity design," Journal of Econometrics, Elsevier, vol. 221(1), pages 138-159.
    3. Leonardo Bursztyn & Georgy Egorov & Robert Jensen, 2019. "Cool to be Smart or Smart to be Cool? Understanding Peer Pressure in Education," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 86(4), pages 1487-1526.
    4. Carolina Caetano, 2015. "A Test of Exogeneity Without Instrumental Variables in Models With Bunching," Econometrica, Econometric Society, vol. 83(4), pages 1581-1600, July.
    5. Bertanha, Marinho & Moreira, Marcelo J., 2020. "Impossible inference in econometrics: Theory and applications," Journal of Econometrics, Elsevier, vol. 218(2), pages 247-270.
    6. Abou-Chadi, Tarik & Krause, Werner, 2020. "The Causal Effect of Radical Right Success on Mainstream Parties’ Policy Positions: A Regression Discontinuity Approach," British Journal of Political Science, Cambridge University Press, vol. 50(3), pages 829-847, July.
    7. Markus Pauly & Edgar Brunner & Frank Konietschke, 2015. "Asymptotic permutation tests in general factorial designs," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 77(2), pages 461-473, March.
    8. Scott Cunningham & Manisha Shah, 2018. "Decriminalizing Indoor Prostitution: Implications for Sexual Violence and Public Health," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 85(3), pages 1683-1715.
    9. Ivan A Canay & Vishal Kamat, 2018. "Approximate Permutation Tests and Induced Order Statistics in the Regression Discontinuity Design," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 85(3), pages 1577-1608.
    10. Emmanuel Saez, 2010. "Do Taxpayers Bunch at Kink Points?," American Economic Journal: Economic Policy, American Economic Association, vol. 2(3), pages 180-212, August.
    11. Guido W. Imbens & Paul R. Rosenbaum, 2005. "Robust, accurate confidence intervals with a weak instrument: quarter of birth and education," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(1), pages 109-126, January.
    12. Neubert, Karin & Brunner, Edgar, 2007. "A studentized permutation test for the non-parametric Behrens-Fisher problem," Computational Statistics & Data Analysis, Elsevier, vol. 51(10), pages 5192-5204, June.
    13. Gautam Rao, 2019. "Familiarity Does Not Breed Contempt: Generosity, Discrimination, and Diversity in Delhi Schools," American Economic Review, American Economic Association, vol. 109(3), pages 774-809, March.
    14. Roberto Ramos & Carlos Sanz, 2018. "Backing the incumbent in difficult times: the electoral impact of wildfires," Working Papers 1810, Banco de España.
    15. Hahn, Jinyong & Todd, Petra & Van der Klaauw, Wilbert, 2001. "Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design," Econometrica, Econometric Society, vol. 69(1), pages 201-209, January.
    16. Guido Imbens & Karthik Kalyanaraman, 2012. "Optimal Bandwidth Choice for the Regression Discontinuity Estimator," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 79(3), pages 933-959.
    17. Alexander Bick & Nicola Fuchs-Schündeln & David Lagakos, 2018. "How Do Hours Worked Vary with Income? Cross-Country Evidence and Implications," American Economic Review, American Economic Association, vol. 108(1), pages 170-199, January.
    18. Abou-Chadi, Tarik & Krause, Werner, 2020. "The Causal Effect of Radical Right Success on Mainstream Parties’ Policy Positions: A Regression Discontinuity Approach," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 50(3), pages 829-847.
    19. Sule Alan & Teodora Boneva & Seda Ertac, 2019. "Ever Failed, Try Again, Succeed Better: Results from a Randomized Educational Intervention on Grit," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(3), pages 1121-1162.
    20. Pollard, David, 1991. "Asymptotics for Least Absolute Deviation Regression Estimators," Econometric Theory, Cambridge University Press, vol. 7(2), pages 186-199, June.
    21. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bertanha, Marinho & McCallum, Andrew H. & Seegert, Nathan, 2023. "Better bunching, nicer notching," Journal of Econometrics, Elsevier, vol. 237(2).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Blaise Melly & Rafael Lalive, 2020. "Estimation, Inference, and Interpretation in the Regression Discontinuity Design," Diskussionsschriften dp2016, Universitaet Bern, Departement Volkswirtschaft.
    2. Bertanha, Marinho & Moreira, Marcelo J., 2020. "Impossible inference in econometrics: Theory and applications," Journal of Econometrics, Elsevier, vol. 218(2), pages 247-270.
    3. Matias D. Cattaneo & Rocío Titiunik, 2022. "Regression Discontinuity Designs," Annual Review of Economics, Annual Reviews, vol. 14(1), pages 821-851, August.
    4. Atı̇la Abdulkadı̇roğlu & Joshua D. Angrist & Yusuke Narita & Parag Pathak, 2022. "Breaking Ties: Regression Discontinuity Design Meets Market Design," Econometrica, Econometric Society, vol. 90(1), pages 117-151, January.
    5. Yoichi Arai & Yu‐Chin Hsu & Toru Kitagawa & Ismael Mourifié & Yuanyuan Wan, 2022. "Testing identifying assumptions in fuzzy regression discontinuity designs," Quantitative Economics, Econometric Society, vol. 13(1), pages 1-28, January.
    6. Bugni, Federico A. & Canay, Ivan A., 2021. "Testing continuity of a density via g-order statistics in the regression discontinuity design," Journal of Econometrics, Elsevier, vol. 221(1), pages 138-159.
    7. Yingying DONG & Ying-Ying LEE & Michael GOU, 2019. "Regression Discontinuity Designs with a Continuous Treatment," Discussion papers 19058, Research Institute of Economy, Trade and Industry (RIETI).
    8. Bertanha, Marinho Angelo & Moreira, Marcelo J., 2017. "Impossible inference in econometrics: theory and applications to regression discontinuity, bunching, and exogeneity tests," FGV EPGE Economics Working Papers (Ensaios Economicos da EPGE) 787, EPGE Brazilian School of Economics and Finance - FGV EPGE (Brazil).
    9. Adam C. Sales & Ben B. Hansen, 2020. "Limitless Regression Discontinuity," Journal of Educational and Behavioral Statistics, , vol. 45(2), pages 143-174, April.
    10. David Card & David S. Lee & Zhuan Pei & Andrea Weber, 2015. "Inference on Causal Effects in a Generalized Regression Kink Design," Econometrica, Econometric Society, vol. 83, pages 2453-2483, November.
    11. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    12. Purevdorj Tuvaandorj, 2021. "Robust Permutation Tests in Linear Instrumental Variables Regression," Papers 2111.13774, arXiv.org, revised Jun 2023.
    13. Babii, Andrii & Kumar, Rohit, 2023. "Isotonic regression discontinuity designs," Journal of Econometrics, Elsevier, vol. 234(2), pages 371-393.
    14. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    15. Takuya Ishihara & Masayuki Sawada, 2020. "Manipulation-Robust Regression Discontinuity Designs," Papers 2009.07551, arXiv.org, revised Oct 2023.
    16. Timothy B. Armstrong & Michal Kolesár, 2020. "Simple and honest confidence intervals in nonparametric regression," Quantitative Economics, Econometric Society, vol. 11(1), pages 1-39, January.
    17. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    18. Yusuke Narita & Kohei Yata, 2021. "Algorithm is Experiment: Machine Learning, Market Design, and Policy Eligibility Rules," Working Papers 2021-022, Human Capital and Economic Opportunity Working Group.
    19. Bertanha, Marinho & McCallum, Andrew H. & Seegert, Nathan, 2023. "Better bunching, nicer notching," Journal of Econometrics, Elsevier, vol. 237(2).
    20. Anna Alberini & Markus Bareit & Massimo Filippini, 2016. "What is the Effect of Fuel Efficiency Information on Car Prices? Evidence from Switzerland," The Energy Journal, , vol. 37(3), pages 315-342, July.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:jnlasa:v:118:y:2023:i:544:p:2833-2846. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/UASA20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.