IDEAS home Printed from https://ideas.repec.org/a/taf/jnlasa/v112y2017i519p1211-1220.html
   My bibliography  Save this article

Robust Permutation Tests For Correlation And Regression Coefficients

Author

Listed:
  • Cyrus J. DiCiccio
  • Joseph P. Romano

Abstract

Given a sample from a bivariate distribution, consider the problem of testing independence. A permutation test based on the sample correlation is known to be an exact level α test. However, when used to test the null hypothesis that the samples are uncorrelated, the permutation test can have rejection probability that is far from the nominal level. Further, the permutation test can have a large Type 3 (directional) error rate, whereby there can be a large probability that the permutation test rejects because the sample correlation is a large positive value, when in fact the true correlation is negative. It will be shown that studentizing the sample correlation leads to a permutation test which is exact under independence and asymptotically controls the probability of Type 1 (or Type 3) errors. These conclusions are based on our results describing the almost sure limiting behavior of the randomization distribution. We will also present asymptotically robust randomization tests for regression coefficients, including a result based on a modified procedure of Freedman and Lane. Simulations and empirical applications are included. Supplementary materials for this article are available online.

Suggested Citation

  • Cyrus J. DiCiccio & Joseph P. Romano, 2017. "Robust Permutation Tests For Correlation And Regression Coefficients," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 1211-1220, July.
    • Handle: RePEc:taf:jnlasa:v:112:y:2017:i:519:p:1211-1220
    DOI: 10.1080/01621459.2016.1202117
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/01621459.2016.1202117
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/01621459.2016.1202117?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Freedman, David & Lane, David, 1983. "A Nonstochastic Interpretation of Reported Significance Levels," Journal of Business & Economic Statistics, American Statistical Association, vol. 1(4), pages 292-298, October.
    2. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    3. Janssen, Arnold, 1997. "Studentized permutation tests for non-i.i.d. hypotheses and the generalized Behrens-Fisher problem," Statistics & Probability Letters, Elsevier, vol. 36(1), pages 9-21, November.
    4. Arnold Janssen, 2005. "Resampling student'st-type statistics," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 57(3), pages 507-529, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. MacKinnon, James G. & Webb, Matthew D., 2020. "Randomization inference for difference-in-differences with few treated clusters," Journal of Econometrics, Elsevier, vol. 218(2), pages 435-450.
    2. Yuehao Bai & Joseph P. Romano & Azeem M. Shaikh, 2022. "Inference in Experiments With Matched Pairs," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 117(540), pages 1726-1737, October.
    3. Damian Clarke & Joseph P. Romano & Michael Wolf, 2020. "The Romano–Wolf multiple-hypothesis correction in Stata," Stata Journal, StataCorp LP, vol. 20(4), pages 812-843, December.
    4. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    5. Federico A. Bugni & Jia Li & Qiyuan Li, 2023. "Permutation‐based tests for discontinuities in event studies," Quantitative Economics, Econometric Society, vol. 14(1), pages 37-70, January.
    6. Joseph P. Romano & Marius A. Tirlea, 2022. "Permutation testing for dependence in time series," Journal of Time Series Analysis, Wiley Blackwell, vol. 43(5), pages 781-807, September.
    7. Nathaniel E. Helwig, 2022. "Robust Permutation Tests for Penalized Splines," Stats, MDPI, vol. 5(3), pages 1-18, September.
    8. Purevdorj Tuvaandorj, 2021. "Robust Permutation Tests in Linear Instrumental Variables Regression," Papers 2111.13774, arXiv.org, revised Jun 2023.
    9. Federico A. Bugni & Jia Li & Qiyuan Li, 2020. "Permutation-based tests for discontinuities in event studies," Papers 2007.09837, arXiv.org, revised Jul 2022.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    2. Hagemann, Andreas, 2019. "Placebo inference on treatment effects when the number of clusters is small," Journal of Econometrics, Elsevier, vol. 213(1), pages 190-209.
    3. Purevdorj Tuvaandorj, 2021. "Robust Permutation Tests in Linear Instrumental Variables Regression," Papers 2111.13774, arXiv.org, revised Jun 2023.
    4. Friedrich, Sarah & Brunner, Edgar & Pauly, Markus, 2017. "Permuting longitudinal data in spite of the dependencies," Journal of Multivariate Analysis, Elsevier, vol. 153(C), pages 255-265.
    5. Chung, EunYi & Romano, Joseph P., 2016. "Multivariate and multiple permutation tests," Journal of Econometrics, Elsevier, vol. 193(1), pages 76-91.
    6. Juwon Seo, 2018. "Randomization Tests for Equality in Dependence Structure," Papers 1811.02105, arXiv.org.
    7. Mittelhammer, Ron C. & Judge, George G., 2005. "Combining estimators to improve structural model estimation and inference under quadratic loss," Journal of Econometrics, Elsevier, vol. 128(1), pages 1-29, September.
    8. Nathaniel E. Helwig, 2022. "Robust Permutation Tests for Penalized Splines," Stats, MDPI, vol. 5(3), pages 1-18, September.
    9. Young, Alwyn, 2019. "Channeling Fisher: randomization tests and the statistical insignificance of seemingly significant experimental results," LSE Research Online Documents on Economics 101401, London School of Economics and Political Science, LSE Library.
    10. repec:sip:wpaper:12-026 is not listed on IDEAS
    11. Dominik Schreyer, 2019. "Football spectator no-show behaviour in the German Bundesliga," Applied Economics, Taylor & Francis Journals, vol. 51(45), pages 4882-4901, September.
    12. S. Arunachalam & Sridhar N. Ramaswami & Pol Herrmann & Doug Walker, 2018. "Innovation pathway to profitability: the role of entrepreneurial orientation and marketing capabilities," Journal of the Academy of Marketing Science, Springer, vol. 46(4), pages 744-766, July.
    13. Timothy Erickson & Toni M. Whited, 2000. "Measurement Error and the Relationship between Investment and q," Journal of Political Economy, University of Chicago Press, vol. 108(5), pages 1027-1057, October.
    14. Paul W. Miller & Barry R. Chiswick, 2002. "Immigrant earnings: Language skills, linguistic concentrations and the business cycle," Journal of Population Economics, Springer;European Society for Population Economics, vol. 15(1), pages 31-57.
    15. Fors, Gunnar & Zejan, Mario, 1996. "Overseas R&D by Multinationals in foreign Centers of Excellence," SSE/EFI Working Paper Series in Economics and Finance 111, Stockholm School of Economics.
    16. Rodrigo M. S. Moita & Claudio Paiva, 2013. "Political Price Cycles in Regulated Industries: Theory and Evidence," American Economic Journal: Economic Policy, American Economic Association, vol. 5(1), pages 94-121, February.
    17. Evens Salies & Catherine Waddams, 2003. "Pricing structure in the deregulated UK electricity market," SciencePo Working papers Main hal-03592457, HAL.
    18. Butler, Marty & Leone, Andrew J. & Willenborg, Michael, 2004. "An empirical analysis of auditor reporting and its association with abnormal accruals," Journal of Accounting and Economics, Elsevier, vol. 37(2), pages 139-165, June.
    19. Baiyegunhi, L.J.S. & Oppong, B.B., 2016. "Commercialisation of mopane worm (Imbrasia belina) in rural households in Limpopo Province, South Africa," Forest Policy and Economics, Elsevier, vol. 62(C), pages 141-148.
    20. MacKinnon, J G, 1989. "Heteroskedasticity-Robust Tests for Structural Change," Empirical Economics, Springer, vol. 14(2), pages 77-92.
    21. Fenech, Jean-Pierre & Skully, Michael & Xuguang, Han, 2014. "Franking credits and market reactions: Evidence from the Australian convertible security market," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 32(C), pages 1-19.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:jnlasa:v:112:y:2017:i:519:p:1211-1220. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/UASA20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.