IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v12y2024i17p2617-d1462910.html
   My bibliography  Save this article

Review about the Permutation Approach in Hypothesis Testing

Author

Listed:
  • Stefano Bonnini

    (Department of Economics and Management, University of Ferrara, Via Voltapaletto 11, 44121 Ferrara, Italy)

  • Getnet Melak Assegie

    (Department of Economics and Management, University of Parma, 43125 Parma, Italy)

  • Kamila Trzcinska

    (Department of Statistical Methods, University of Lodz, 41 Rewolucji 1905 r. St., 92-2014 Lodz, Poland)

Abstract

Today, permutation tests represent a powerful and increasingly widespread tool of statistical inference for hypothesis-testing problems. To the best of our knowledge, a review of the application of permutation tests for complex data in practical data analysis for hypothesis testing is missing. In particular, it is essential to review the application of permutation tests in two-sample or multi-sample problems and in regression analysis. The aim of this paper is to consider the main scientific contributions on the subject of permutation methods for hypothesis testing in the mentioned fields. Notes on their use to address the problem of missing data and, in particular, right-censored data, will also be included. This review also tries to highlight the limits and advantages of the works cited with a critical eye and also to provide practical indications to researchers and practitioners who need to identify flexible and distribution-free solutions for the most disparate hypothesis-testing problems.

Suggested Citation

  • Stefano Bonnini & Getnet Melak Assegie & Kamila Trzcinska, 2024. "Review about the Permutation Approach in Hypothesis Testing," Mathematics, MDPI, vol. 12(17), pages 1-29, August.
    • Handle: RePEc:gam:jmathe:v:12:y:2024:i:17:p:2617-:d:1462910
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/12/17/2617/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/12/17/2617/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Arboretti, Rosa & Bonnini, Stefano & Corain, Livio & Salmaso, Luigi, 2014. "A permutation approach for ranking of multivariate populations," Journal of Multivariate Analysis, Elsevier, vol. 132(C), pages 39-57.
    2. Biswas, Munmun & Ghosh, Anil K., 2014. "A nonparametric two-sample test applicable to high dimensional data," Journal of Multivariate Analysis, Elsevier, vol. 123(C), pages 160-171.
    3. Paul R. Rosenbaum, 2005. "An exact distribution‐free test comparing two multivariate distributions based on adjacency," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(4), pages 515-530, September.
    4. Marinho Bertanha & Eunyi Chung, 2023. "Permutation Tests at Nonparametric Rates," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 118(544), pages 2833-2846, October.
    5. Markus Pauly & Edgar Brunner & Frank Konietschke, 2015. "Asymptotic permutation tests in general factorial designs," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 77(2), pages 461-473, March.
    6. Neuhaus, Georg & Zhu, Li-Xing, 1999. "Permutation Tests for Multivariate Location Problems," Journal of Multivariate Analysis, Elsevier, vol. 69(2), pages 167-192, May.
    7. Cyrus J. DiCiccio & Joseph P. Romano, 2017. "Robust Permutation Tests For Correlation And Regression Coefficients," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 1211-1220, July.
    8. Marco Marozzi, 2002. "Some notes on nonparametric inferences and permutation tests," Metron - International Journal of Statistics, Dipartimento di Statistica, Probabilità e Statistiche Applicate - University of Rome, vol. 0(3-4), pages 139-151.
    9. Kennedy, Peter E, 1995. "Randomization Tests in Econometrics," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(1), pages 85-94, January.
    10. Rosa Giancristofaro & Stefano Bonnini, 2008. "Moment-based multivariate permutation tests for ordinal categorical data," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 20(5), pages 383-393.
    11. Sonja Hahn & Luigi Salmaso, 2017. "A comparison of different synchronized permutation approaches to testing effects in two-level two-factor unbalanced ANOVA designs," Statistical Papers, Springer, vol. 58(1), pages 123-146, March.
    12. Jesse Hemerik & Jelle Goeman, 2018. "Exact testing with random permutations," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 27(4), pages 811-825, December.
    13. Hayfield, Tristen & Racine, Jeffrey S., 2008. "Nonparametric Econometrics: The np Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 27(i05).
    14. Roderick J. Little & Donald B. Rubin & Sahar Z. Zangeneh, 2017. "Conditions for Ignoring the Missing-Data Mechanism in Likelihood Inferences for Parameter Subsets," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(517), pages 314-320, January.
    15. Monjed H. Samuh & Fortunato Pesarin, 2018. "Applications of conditional power function of two-sample permutation test," Computational Statistics, Springer, vol. 33(4), pages 1847-1862, December.
    16. Edgar Brunner & Madan Puri, 2001. "Nonparametric methods in factorial designs," Statistical Papers, Springer, vol. 42(1), pages 1-52, January.
    17. Kherad-Pajouh, Sara & Renaud, Olivier, 2010. "An exact permutation method for testing any effect in balanced and unbalanced fixed effect ANOVA," Computational Statistics & Data Analysis, Elsevier, vol. 54(7), pages 1881-1893, July.
    18. Ute Hahn, 2012. "A Studentized Permutation Test for the Comparison of Spatial Point Patterns," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(498), pages 754-764, June.
    19. Peng Ding & Avi Feller & Luke Miratrix, 2016. "Randomization inference for treatment effect variation," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(3), pages 655-671, June.
    20. Sara Kherad-Pajouh & Olivier Renaud, 2015. "A general permutation approach for analyzing repeated measures ANOVA and mixed-model designs," Statistical Papers, Springer, vol. 56(4), pages 947-967, November.
    21. Rosa Arboretti Giancristofaro & Stefano Bonnini, 2009. "Some new results on univariate and multivariate permutation tests for ordinal categorical variables under restricted alternatives," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 18(2), pages 221-236, July.
    22. Rosa Arboretti Giancristofaro & Chiara Brombin, 2014. "Overview of NonParametric Combination-based permutation tests for Multivariate multi-sample problems," Statistica, Department of Statistics, University of Bologna, vol. 74(3), pages 233-246.
    23. Neubert, Karin & Brunner, Edgar, 2007. "A studentized permutation test for the non-parametric Behrens-Fisher problem," Computational Statistics & Data Analysis, Elsevier, vol. 51(10), pages 5192-5204, June.
    24. Han Yu & Alan D. Hutson, 2024. "A robust Spearman correlation coefficient permutation test," Communications in Statistics - Theory and Methods, Taylor & Francis Journals, vol. 53(6), pages 2141-2153, March.
    25. Stefano Bonnini & Livio Corain & Fortunato Munaò & Luigi Salmaso, 2006. "Neurocognitive Effects in Welders Exposed to Aluminium: An Application of the NPC Test and NPC Ranking Methods," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 15(2), pages 191-208, August.
    26. Michael Brendel & Arnold Janssen & Claus-Dieter Mayer & Markus Pauly, 2014. "Weighted Logrank Permutation Tests for Randomly Right Censored Life Science Data," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 41(3), pages 742-761, September.
    27. Zhang, Long-Wen & Dang, Chao & Zhao, Yan-Gang, 2023. "An efficient method for accessing structural reliability indexes via power transformation family," Reliability Engineering and System Safety, Elsevier, vol. 233(C).
    28. Stefano Bonnini & Livio Corain & Fortunato Munaò & Luigi Salmaso, 2006. "Neurocognitive Effects in Welders Exposed to Aluminium: An Application of the NPC Test and NPC Ranking Methods," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 15(2), pages 191-208, August.
    29. Konietschke, Frank & Bathke, Arne C. & Harrar, Solomon W. & Pauly, Markus, 2015. "Parametric and nonparametric bootstrap methods for general MANOVA," Journal of Multivariate Analysis, Elsevier, vol. 140(C), pages 291-301.
    30. Brombin, Chiara & Salmaso, Luigi, 2009. "Multi-aspect permutation tests in shape analysis with small sample size," Computational Statistics & Data Analysis, Elsevier, vol. 53(12), pages 3921-3931, October.
    31. Rosa Arboretti & Riccardo Ceccato & Livio Corain & Fabrizio Ronchi & Luigi Salmaso, 2018. "Multivariate small sample tests for two-way designs with applications to industrial statistics," Statistical Papers, Springer, vol. 59(4), pages 1483-1503, December.
    32. J. Gower & P. Legendre, 1986. "Metric and Euclidean properties of dissimilarity coefficients," Journal of Classification, Springer;The Classification Society, vol. 3(1), pages 5-48, March.
    33. Janssen, Arnold, 1997. "Studentized permutation tests for non-i.i.d. hypotheses and the generalized Behrens-Fisher problem," Statistics & Probability Letters, Elsevier, vol. 36(1), pages 9-21, November.
    34. Rodriguez-Poo, Juan M. & Soberón, Alexandra, 2015. "Nonparametric estimation of fixed effects panel data varying coefficient models," Journal of Multivariate Analysis, Elsevier, vol. 133(C), pages 95-122.
    35. Johannes Kaiser, 2007. "An exact and a Monte Carlo proposal to the Fisher–Pitman permutation tests for paired replicates and for independent samples," Stata Journal, StataCorp LP, vol. 7(3), pages 402-412, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David M. Ritzwoller & Joseph P. Romano & Azeem M. Shaikh, 2024. "Randomization Inference: Theory and Applications," Papers 2406.09521, arXiv.org.
    2. Dennis Dobler & Markus Pauly, 2018. "Bootstrap- and permutation-based inference for the Mann–Whitney effect for right-censored and tied data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 27(3), pages 639-658, September.
    3. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    4. Ditzhaus, Marc & Smaga, Łukasz, 2022. "Permutation test for the multivariate coefficient of variation in factorial designs," Journal of Multivariate Analysis, Elsevier, vol. 187(C).
    5. Smaga, Łukasz, 2015. "Wald-type statistics using {2}-inverses for hypothesis testing in general factorial designs," Statistics & Probability Letters, Elsevier, vol. 107(C), pages 215-220.
    6. Purevdorj Tuvaandorj, 2021. "Robust Permutation Tests in Linear Instrumental Variables Regression," Papers 2111.13774, arXiv.org, revised Jul 2024.
    7. Friedrich, Sarah & Brunner, Edgar & Pauly, Markus, 2017. "Permuting longitudinal data in spite of the dependencies," Journal of Multivariate Analysis, Elsevier, vol. 153(C), pages 255-265.
    8. Chung, EunYi & Romano, Joseph P., 2016. "Multivariate and multiple permutation tests," Journal of Econometrics, Elsevier, vol. 193(1), pages 76-91.
    9. Friedrich, Sarah & Konietschke, Frank & Pauly, Markus, 2017. "A wild bootstrap approach for nonparametric repeated measurements," Computational Statistics & Data Analysis, Elsevier, vol. 113(C), pages 38-52.
    10. Lovato, Ilenia & Pini, Alessia & Stamm, Aymeric & Vantini, Simone, 2020. "Model-free two-sample test for network-valued data," Computational Statistics & Data Analysis, Elsevier, vol. 144(C).
    11. Ditzhaus, Marc & Pauly, Markus, 2019. "Wild bootstrap logrank tests with broader power functions for testing superiority," Computational Statistics & Data Analysis, Elsevier, vol. 136(C), pages 1-11.
    12. Friedrich, Sarah & Pauly, Markus, 2018. "MATS: Inference for potentially singular and heteroscedastic MANOVA," Journal of Multivariate Analysis, Elsevier, vol. 165(C), pages 166-179.
    13. Dickhaus, Thorsten & Sirotko-Sibirskaya, Natalia, 2019. "Simultaneous statistical inference in dynamic factor models: Chi-square approximation and model-based bootstrap," Computational Statistics & Data Analysis, Elsevier, vol. 129(C), pages 30-46.
    14. Marinho Bertanha & Eunyi Chung, 2023. "Permutation Tests at Nonparametric Rates," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 118(544), pages 2833-2846, October.
    15. Nicolas Städler & Sach Mukherjee, 2017. "Two-sample testing in high dimensions," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(1), pages 225-246, January.
    16. H. V. Kulkarni & S. M. Patil, 2021. "Uniformly implementable small sample integrated likelihood ratio test for one-way and two-way ANOVA under heteroscedasticity and normality," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 105(2), pages 273-305, June.
    17. Shin-ichi Tsukada, 2019. "High dimensional two-sample test based on the inter-point distance," Computational Statistics, Springer, vol. 34(2), pages 599-615, June.
    18. Chung, EunYi & Olivares, Mauricio, 2021. "Permutation test for heterogeneous treatment effects with a nuisance parameter," Journal of Econometrics, Elsevier, vol. 225(2), pages 148-174.
    19. Yuehao Bai & Joseph P. Romano & Azeem M. Shaikh, 2022. "Inference in Experiments With Matched Pairs," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 117(540), pages 1726-1737, October.
    20. Juwon Seo, 2018. "Randomization Tests for Equality in Dependence Structure," Papers 1811.02105, arXiv.org.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:12:y:2024:i:17:p:2617-:d:1462910. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.