IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v54y2010i7p1881-1893.html
   My bibliography  Save this article

An exact permutation method for testing any effect in balanced and unbalanced fixed effect ANOVA

Author

Listed:
  • Kherad-Pajouh, Sara
  • Renaud, Olivier

Abstract

The ANOVA method and permutation tests, two heritages of Fisher, have been extensively studied. Several permutation strategies have been proposed by others to obtain a distribution-free test for factors in a fixed effect ANOVA (i.e., single error term ANOVA). The resulting tests are either approximate or exact. However, there exists no universal exact permutation test which can be applied to an arbitrary design to test a desired factor. An exact permutation strategy applicable to fixed effect analysis of variance is presented. The proposed method can be used to test any factor, even in the presence of higher-order interactions. In addition, the method has the advantage of being applicable in unbalanced designs (all-cell-filled), which is a very common situation in practice, and it is the first method with this capability. Simulation studies show that the proposed method has an actual level which stays remarkably close to the nominal level, and its power is always competitive. This is the case even with very small datasets, strongly unbalanced designs and non-Gaussian errors. No other competitor show such an enviable behavior.

Suggested Citation

  • Kherad-Pajouh, Sara & Renaud, Olivier, 2010. "An exact permutation method for testing any effect in balanced and unbalanced fixed effect ANOVA," Computational Statistics & Data Analysis, Elsevier, vol. 54(7), pages 1881-1893, July.
    • Handle: RePEc:eee:csdana:v:54:y:2010:i:7:p:1881-1893
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(10)00080-0
    Download Restriction: Full text for ScienceDirect subscribers only.
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Freedman, David & Lane, David, 1983. "A Nonstochastic Interpretation of Reported Significance Levels," Journal of Business & Economic Statistics, American Statistical Association, vol. 1(4), pages 292-298, October.
    2. David, Herbert A., 2008. "The Beginnings of Randomization Tests," The American Statistician, American Statistical Association, vol. 62, pages 70-72, February.
    3. Brombin, Chiara & Salmaso, Luigi, 2009. "Multi-aspect permutation tests in shape analysis with small sample size," Computational Statistics & Data Analysis, Elsevier, vol. 53(12), pages 3921-3931, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sonja Hahn & Luigi Salmaso, 2017. "A comparison of different synchronized permutation approaches to testing effects in two-level two-factor unbalanced ANOVA designs," Statistical Papers, Springer, vol. 58(1), pages 123-146, March.
    2. Sara Kherad-Pajouh & Olivier Renaud, 2015. "A general permutation approach for analyzing repeated measures ANOVA and mixed-model designs," Statistical Papers, Springer, vol. 56(4), pages 947-967, November.
    3. Finos, Livio, 2011. "A note on Left-Spherically Distributed test with covariates," Statistics & Probability Letters, Elsevier, vol. 81(6), pages 639-641, June.
    4. Stefano Bonnini & Getnet Melak Assegie & Kamila Trzcinska, 2024. "Review about the Permutation Approach in Hypothesis Testing," Mathematics, MDPI, vol. 12(17), pages 1-29, August.
    5. Federico Ferraccioli & Laura M. Sangalli & Livio Finos, 2023. "Nonparametric tests for semiparametric regression models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 32(3), pages 1106-1130, September.
    6. Ferraccioli, Federico & Sangalli, Laura M. & Finos, Livio, 2022. "Some first inferential tools for spatial regression with differential regularization," Journal of Multivariate Analysis, Elsevier, vol. 189(C).
    7. Cristina Davino & Giuseppe Lamberti & Domenico Vistocco, 2024. "Testing heterogeneity in quantile regression: a multigroup approach," Computational Statistics, Springer, vol. 39(1), pages 117-140, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stefano Bonnini & Michela Borghesi, 2022. "Relationship between Mental Health and Socio-Economic, Demographic and Environmental Factors in the COVID-19 Lockdown Period—A Multivariate Regression Analysis," Mathematics, MDPI, vol. 10(18), pages 1-15, September.
    2. Purevdorj Tuvaandorj, 2021. "Robust Permutation Tests in Linear Instrumental Variables Regression," Papers 2111.13774, arXiv.org, revised Jul 2024.
    3. Chiara Brombin & Luigi Salmaso & Lara Fontanella & Luigi Ippoliti, 2015. "Nonparametric combination-based tests in dynamic shape analysis," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 27(4), pages 460-484, December.
    4. Stefano Bonnini & Getnet Melak Assegie & Kamila Trzcinska, 2024. "Review about the Permutation Approach in Hypothesis Testing," Mathematics, MDPI, vol. 12(17), pages 1-29, August.
    5. Hu, Yuan & Behrman, Jere R. & Zhang, Junsen, 2021. "The causal effects of parents’ schooling on children's schooling in urban China," Journal of Comparative Economics, Elsevier, vol. 49(1), pages 258-276.
    6. Daly, Michael & Delaney, Liam & Doyle, Orla & Fitzpatrick, Nick & O'Farrelly, Christine, 2014. "Can Early Intervention Policies Improve Well-being? Evidence from a randomized controlled trial," SIRE Discussion Papers 2015-03, Scottish Institute for Research in Economics (SIRE).
    7. Gill,Indermit S., 1990. "Does the structure of production affect demand for schooling in Peru?," Policy Research Working Paper Series 468, The World Bank.
    8. Pinto, Rodrigo, 2010. "Evaluation of Small-sample Compromised Randomization: Long-term Effects of Early Childhood Intervention on Health and Addictive Behavior," Brazilian Review of Econometrics, Sociedade Brasileira de Econometria - SBE, vol. 30(2), December.
    9. Sara Kherad-Pajouh & Olivier Renaud, 2015. "A general permutation approach for analyzing repeated measures ANOVA and mixed-model designs," Statistical Papers, Springer, vol. 56(4), pages 947-967, November.
    10. Pini, Alessia & Stamm, Aymeric & Vantini, Simone, 2018. "Hotelling’s T2 in separable Hilbert spaces," Journal of Multivariate Analysis, Elsevier, vol. 167(C), pages 284-305.
    11. Mittelhammer, Ron C. & Judge, George G., 2005. "Combining estimators to improve structural model estimation and inference under quadratic loss," Journal of Econometrics, Elsevier, vol. 128(1), pages 1-29, September.
    12. Orla Doyle, 2024. "Can Early Intervention Have a Sustained Effect on Human Capital?," Journal of Human Resources, University of Wisconsin Press, vol. 59(5), pages 1599-1636.
    13. Nathaniel E. Helwig, 2022. "Robust Permutation Tests for Penalized Splines," Stats, MDPI, vol. 5(3), pages 1-18, September.
    14. Karel Hron & Jitka Machalová & Alessandra Menafoglio, 2023. "Bivariate densities in Bayes spaces: orthogonal decomposition and spline representation," Statistical Papers, Springer, vol. 64(5), pages 1629-1667, October.
    15. Soo Hong Chew & Junjian Yi & Junsen Zhang & Songfa Zhong, 2016. "Education and anomalies in decision making: Experimental evidence from Chinese adult twins," Journal of Risk and Uncertainty, Springer, vol. 53(2), pages 163-200, December.
    16. Bryan Keller, 2012. "Detecting Treatment Effects with Small Samples: The Power of Some Tests Under the Randomization Model," Psychometrika, Springer;The Psychometric Society, vol. 77(2), pages 324-338, April.
    17. David Dekker & David Krackhardt & Tom Snijders, 2007. "Sensitivity of MRQAP Tests to Collinearity and Autocorrelation Conditions," Psychometrika, Springer;The Psychometric Society, vol. 72(4), pages 563-581, December.
    18. Veronika Římalová & Alessandra Menafoglio & Alessia Pini & Vilém Pechanec & Eva Fišerová, 2020. "A permutation approach to the analysis of spatiotemporal geochemical data in the presence of heteroscedasticity," Environmetrics, John Wiley & Sons, Ltd., vol. 31(4), June.
    19. Jiří Dvořák & Tomáš Mrkvička & Jorge Mateu & Jonatan A. González, 2022. "Nonparametric Testing of the Dependence Structure Among Points–Marks–Covariates in Spatial Point Patterns," International Statistical Review, International Statistical Institute, vol. 90(3), pages 592-621, December.
    20. Orla Doyle, 2017. "The First 2,000 Days and Child Skills: Evidence from a Randomized Experiment of Home Visiting," Working Papers 201715, School of Economics, University College Dublin.

    More about this item

    Keywords

    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:54:y:2010:i:7:p:1881-1893. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.