IDEAS home Printed from https://ideas.repec.org/a/spr/stpapr/v65y2024i3d10.1007_s00362-023-01455-8.html
   My bibliography  Save this article

Inferring the finest pattern of mutual independence from data

Author

Listed:
  • Guillaume Marrelec

    (Sorbonne Université, CNRS, Inserm, Laboratoire d’imagerie biomédicale, LIB)

  • Alain Giron

    (Sorbonne Université, CNRS, Inserm, Laboratoire d’imagerie biomédicale, LIB)

Abstract

For a random variable X, we are interested in the blind extraction of its finest mutual independence pattern $$\mu (X)$$ μ ( X ) . We introduce a specific kind of independence that we call dichotomic. If $$\Delta (X)$$ Δ ( X ) stands for the set of all patterns of dichotomic independence that hold for X, we show that $$\mu (X)$$ μ ( X ) can be obtained as the intersection of all elements of $$\Delta (X)$$ Δ ( X ) . We then propose a method to estimate $$\Delta (X)$$ Δ ( X ) when the data are independent and identically (i.i.d.) realizations of a multivariate normal distribution. If $$\hat{\Delta } (X)$$ Δ ^ ( X ) is the estimated set of valid patterns of dichotomic independence, we estimate $$\mu (X)$$ μ ( X ) as the intersection of all patterns of $$\hat{\Delta } (X)$$ Δ ^ ( X ) . The method is tested on simulated data, showing its advantages and limits. We also consider an application to a toy example as well as to experimental data.

Suggested Citation

  • Guillaume Marrelec & Alain Giron, 2024. "Inferring the finest pattern of mutual independence from data," Statistical Papers, Springer, vol. 65(3), pages 1677-1702, May.
    • Handle: RePEc:spr:stpapr:v:65:y:2024:i:3:d:10.1007_s00362-023-01455-8
    DOI: 10.1007/s00362-023-01455-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00362-023-01455-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00362-023-01455-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Marrelec, Guillaume & Benali, Habib, 2006. "Asymptotic Bayesian structure learning using graph supports for Gaussian graphical models," Journal of Multivariate Analysis, Elsevier, vol. 97(6), pages 1451-1466, July.
    2. Guillaume Marrelec & Arnaud Messé & Pierre Bellec, 2015. "A Bayesian Alternative to Mutual Information for the Hierarchical Clustering of Dependent Random Variables," PLOS ONE, Public Library of Science, vol. 10(9), pages 1-26, September.
    3. James R. Schott, 2005. "Testing for complete independence in high dimensions," Biometrika, Biometrika Trust, vol. 92(4), pages 951-956, December.
    4. Schott, James R., 2008. "A test for independence of two sets of variables when the number of variables is large relative to the sample size," Statistics & Probability Letters, Elsevier, vol. 78(17), pages 3096-3102, December.
    5. Bakirov, Nail K. & Rizzo, Maria L. & Szekely, Gábor J., 2006. "A multivariate nonparametric test of independence," Journal of Multivariate Analysis, Elsevier, vol. 97(8), pages 1742-1756, September.
    6. Niklas Pfister & Peter Bühlmann & Bernhard Schölkopf & Jonas Peters, 2018. "Kernel‐based tests for joint independence," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 80(1), pages 5-31, January.
    7. Székely, Gábor J. & Rizzo, Maria L., 2013. "The distance correlation t-test of independence in high dimension," Journal of Multivariate Analysis, Elsevier, vol. 117(C), pages 193-213.
    8. John D. Storey & Jonathan E. Taylor & David Siegmund, 2004. "Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(1), pages 187-205, February.
    9. Guangyu Mao, 2017. "Robust test for independence in high dimensions," Communications in Statistics - Theory and Methods, Taylor & Francis Journals, vol. 46(20), pages 10036-10050, October.
    10. Mao, Guangyu, 2018. "Testing independence in high dimensions using Kendall’s tau," Computational Statistics & Data Analysis, Elsevier, vol. 117(C), pages 128-137.
    11. Yoav Benjamini, 2010. "Discovering the false discovery rate," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(4), pages 405-416, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. He, Daojiang & Liu, Huanyu & Xu, Kai & Cao, Mingxiang, 2021. "Generalized Schott type tests for complete independence in high dimensions," Journal of Multivariate Analysis, Elsevier, vol. 183(C).
    2. Liqi Xia & Ruiyuan Cao & Jiang Du & Jun Dai, 2025. "Consistent complete independence test in high dimensions based on Chatterjee correlation coefficient," Statistical Papers, Springer, vol. 66(1), pages 1-32, January.
    3. Zhang, Qingyang, 2019. "Independence test for large sparse contingency tables based on distance correlation," Statistics & Probability Letters, Elsevier, vol. 148(C), pages 17-22.
    4. Mao, Guangyu, 2018. "Testing independence in high dimensions using Kendall’s tau," Computational Statistics & Data Analysis, Elsevier, vol. 117(C), pages 128-137.
    5. Ivair R. Silva & Yan Zhuang & Julio C. A. da Silva Junior, 2022. "Kronecker delta method for testing independence between two vectors in high-dimension," Statistical Papers, Springer, vol. 63(2), pages 343-365, April.
    6. Andrew Y. Chen, 2021. "The Limits of p‐Hacking: Some Thought Experiments," Journal of Finance, American Finance Association, vol. 76(5), pages 2447-2480, October.
    7. Marta Cousido‐Rocha & Jacobo de Uña‐Álvarez & Sebastian Döhler, 2022. "Multiple comparison procedures for discrete uniform and homogeneous tests," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(1), pages 219-243, January.
    8. Georgios Sermpinis & Arman Hassanniakalager & Charalampos Stasinakis & Ioannis Psaradellis, 2018. "Technical Analysis and Discrete False Discovery Rate: Evidence from MSCI Indices," Papers 1811.06766, arXiv.org, revised Jun 2019.
    9. Chen, Song Xi & Qin, Yingli, 2010. "A Two Sample Test for High Dimensional Data with Applications to Gene-set Testing," MPRA Paper 59642, University Library of Munich, Germany.
    10. Andrew Y. Chen, 2022. "Do t-Statistic Hurdles Need to be Raised?," Papers 2204.10275, arXiv.org, revised Apr 2024.
    11. Székely, Gábor J. & Rizzo, Maria L., 2013. "The distance correlation t-test of independence in high dimension," Journal of Multivariate Analysis, Elsevier, vol. 117(C), pages 193-213.
    12. S Gorsky & L Ma, 2022. "Multi-scale Fisher’s independence test for multivariate dependence [A simple measure of conditional dependence]," Biometrika, Biometrika Trust, vol. 109(3), pages 569-587.
    13. Jianqing Fan & Xu Han, 2017. "Estimation of the false discovery proportion with unknown dependence," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(4), pages 1143-1164, September.
    14. Shigeyuki Matsui & Hisashi Noma, 2011. "Estimating Effect Sizes of Differentially Expressed Genes for Power and Sample-Size Assessments in Microarray Experiments," Biometrics, The International Biometric Society, vol. 67(4), pages 1225-1235, December.
    15. Lianming Wang & David B. Dunson, 2010. "Semiparametric Bayes Multiple Testing: Applications to Tumor Data," Biometrics, The International Biometric Society, vol. 66(2), pages 493-501, June.
    16. Hanck, Christoph, 2011. "Now, whose schools are really better (or weaker) than Germany's? A multiple testing approach," Economic Modelling, Elsevier, vol. 28(4), pages 1739-1746, July.
    17. Ghosh Debashis, 2012. "Incorporating the Empirical Null Hypothesis into the Benjamini-Hochberg Procedure," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 11(4), pages 1-21, July.
    18. Dean Palejev & Mladen Savov, 2021. "On the Convergence of the Benjamini–Hochberg Procedure," Mathematics, MDPI, vol. 9(17), pages 1-19, September.
    19. Moon, H.R. & Perron, B., 2012. "Beyond panel unit root tests: Using multiple testing to determine the nonstationarity properties of individual series in a panel," Journal of Econometrics, Elsevier, vol. 169(1), pages 29-33.
    20. Rauf Ahmad, M., 2019. "A significance test of the RV coefficient in high dimensions," Computational Statistics & Data Analysis, Elsevier, vol. 131(C), pages 116-130.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stpapr:v:65:y:2024:i:3:d:10.1007_s00362-023-01455-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.