IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v240y2024i2s0304407622001609.html
   My bibliography  Save this article

Testing unconditional and conditional independence via mutual information

Author

Listed:
  • Ai, Chunrong
  • Sun, Li-Hsien
  • Zhang, Zheng
  • Zhu, Liping

Abstract

Testing independence has garnered increasing attention in the econometric and statistical literature. Many tests have been proposed, most of which are inconsistent against all departures from independence. Few of those tests, though consistent, suffer a significant loss of local power. This study proposes a mutual information test for testing independence. The proposed test is simple to implement and, with a slight loss of local power, is consistent against all departures from independence. The key driving factor is that we estimate the density ratio directly. This value is constant in a state of independence. This is in contrast with related studies that estimate the joint and marginal density functions to form the density ratio. A small-scale simulation study indicates that the proposed test outperforms the existing alternatives in various dependence structures.

Suggested Citation

  • Ai, Chunrong & Sun, Li-Hsien & Zhang, Zheng & Zhu, Liping, 2024. "Testing unconditional and conditional independence via mutual information," Journal of Econometrics, Elsevier, vol. 240(2).
    • Handle: RePEc:eee:econom:v:240:y:2024:i:2:s0304407622001609
    DOI: 10.1016/j.jeconom.2022.07.011
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304407622001609
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jeconom.2022.07.011?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Angrist, Joshua D., 1997. "Conditional independence in sample selection models," Economics Letters, Elsevier, vol. 54(2), pages 103-112, February.
    2. Whitney K. Newey & Richard J. Smith, 2004. "Higher Order Properties of Gmm and Generalized Empirical Likelihood Estimators," Econometrica, Econometric Society, vol. 72(1), pages 219-255, January.
    3. Su, Liangjun & White, Halbert, 2007. "A consistent characteristic function-based test for conditional independence," Journal of Econometrics, Elsevier, vol. 141(2), pages 807-834, December.
    4. Huang, Meng & Sun, Yixiao & White, Halbert, 2016. "A Flexible Nonparametric Test For Conditional Independence," Econometric Theory, Cambridge University Press, vol. 32(6), pages 1434-1482, December.
    5. Yongmiao Hong, 1998. "Testing for pairwise serial independence via the empirical distribution function," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 60(2), pages 429-453.
    6. Jacob A. Mincer, 1974. "Introduction to "Schooling, Experience, and Earnings"," NBER Chapters, in: Schooling, Experience, and Earnings, pages 1-4, National Bureau of Economic Research, Inc.
    7. Su, Liangjun & White, Halbert, 2008. "A Nonparametric Hellinger Metric Test For Conditional Independence," Econometric Theory, Cambridge University Press, vol. 24(4), pages 829-864, August.
    8. Congjia Huo & Lingming Chen, 2022. "Does individual educational attainment affect educational homogamy?—Evidence from CGSS," International Studies of Economics, John Wiley & Sons, vol. 17(1), pages 82-107, June.
    9. Mohammad Altaf‐Ul Alam & Xiaobo He, 2022. "Is the discrimination against migrant workers tending toward zero in urban China?," International Studies of Economics, John Wiley & Sons, vol. 17(1), pages 65-81, June.
    10. Yuichi Kitamura & Michael Stutzer, 1997. "An Information-Theoretic Alternative to Generalized Method of Moments Estimation," Econometrica, Econometric Society, vol. 65(4), pages 861-874, July.
    11. Su, Liangjun & White, Halbert, 2014. "Testing conditional independence via empirical likelihood," Journal of Econometrics, Elsevier, vol. 182(1), pages 27-44.
    12. Oliver Linton & Pedro Gozalo, 2014. "Testing Conditional Independence Restrictions," Econometric Reviews, Taylor & Francis Journals, vol. 33(5-6), pages 523-552, August.
    13. Jacob A. Mincer, 1974. "Schooling, Experience, and Earnings," NBER Books, National Bureau of Economic Research, Inc, number minc74-1.
    14. Horowitz, Joel L & Spokoiny, Vladimir G, 2001. "An Adaptive, Rate-Optimal Test of a Parametric Mean-Regression Model against a Nonparametric Alternative," Econometrica, Econometric Society, vol. 69(3), pages 599-631, May.
    15. Chunrong Ai & Xiaohong Chen, 2003. "Efficient Estimation of Models with Conditional Moment Restrictions Containing Unknown Functions," Econometrica, Econometric Society, vol. 71(6), pages 1795-1843, November.
    16. Guido W. Imbens & Richard H. Spady & Phillip Johnson, 1998. "Information Theoretic Approaches to Inference in Moment Condition Models," Econometrica, Econometric Society, vol. 66(2), pages 333-358, March.
    17. Blackburn, McKinley L & Neumark, David, 1993. "Omitted-Ability Bias and the Increase in the Return to Schooling," Journal of Labor Economics, University of Chicago Press, vol. 11(3), pages 521-544, July.
    18. Pinkse, Joris, 1998. "A consistent nonparametric test for serial independence," Journal of Econometrics, Elsevier, vol. 84(2), pages 205-231, June.
    19. Newey, Whitney K, 1994. "The Asymptotic Variance of Semiparametric Estimators," Econometrica, Econometric Society, vol. 62(6), pages 1349-1382, November.
    20. Newey, Whitney K., 1997. "Convergence rates and asymptotic normality for series estimators," Journal of Econometrics, Elsevier, vol. 79(1), pages 147-168, July.
    21. Efang Kong & Yingcun Xia & Wei Zhong, 2019. "Composite Coefficient of Determination and Its Application in Ultrahigh Dimensional Variable Screening," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(528), pages 1740-1751, October.
    22. Yuichi Kitamura & Taisuke Otsu & Kirill Evdokimov, 2013. "Robustness, Infinitesimal Neighborhoods, and Moment Restrictions," Econometrica, Econometric Society, vol. 81(3), pages 1185-1201, May.
    23. P. M. Robinson, 1991. "Consistent Nonparametric Entropy-Based Testing," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 58(3), pages 437-453.
    24. Marc Hallin & Madan Lal Puri, 1992. "Rank tests for time-series analysis: a survey," ULB Institutional Repository 2013/2229, ULB -- Universite Libre de Bruxelles.
    25. Chen, Xiaohong, 2007. "Large Sample Sieve Estimation of Semi-Nonparametric Models," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 76, Elsevier.
    26. Martin Huber & Blaise Melly, 2015. "A Test of the Conditional Independence Assumption in Sample Selection Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 30(7), pages 1144-1168, November.
    27. Paul Tseng & Dimitri P. Bertsekas, 1991. "Relaxation Methods for Problems with Strictly Convex Costs and Linear Constraints," Mathematics of Operations Research, INFORMS, vol. 16(3), pages 462-481, August.
    28. Yongmiao Hong & Halbert White, 2005. "Asymptotic Distribution Theory for Nonparametric Entropy Measures of Serial Dependence," Econometrica, Econometric Society, vol. 73(3), pages 837-901, May.
    29. John Xu Zheng, 1996. "A consistent test of functional form via nonparametric estimation techniques," Journal of Econometrics, Elsevier, vol. 75(2), pages 263-289, December.
    30. Yongmiao Hong, 2000. "Generalized spectral tests for serial dependence," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 62(3), pages 557-574.
    31. Matilla-Garci­a, Mariano & Ruiz Mari­n, Manuel, 2008. "A non-parametric independence test using permutation entropy," Journal of Econometrics, Elsevier, vol. 144(1), pages 139-155, May.
    32. Wang, Xia & Hong, Yongmiao, 2018. "Characteristic Function Based Testing For Conditional Independence: A Nonparametric Regression Approach," Econometric Theory, Cambridge University Press, vol. 34(4), pages 815-849, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wang, Hongfei & Liu, Binghui & Feng, Long & Ma, Yanyuan, 2024. "Rank-based max-sum tests for mutual independence of high-dimensional random vectors," Journal of Econometrics, Elsevier, vol. 238(1).
    2. Ai, Chunrong & Linton, Oliver & Zhang, Zheng, 2022. "Estimation and inference for the counterfactual distribution and quantile functions in continuous treatment models," Journal of Econometrics, Elsevier, vol. 228(1), pages 39-61.
    3. Xiaojun Song & Haoyu Wei, 2021. "Nonparametric Tests of Conditional Independence for Time Series," Papers 2110.04847, arXiv.org.
    4. Xuehu Zhu & Jun Lu & Jun Zhang & Lixing Zhu, 2021. "Testing for conditional independence: A groupwise dimension reduction‐based adaptive‐to‐model approach," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 48(2), pages 549-576, June.
    5. Qi Li & Jeffrey Scott Racine, 2006. "Nonparametric Econometrics: Theory and Practice," Economics Books, Princeton University Press, edition 1, volume 1, number 8355.
    6. Chunrong Ai & Oliver Linton & Kaiji Motegi & Zheng Zhang, 2021. "A unified framework for efficient estimation of general treatment models," Quantitative Economics, Econometric Society, vol. 12(3), pages 779-816, July.
    7. Taoufik Bouezmarni & Jeroen V.K. Rombouts & Abderrahim Taamouti, 2011. "Nonparametric Copula-Based Test for Conditional Independence with Applications to Granger Causality," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(2), pages 275-287, October.
    8. Cui, Li-E & Zhao, Puying & Tang, Niansheng, 2022. "Generalized empirical likelihood for nonsmooth estimating equations with missing data," Journal of Multivariate Analysis, Elsevier, vol. 190(C).
    9. Su, Liangjun & Zheng, Xin, 2017. "A martingale-difference-divergence-based test for specification," Economics Letters, Elsevier, vol. 156(C), pages 162-167.
    10. Taisuke Otsu & Chen Qiu, 2018. "Information theoretic approach to high dimensional multiplicative models: Stochastic discount factor and treatment effect," STICERD - Econometrics Paper Series 595, Suntory and Toyota International Centres for Economics and Related Disciplines, LSE.
    11. Joachim Inkmann, 2010. "Estimating Firm Size Elasticities of Product and Process R&D," Economica, London School of Economics and Political Science, vol. 77(306), pages 384-402, April.
    12. Dong, Chaohua & Gao, Jiti & Linton, Oliver, 2023. "High dimensional semiparametric moment restriction models," Journal of Econometrics, Elsevier, vol. 232(2), pages 320-345.
    13. Inkmann, J., 2005. "Inverse Probability Weighted Generalised Empirical Likelihood Estimators : Firm Size and R&D Revisited," Other publications TiSEM c39cff1f-16c1-4446-a83f-c, Tilburg University, School of Economics and Management.
    14. Heckman, James J. & Schmierer, Daniel, 2010. "Tests of hypotheses arising in the correlated random coefficient model," Economic Modelling, Elsevier, vol. 27(6), pages 1355-1367, November.
    15. Parente, Paulo M.D.C. & Smith, Richard J., 2017. "Tests of additional conditional moment restrictions," Journal of Econometrics, Elsevier, vol. 200(1), pages 1-16.
    16. Huang, Meng & Sun, Yixiao & White, Halbert, 2016. "A Flexible Nonparametric Test For Conditional Independence," Econometric Theory, Cambridge University Press, vol. 32(6), pages 1434-1482, December.
    17. Chang, Jinyuan & Chen, Song Xi & Chen, Xiaohong, 2015. "High dimensional generalized empirical likelihood for moment restrictions with dependent data," Journal of Econometrics, Elsevier, vol. 185(1), pages 283-304.
    18. Fan, Yanqin & Gentry, Matthew & Li, Tong, 2011. "A new class of asymptotically efficient estimators for moment condition models," Journal of Econometrics, Elsevier, vol. 162(2), pages 268-277, June.
    19. Chen, Xiaohong, 2007. "Large Sample Sieve Estimation of Semi-Nonparametric Models," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 76, Elsevier.
    20. repec:wyi:journl:002087 is not listed on IDEAS
    21. repec:cep:stiecm:/2014/572 is not listed on IDEAS
    22. Hoderlein, Stefan & Su, Liangjun & White, Halbert & Yang, Thomas Tao, 2016. "Testing for monotonicity in unobservables under unconfoundedness," Journal of Econometrics, Elsevier, vol. 193(1), pages 183-202.

    More about this item

    Keywords

    Convex optimization; Density ratio; Independence test; Mutual information;
    All these keywords.

    JEL classification:

    • C01 - Mathematical and Quantitative Methods - - General - - - Econometrics
    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:240:y:2024:i:2:s0304407622001609. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.