IDEAS home Printed from https://ideas.repec.org/a/spr/stpapr/v66y2025i3d10.1007_s00362-025-01676-z.html
   My bibliography  Save this article

Dimensionality reduction in multivariate nonparametric regression via nuclear norm penalization

Author

Listed:
  • Donghwi Nam

    (Korea University)

  • Ja-Yong Koo

    (Korea University)

  • Kwan-Young Bak

    (Sungshin Women’s University
    Sungshin Women’s University)

Abstract

This paper reports on our study of a nonparametric reduced-rank regression method within an additive model framework. The nuclear norm of component functions is penalized to incorporate inherent low-dimensional structure into the estimation process. The proposed penalization scheme introduces sparsity to the singular values of coefficient matrices of basis functions, thereby enabling the identification of low-rank structures in the function space. A non-asymptotic oracle inequality is established to investigate the theoretical properties of the proposed estimator. Minimax upper and lower bounds are obtained to measure the complexity of the nonparametric multivariate regression problem with a low-rank structure under various high-dimensional asymptotic scenarios. The minimax analysis demonstrates the dimensionality reduction effects of the reduced-rank and additive modeling framework. The results also show that the proposed estimator is rate optimal in the minimax sense. The proposed method is implemented with the backfitting and the alternating direction method of multipliers algorithm. Simulation studies are conducted to complement the theoretical findings. To demonstrate the practicality of the proposed method, we apply it to the biochemical data and Arabidopsis thaliana data.

Suggested Citation

  • Donghwi Nam & Ja-Yong Koo & Kwan-Young Bak, 2025. "Dimensionality reduction in multivariate nonparametric regression via nuclear norm penalization," Statistical Papers, Springer, vol. 66(3), pages 1-33, April.
    • Handle: RePEc:spr:stpapr:v:66:y:2025:i:3:d:10.1007_s00362-025-01676-z
    DOI: 10.1007/s00362-025-01676-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00362-025-01676-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00362-025-01676-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. A. Mukherjee & K. Chen & N. Wang & J. Zhu, 2015. "On the degrees of freedom of reduced-rank estimators in multivariate regression," Biometrika, Biometrika Trust, vol. 102(2), pages 457-477.
    2. Gen Li & Xiaokang Liu & Kun Chen, 2019. "Integrative multi‐view regression: Bridging group‐sparse and low‐rank models," Biometrics, The International Biometric Society, vol. 75(2), pages 593-602, June.
    3. Ming Yuan & Ali Ekici & Zhaosong Lu & Renato Monteiro, 2007. "Dimension reduction and coefficient estimation in multivariate linear regression," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(3), pages 329-346, June.
    4. Izenman, Alan Julian, 1975. "Reduced-rank regression for the multivariate linear model," Journal of Multivariate Analysis, Elsevier, vol. 5(2), pages 248-264, June.
    5. Jae-Hwan Jhong & Kwan-Young Bak & Jae-Kyung Shin & Ja-Yong Koo, 2021. "Additive regression splines with total variation and non negative garrote penalties," Communications in Statistics - Theory and Methods, Taylor & Francis Journals, vol. 51(22), pages 7713-7736, September.
    6. Robinson, P. M., 1973. "Generalized canonical analysis for time series," Journal of Multivariate Analysis, Elsevier, vol. 3(2), pages 141-160, June.
    7. Mark Koudstaal & Fang Yao, 2018. "From multiple Gaussian sequences to functional data and beyond: a Stein estimation approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 80(2), pages 319-342, March.
    8. Pradeep Ravikumar & John Lafferty & Han Liu & Larry Wasserman, 2009. "Sparse additive models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 71(5), pages 1009-1030, November.
    9. P. T. Davies & M. K‐S. Tso, 1982. "Procedures for Reduced‐Rank Regression," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 31(3), pages 244-255, November.
    10. Kwan-Young Bak & Ja-Yong Koo, 2023. "Minimax estimation in multi-task regression under low-rank structures," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 35(1), pages 122-144, January.
    11. Kun Chen & Hongbo Dong & Kung-Sik Chan, 2013. "Reduced rank regression via adaptive nuclear norm penalization," Biometrika, Biometrika Trust, vol. 100(4), pages 901-920.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Goh, Gyuhyeong & Dey, Dipak K. & Chen, Kun, 2017. "Bayesian sparse reduced rank multivariate regression," Journal of Multivariate Analysis, Elsevier, vol. 157(C), pages 14-28.
    2. Kohei Yoshikawa & Shuichi Kawano, 2023. "Sparse reduced-rank regression for simultaneous rank and variable selection via manifold optimization," Computational Statistics, Springer, vol. 38(1), pages 53-75, March.
    3. Luo, Chongliang & Liang, Jian & Li, Gen & Wang, Fei & Zhang, Changshui & Dey, Dipak K. & Chen, Kun, 2018. "Leveraging mixed and incomplete outcomes via reduced-rank modeling," Journal of Multivariate Analysis, Elsevier, vol. 167(C), pages 378-394.
    4. Chen, Canyi & Xu, Wangli & Zhu, Liping, 2022. "Distributed estimation in heterogeneous reduced rank regression: With application to order determination in sufficient dimension reduction," Journal of Multivariate Analysis, Elsevier, vol. 190(C).
    5. Bin Jiang & Anastasios Panagiotelis & George Athanasopoulos & Rob Hyndman & Farshid Vahid, 2016. "Bayesian Rank Selection in Multivariate Regression," Monash Econometrics and Business Statistics Working Papers 6/16, Monash University, Department of Econometrics and Business Statistics.
    6. Zhou, Guofu, 1995. "Small sample rank tests with applications to asset pricing," Journal of Empirical Finance, Elsevier, vol. 2(1), pages 71-93, March.
    7. Kun Chen & Kung-Sik Chan & Nils Chr. Stenseth, 2014. "Source-Sink Reconstruction Through Regularized Multicomponent Regression Analysis-With Application to Assessing Whether North Sea Cod Larvae Contributed to Local Fjord Cod in Skagerrak," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(506), pages 560-573, June.
    8. Chao, Shih-Kang & Härdle, Wolfgang K. & Huang, Chen, 2018. "Multivariate factorizable expectile regression with application to fMRI data," Computational Statistics & Data Analysis, Elsevier, vol. 121(C), pages 1-19.
    9. Marie Levakova & Susanne Ditlevsen, 2024. "Penalisation Methods in Fitting High‐Dimensional Cointegrated Vector Autoregressive Models: A Review," International Statistical Review, International Statistical Institute, vol. 92(2), pages 160-193, August.
    10. Chao, Shih-Kang & Härdle, Wolfgang Karl & Huang, Chen, 2016. "Multivariate factorisable sparse asymmetric least squares regression," SFB 649 Discussion Papers 2016-058, Humboldt University Berlin, Collaborative Research Center 649: Economic Risk.
    11. Lian, Heng & Kim, Yongdai, 2016. "Nonconvex penalized reduced rank regression and its oracle properties in high dimensions," Journal of Multivariate Analysis, Elsevier, vol. 143(C), pages 383-393.
    12. Fujikoshi, Yasunori & Sakurai, Tetsuro, 2016. "High-dimensional consistency of rank estimation criteria in multivariate linear model," Journal of Multivariate Analysis, Elsevier, vol. 149(C), pages 199-212.
    13. Mishra, Aditya & Dey, Dipak K. & Chen, Yong & Chen, Kun, 2021. "Generalized co-sparse factor regression," Computational Statistics & Data Analysis, Elsevier, vol. 157(C).
    14. Yang, Yuehan & Xia, Siwei & Yang, Hu, 2023. "Multivariate sparse Laplacian shrinkage for joint estimation of two graphical structures," Computational Statistics & Data Analysis, Elsevier, vol. 178(C).
    15. repec:hum:wpaper:sfb649dp2016-058 is not listed on IDEAS
    16. Chao, Shih-Kang & Härdle, Wolfgang Karl & Yuan, Ming, 2015. "Factorisable sparse tail event curves," SFB 649 Discussion Papers 2015-034, Humboldt University Berlin, Collaborative Research Center 649: Economic Risk.
    17. Fan, Jianqing & Gong, Wenyan & Zhu, Ziwei, 2019. "Generalized high-dimensional trace regression via nuclear norm regularization," Journal of Econometrics, Elsevier, vol. 212(1), pages 177-202.
    18. Burkett, John P., 1998. "Bureaucratic behavior modeled by reduced-rank regression: The case of expenditures from the Soviet state budget," Journal of Economic Behavior & Organization, Elsevier, vol. 34(1), pages 173-187, January.
    19. Kargin, Vladislav, 2015. "On estimation in the reduced-rank regression with a large number of responses and predictors," Journal of Multivariate Analysis, Elsevier, vol. 140(C), pages 377-394.
    20. repec:hum:wpaper:sfb649dp2015-034 is not listed on IDEAS
    21. Alain-Philippe Fortin & Patrick Gagliardini & O. Scaillet, 2022. "Eigenvalue tests for the number of latent factors in short panels," Swiss Finance Institute Research Paper Series 22-81, Swiss Finance Institute.
    22. Hang Yu & Yuanjia Wang & Donglin Zeng, 2023. "A general framework of nonparametric feature selection in high‐dimensional data," Biometrics, The International Biometric Society, vol. 79(2), pages 951-963, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stpapr:v:66:y:2025:i:3:d:10.1007_s00362-025-01676-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.