IDEAS home Printed from https://ideas.repec.org/p/crs/wpaper/2017-67.html
   My bibliography  Save this paper

Slope heuristics and V-Fold model selection in heteroscedastic regression using strongly localized bases

Author

Listed:
  • Fabien Navarro

    (CREST;ENSAI)

  • Adrien Saumard

    (CREST;ENSAI)

Abstract

We investigate the optimality for model selection of the so-called slope heuristics, V -fold cross-validation and V -fold penalization in a heteroscedatic with random design regression context. We consider a new class of linear models that we call strongly localized bases and that generalize histograms, piecewise polynomials and compactly supported wavelets. We derive sharp oracle inequalities that prove the asymptotic optimality of the slope heuristics—when the optimal penalty shape is known—and V -fold penalization. Furthermore, V -fold cross-validation seems to be suboptimal for a ?xed value of V since it recovers asymptotically the oracle learned from a sample size equal to 1-V -1 of the original amount of data. Our results are based on genuine concentration inequalities for the true and empirical excess risks that are of independent interest. We show in our experiments the good behavior of the slope heuristics for the selection of linear wavelet models. Furthermore, V -fold cross-validation and V -fold penalization have comparable e?ciency.

Suggested Citation

  • Fabien Navarro & Adrien Saumard, 2017. "Slope heuristics and V-Fold model selection in heteroscedastic regression using strongly localized bases," Working Papers 2017-67, Center for Research in Economics and Statistics.
    • Handle: RePEc:crs:wpaper:2017-67
    as

    Download full text from publisher

    File URL: http://crest.science/RePEc/wpstorage/2017-67.pdf
    File Function: CREST working paper version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Antoniadis, Anestis & Bigot, Jeremie & Sapatinas, Theofanis, 2001. "Wavelet Estimators in Nonparametric Regression: A Comparative Simulation Study," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 6(i06).
    2. Cai, T. Tony & Brown, Lawrence D., 1999. "Wavelet estimation for samples with random uniform design," Statistics & Probability Letters, Elsevier, vol. 42(3), pages 313-321, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Fabien Navarro & Adrien Saumard, 2017. "E?ciency of the V-fold model selection for localized bases," Working Papers 2017-65, Center for Research in Economics and Statistics.
    2. Christophe Chesneau & Salima El Kolei & Junke Kou & Fabien Navarro, 2019. "Nonparametric estimation in a regression model with additive and multiplicative noise," Papers 1906.07695, arXiv.org, revised Jun 2020.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Autin, Florent & Freyermuth, Jean-Marc & von Sachs, Rainer, 2011. "Combining thresholding rules: a new way to improve the performance of wavelet estimators," LIDAM Discussion Papers ISBA 2011021, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    2. De Canditiis, Daniela, 2014. "A frame based shrinkage procedure for fast oscillating functions," Computational Statistics & Data Analysis, Elsevier, vol. 75(C), pages 142-150.
    3. Yu, Dengdeng & Zhang, Li & Mizera, Ivan & Jiang, Bei & Kong, Linglong, 2019. "Sparse wavelet estimation in quantile regression with multiple functional predictors," Computational Statistics & Data Analysis, Elsevier, vol. 136(C), pages 12-29.
    4. Chesneau, Christophe, 2007. "Regression with random design: A minimax study," Statistics & Probability Letters, Elsevier, vol. 77(1), pages 40-53, January.
    5. Umberto Amato & Anestis Antoniadis & Italia Feis & Irène Gijbels, 2022. "Penalized wavelet estimation and robust denoising for irregular spaced data," Computational Statistics, Springer, vol. 37(4), pages 1621-1651, September.
    6. T. Palanisamy & J. Ravichandran, 2015. "A wavelet-based hybrid approach to estimate variance function in heteroscedastic regression models," Statistical Papers, Springer, vol. 56(3), pages 911-932, August.
    7. Autin, Florent & Freyermuth, Jean-Marc & von Sachs, Rainer, 2011. "Block-Threshold-Adapted Estimators via a maxiset approach," LIDAM Discussion Papers ISBA 2011017, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    8. Wooyoung Shin & Yoonsuh Jung, 2023. "Deep support vector quantile regression with non-crossing constraints," Computational Statistics, Springer, vol. 38(4), pages 1947-1976, December.
    9. Zeng, Jing & Wang, Zhenjun & Chen, Guobin, 2021. "Biological characteristics of energy conversion in carbon fixation by microalgae," Renewable and Sustainable Energy Reviews, Elsevier, vol. 152(C).
    10. Beknazaryan, Aleksandr & Sang, Hailin, 2022. "Nonparametric regression with modified ReLU networks," Statistics & Probability Letters, Elsevier, vol. 190(C).
    11. Youssef Taleb & Edward A. K. Cohen, 2021. "Multiresolution analysis of point processes and statistical thresholding for Haar wavelet-based intensity estimation," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 73(2), pages 395-423, April.
    12. Michel H. Montoril & Woojin Chang & Brani Vidakovic, 2019. "Wavelet-Based Estimation of Generalized Discriminant Functions," Sankhya B: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 81(2), pages 318-349, December.
    13. Antoniadis, Anestis & Sapatinas, Theofanis, 2003. "Wavelet methods for continuous-time prediction using Hilbert-valued autoregressive processes," Journal of Multivariate Analysis, Elsevier, vol. 87(1), pages 133-158, October.
    14. Lee, Kichun & Vidakovic, Brani, 2012. "Semi-supervised wavelet shrinkage," Computational Statistics & Data Analysis, Elsevier, vol. 56(6), pages 1681-1691.
    15. repec:ibn:ijspnl:v:8:y:2019:i:4:p:32 is not listed on IDEAS
    16. Céline Lacaux & Aurélie Muller-Gueudin & Radu Ranta & Samy Tindel, 2014. "Convergence and performance of the peeling wavelet denoising algorithm," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 77(4), pages 509-537, May.
    17. Nilotpal Sanyal & Marco A. R. Ferreira, 2017. "Bayesian Wavelet Analysis Using Nonlocal Priors with an Application to fMRI Analysis," Sankhya B: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 79(2), pages 361-388, November.
    18. Florent Autin & Jean-Marc Freyermuth & Rainer Von Sachs, 2014. "Block-threshold-adapted Estimators via a Maxiset Approach," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 41(1), pages 240-258, March.
    19. Vincent Rivoirard, 2004. "Thresholding procedure with priors based on Pareto distributions," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 13(1), pages 213-246, June.
    20. Helida Nurcahayani & I Nyoman Budiantara & Ismaini Zain, 2021. "The Curve Estimation of Combined Truncated Spline and Fourier Series Estimators for Multiresponse Nonparametric Regression," Mathematics, MDPI, vol. 9(10), pages 1-22, May.
    21. Serban, Nicoleta, 2010. "Noise reduction for enhanced component identification in multi-dimensional biomolecular NMR studies," Computational Statistics & Data Analysis, Elsevier, vol. 54(4), pages 1051-1065, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:crs:wpaper:2017-67. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Secretariat General (email available below). General contact details of provider: https://edirc.repec.org/data/crestfr.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.