IDEAS home Printed from https://ideas.repec.org/p/crs/wpaper/2017-67.html
   My bibliography  Save this paper

Slope heuristics and V-Fold model selection in heteroscedastic regression using strongly localized bases

Author

Listed:
  • Fabien Navarro

    (CREST;ENSAI)

  • Adrien Saumard

    (CREST;ENSAI)

Abstract

We investigate the optimality for model selection of the so-called slope heuristics, V -fold cross-validation and V -fold penalization in a heteroscedatic with random design regression context. We consider a new class of linear models that we call strongly localized bases and that generalize histograms, piecewise polynomials and compactly supported wavelets. We derive sharp oracle inequalities that prove the asymptotic optimality of the slope heuristics—when the optimal penalty shape is known—and V -fold penalization. Furthermore, V -fold cross-validation seems to be suboptimal for a ?xed value of V since it recovers asymptotically the oracle learned from a sample size equal to 1-V -1 of the original amount of data. Our results are based on genuine concentration inequalities for the true and empirical excess risks that are of independent interest. We show in our experiments the good behavior of the slope heuristics for the selection of linear wavelet models. Furthermore, V -fold cross-validation and V -fold penalization have comparable e?ciency.

Suggested Citation

  • Fabien Navarro & Adrien Saumard, 2017. "Slope heuristics and V-Fold model selection in heteroscedastic regression using strongly localized bases," Working Papers 2017-67, Center for Research in Economics and Statistics.
    • Handle: RePEc:crs:wpaper:2017-67
    as

    Download full text from publisher

    File URL: http://crest.science/RePEc/wpstorage/2017-67.pdf
    File Function: CREST working paper version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Antoniadis, Anestis & Bigot, Jeremie & Sapatinas, Theofanis, 2001. "Wavelet Estimators in Nonparametric Regression: A Comparative Simulation Study," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 6(i06).
    2. Cai, T. Tony & Brown, Lawrence D., 1999. "Wavelet estimation for samples with random uniform design," Statistics & Probability Letters, Elsevier, vol. 42(3), pages 313-321, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Fabien Navarro & Adrien Saumard, 2017. "E?ciency of the V-fold model selection for localized bases," Working Papers 2017-65, Center for Research in Economics and Statistics.
    2. Christophe Chesneau & Salima El Kolei & Junke Kou & Fabien Navarro, 2019. "Nonparametric estimation in a regression model with additive and multiplicative noise," Papers 1906.07695, arXiv.org, revised Jun 2020.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Autin, Florent & Freyermuth, Jean-Marc & von Sachs, Rainer, 2011. "Combining thresholding rules: a new way to improve the performance of wavelet estimators," LIDAM Discussion Papers ISBA 2011021, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    2. De Canditiis, Daniela, 2014. "A frame based shrinkage procedure for fast oscillating functions," Computational Statistics & Data Analysis, Elsevier, vol. 75(C), pages 142-150.
    3. Yu, Dengdeng & Zhang, Li & Mizera, Ivan & Jiang, Bei & Kong, Linglong, 2019. "Sparse wavelet estimation in quantile regression with multiple functional predictors," Computational Statistics & Data Analysis, Elsevier, vol. 136(C), pages 12-29.
    4. Nilotpal Sanyal & Marco A. R. Ferreira, 2017. "Bayesian Wavelet Analysis Using Nonlocal Priors with an Application to fMRI Analysis," Sankhya B: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 79(2), pages 361-388, November.
    5. Chesneau, Christophe, 2007. "Regression with random design: A minimax study," Statistics & Probability Letters, Elsevier, vol. 77(1), pages 40-53, January.
    6. Umberto Amato & Anestis Antoniadis & Italia Feis & Irène Gijbels, 2022. "Penalized wavelet estimation and robust denoising for irregular spaced data," Computational Statistics, Springer, vol. 37(4), pages 1621-1651, September.
    7. T. Palanisamy & J. Ravichandran, 2015. "A wavelet-based hybrid approach to estimate variance function in heteroscedastic regression models," Statistical Papers, Springer, vol. 56(3), pages 911-932, August.
    8. Florent Autin & Jean-Marc Freyermuth & Rainer Von Sachs, 2014. "Block-threshold-adapted Estimators via a Maxiset Approach," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 41(1), pages 240-258, March.
    9. Vincent Rivoirard, 2004. "Thresholding procedure with priors based on Pareto distributions," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 13(1), pages 213-246, June.
    10. Autin, Florent & Freyermuth, Jean-Marc & von Sachs, Rainer, 2011. "Block-Threshold-Adapted Estimators via a maxiset approach," LIDAM Discussion Papers ISBA 2011017, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    11. Helida Nurcahayani & I Nyoman Budiantara & Ismaini Zain, 2021. "The Curve Estimation of Combined Truncated Spline and Fourier Series Estimators for Multiresponse Nonparametric Regression," Mathematics, MDPI, vol. 9(10), pages 1-22, May.
    12. Serban, Nicoleta, 2010. "Noise reduction for enhanced component identification in multi-dimensional biomolecular NMR studies," Computational Statistics & Data Analysis, Elsevier, vol. 54(4), pages 1051-1065, April.
    13. Aminghafari, Mina & Cheze, Nathalie & Poggi, Jean-Michel, 2006. "Multivariate denoising using wavelets and principal component analysis," Computational Statistics & Data Analysis, Elsevier, vol. 50(9), pages 2381-2398, May.
    14. Luz M. Gómez & Rogério F. Porto & Pedro A. Morettin, 2021. "Nonparametric regression with warped wavelets and strong mixing processes," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 73(6), pages 1203-1228, December.
    15. Oleg Shestakov, 2020. "Wavelet Thresholding Risk Estimate for the Model with Random Samples and Correlated Noise," Mathematics, MDPI, vol. 8(3), pages 1-8, March.
    16. Maarten Jansen & Guy P. Nason & B. W. Silverman, 2009. "Multiscale methods for data on graphs and irregular multidimensional situations," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 71(1), pages 97-125, January.
    17. Zeng, Jing & Wang, Zhenjun & Chen, Guobin, 2021. "Biological characteristics of energy conversion in carbon fixation by microalgae," Renewable and Sustainable Energy Reviews, Elsevier, vol. 152(C).
    18. Fryzlewicz, Piotr, 2007. "Bivariate hard thresholding in wavelet function estimation," LSE Research Online Documents on Economics 25219, London School of Economics and Political Science, LSE Library.
    19. Christophe Chesneau & Jalal Fadili, 2012. "Adaptive wavelet estimation of a function in an indirect regression model," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 96(1), pages 25-46, January.
    20. T. W. Randolph & Y. Yasui, 2006. "Multiscale Processing of Mass Spectrometry Data," Biometrics, The International Biometric Society, vol. 62(2), pages 589-597, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:crs:wpaper:2017-67. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Secretariat General (email available below). General contact details of provider: https://edirc.repec.org/data/crestfr.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.