IDEAS home Printed from https://ideas.repec.org/a/spr/compst/v39y2024i4d10.1007_s00180-024-01470-9.html
   My bibliography  Save this article

Variational Bayesian Lasso for spline regression

Author

Listed:
  • Larissa C. Alves

    (State University of Campinas: Universidade Estadual de Campinas)

  • Ronaldo Dias

    (State University of Campinas: Universidade Estadual de Campinas)

  • Helio S. Migon

    (State University of Campinas: Universidade Estadual de Campinas)

Abstract

This work presents a new scalable automatic Bayesian Lasso methodology with variational inference for non-parametric splines regression that can capture the non-linear relationship between a response variable and predictor variables. Note that under non-parametric point of view the regression curve is assumed to lie in a infinite dimension space. Regression splines use a finite approximation of this infinite space, representing the regression function by a linear combination of basis functions. The crucial point of the approach is determining the appropriate number of bases or equivalently number of knots, avoiding over-fitting/under-fitting. A decision-theoretic approach was devised for knot selection. Comprehensive simulation studies were conducted in challenging scenarios to compare alternative criteria for knot selection, thereby ensuring the efficacy of the proposed algorithms. Additionally, the performance of the proposed method was assessed using real-world datasets. The novel procedure demonstrated good performance in capturing the underlying data structure by selecting the appropriate number of knots/basis.

Suggested Citation

  • Larissa C. Alves & Ronaldo Dias & Helio S. Migon, 2024. "Variational Bayesian Lasso for spline regression," Computational Statistics, Springer, vol. 39(4), pages 2039-2064, June.
  • Handle: RePEc:spr:compst:v:39:y:2024:i:4:d:10.1007_s00180-024-01470-9
    DOI: 10.1007/s00180-024-01470-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00180-024-01470-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s00180-024-01470-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ullah, Aman & Zinde-Walsh, Victoria, 1985. "Estimation and testing in a regression model with spherically symmetric errors," Economics Letters, Elsevier, vol. 17(1-2), pages 127-132.
    2. Smith, Michael & Kohn, Robert, 1996. "Nonparametric regression using Bayesian variable selection," Journal of Econometrics, Elsevier, vol. 75(2), pages 317-343, December.
    3. Chenlei Leng & Minh-Ngoc Tran & David Nott, 2014. "Bayesian adaptive Lasso," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 66(2), pages 221-244, April.
    4. David M. Blei & Alp Kucukelbir & Jon D. McAuliffe, 2017. "Variational Inference: A Review for Statisticians," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(518), pages 859-877, April.
    5. D. G. T. Denison & B. K. Mallick & A. F. M. Smith, 1998. "Automatic Bayesian curve fitting," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 60(2), pages 333-350.
    6. Kooperberg, Charles & Stone, Charles J., 1991. "A study of logspline density estimation," Computational Statistics & Data Analysis, Elsevier, vol. 12(3), pages 327-347, November.
    7. Ormerod, J. T. & Wand, M. P., 2010. "Explaining Variational Approximations," The American Statistician, American Statistical Association, vol. 64(2), pages 140-153.
    8. Dias, Ronaldo & Garcia, Nancy L., 2007. "Consistent estimator for basis selection based on a proxy of the Kullback-Leibler distance," Journal of Econometrics, Elsevier, vol. 141(1), pages 167-178, November.
    9. Feng Li & Mattias Villani, 2013. "Efficient Bayesian Multivariate Surface Regression," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 40(4), pages 706-723, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mauro Bernardi & Daniele Bianchi & Nicolas Bianco, 2022. "Variational inference for large Bayesian vector autoregressions," Papers 2202.12644, arXiv.org, revised Jun 2023.
    2. Gary Koop & Dimitris Korobilis, 2023. "Bayesian Dynamic Variable Selection In High Dimensions," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 64(3), pages 1047-1074, August.
    3. Bansal, Prateek & Krueger, Rico & Graham, Daniel J., 2021. "Fast Bayesian estimation of spatial count data models," Computational Statistics & Data Analysis, Elsevier, vol. 157(C).
    4. Korobilis, Dimitris & Koop, Gary, 2018. "Variational Bayes inference in high-dimensional time-varying parameter models," Essex Finance Centre Working Papers 22665, University of Essex, Essex Business School.
    5. M. P. Wand, 2000. "A Comparison of Regression Spline Smoothing Procedures," Computational Statistics, Springer, vol. 15(4), pages 443-462, December.
    6. Daziano, Ricardo A., 2022. "Willingness to delay charging of electric vehicles," Research in Transportation Economics, Elsevier, vol. 94(C).
    7. Nolan, Tui H. & Richardson, Sylvia & Ruffieux, Hélène, 2025. "Efficient Bayesian functional principal component analysis of irregularly-observed multivariate curves," Computational Statistics & Data Analysis, Elsevier, vol. 203(C).
    8. Gefang, Deborah & Koop, Gary & Poon, Aubrey, 2023. "Forecasting using variational Bayesian inference in large vector autoregressions with hierarchical shrinkage," International Journal of Forecasting, Elsevier, vol. 39(1), pages 346-363.
    9. Wai-Yin Poon & Hai-Bin Wang, 2014. "Multivariate partially linear single-index models: Bayesian analysis," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 26(4), pages 755-768, December.
    10. Pena, Daniel & Redondas, Dolores, 2006. "Bayesian curve estimation by model averaging," Computational Statistics & Data Analysis, Elsevier, vol. 50(3), pages 688-709, February.
    11. Lopes, Hedibert F. & Dias, Ronaldo, 2011. "Bayesian mixture of parametric and nonparametric density estimation: A Misspecification Problem," Brazilian Review of Econometrics, Sociedade Brasileira de Econometria - SBE, vol. 31(1), March.
    12. Villani, Mattias & Kohn, Robert & Giordani, Paolo, 2007. "Nonparametric Regression Density Estimation Using Smoothly Varying Normal Mixtures," Working Paper Series 211, Sveriges Riksbank (Central Bank of Sweden).
    13. Deborah Gefang & Gary Koop & Aubrey Poon, 2019. "Variational Bayesian Inference in Large Vector Autoregressions with Hierarchical Shrinkage," Discussion Papers in Economics 19/05, Division of Economics, School of Business, University of Leicester.
    14. Leitenstorfer, Florian & Tutz, Gerhard, 2007. "Knot selection by boosting techniques," Computational Statistics & Data Analysis, Elsevier, vol. 51(9), pages 4605-4621, May.
    15. Chenchen Ma & Jing Ouyang & Chun Wang & Gongjun Xu, 2024. "A Note on Improving Variational Estimation for Multidimensional Item Response Theory," Psychometrika, Springer;The Psychometric Society, vol. 89(1), pages 172-204, March.
    16. Badi H. Baltagi & Georges Bresson & Jean-Michel Etienne, 2020. "Growth Empirics: a Bayesian Semiparametric Model With Random Coefficients for a Panel of OECD Countries," Advances in Econometrics, in: Essays in Honor of Cheng Hsiao, volume 41, pages 217-253, Emerald Group Publishing Limited.
    17. Liu, Laura & Moon, Hyungsik Roger & Schorfheide, Frank, 2021. "Panel forecasts of country-level Covid-19 infections," Journal of Econometrics, Elsevier, vol. 220(1), pages 2-22.
    18. Mengbing Li & Daniel E. Park & Maliha Aziz & Cindy M. Liu & Lance B. Price & Zhenke Wu, 2023. "Integrating sample similarities into latent class analysis: a tree‐structured shrinkage approach," Biometrics, The International Biometric Society, vol. 79(1), pages 264-279, March.
    19. Zhao, Kaifeng & Lian, Heng, 2014. "Variational inferences for partially linear additive models with variable selection," Computational Statistics & Data Analysis, Elsevier, vol. 80(C), pages 223-239.
    20. Stefan Lang & Eva-Maria Pronk & Ludwig Fahrmeir, 2002. "Function estimation with locally adaptive dynamic models," Computational Statistics, Springer, vol. 17(4), pages 479-499, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:compst:v:39:y:2024:i:4:d:10.1007_s00180-024-01470-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.