IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v313y2024i3p858-870.html

Convex support vector regression

Author

Listed:
  • Liao, Zhiqiang
  • Dai, Sheng
  • Kuosmanen, Timo

Abstract

Nonparametric regression subject to convexity or concavity constraints is increasingly popular in economics, finance, operations research, machine learning, and statistics. However, the conventional convex regression based on the least squares loss function often suffers from overfitting and outliers. This paper proposes to address these two issues by introducing the convex support vector regression (CSVR) method, which effectively combines the key elements of convex regression and support vector regression. Numerical experiments demonstrate the performance of CSVR in prediction accuracy and robustness that compares favorably with other state-of-the-art methods.

Suggested Citation

  • Liao, Zhiqiang & Dai, Sheng & Kuosmanen, Timo, 2024. "Convex support vector regression," European Journal of Operational Research, Elsevier, vol. 313(3), pages 858-870.
    • Handle: RePEc:eee:ejores:v:313:y:2024:i:3:p:858-870
    DOI: 10.1016/j.ejor.2023.05.009
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221723003715
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2023.05.009?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Tanaka, Kenta & Managi, Shunsuke, 2021. "Industrial agglomeration effect for energy efficiency in Japanese production plants," Energy Policy, Elsevier, vol. 156(C).
    2. Timo Kuosmanen & Andrew Johnson & Antti Saastamoinen, 2015. "Stochastic Nonparametric Approach to Efficiency Analysis: A Unified Framework," International Series in Operations Research & Management Science, in: Joe Zhu (ed.), Data Envelopment Analysis, edition 127, chapter 7, pages 191-244, Springer.
    3. Preciado Arreola, José Luis & Johnson, Andrew L. & Chen, Xun C. & Morita, Hiroshi, 2020. "Estimating stochastic production frontiers: A one-stage multivariate semiparametric Bayesian concave regression method," European Journal of Operational Research, Elsevier, vol. 287(2), pages 699-711.
    4. Dimitris Bertsimas & Nishanth Mundru, 2021. "Sparse Convex Regression," INFORMS Journal on Computing, INFORMS, vol. 33(1), pages 262-279, January.
    5. Rahul Mazumder & Arkopal Choudhury & Garud Iyengar & Bodhisattva Sen, 2019. "A Computational Framework for Multivariate Convex Regression and Its Variants," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(525), pages 318-331, January.
    6. Dai, Sheng, 2023. "Variable selection in convex quantile regression: L1-norm or L0-norm regularization?," European Journal of Operational Research, Elsevier, vol. 305(1), pages 338-355.
    7. Valero-Carreras, Daniel & Aparicio, Juan & Guerrero, Nadia M., 2021. "Support vector frontiers: A new approach for estimating production functions through support vector machines," Omega, Elsevier, vol. 104(C).
    8. Lee, Chia-Yen & Johnson, Andrew L. & Moreno-Centeno, Erick & Kuosmanen, Timo, 2013. "A more efficient algorithm for Convex Nonparametric Least Squares," European Journal of Operational Research, Elsevier, vol. 227(2), pages 391-400.
    9. Timo Kuosmanen, 2008. "Representation theorem for convex nonparametric least squares," Econometrics Journal, Royal Economic Society, vol. 11(2), pages 308-325, July.
    10. Kuosmanen, Timo & Zhou, Xun & Dai, Sheng, 2020. "How much climate policy has cost for OECD countries?," World Development, Elsevier, vol. 125(C).
    11. Kelley Pace, R. & Barry, Ronald, 1997. "Sparse spatial autoregressions," Statistics & Probability Letters, Elsevier, vol. 33(3), pages 291-297, May.
    12. Kuosmanen, Timo, 2012. "Stochastic semi-nonparametric frontier estimation of electricity distribution networks: Application of the StoNED method in the Finnish regulatory model," Energy Economics, Elsevier, vol. 34(6), pages 2189-2199.
    13. Lin, Dongyu & Foster, Dean P. & Ungar, Lyle H., 2011. "VIF Regression: A Fast Regression Algorithm for Large Data," Journal of the American Statistical Association, American Statistical Association, vol. 106(493), pages 232-247.
    14. Daisuke Yagi & Yining Chen & Andrew L. Johnson & Timo Kuosmanen, 2020. "Shape-Constrained Kernel-Weighted Least Squares: Estimating Production Functions for Chilean Manufacturing Industries," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(1), pages 43-54, January.
    15. Johnson, Andrew L. & Kuosmanen, Timo, 2012. "One-stage and two-stage DEA estimation of the effects of contextual variables," European Journal of Operational Research, Elsevier, vol. 220(2), pages 559-570.
    16. Promit Ghosal & Bodhisattva Sen, 2017. "On Univariate Convex Regression," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 79(2), pages 215-253, August.
    17. Timo Kuosmanen & Andrew L. Johnson, 2010. "Data Envelopment Analysis as Nonparametric Least-Squares Regression," Operations Research, INFORMS, vol. 58(1), pages 149-160, February.
    18. Wang, Yongqiao & Wang, Shouyang & Dang, Chuangyin & Ge, Wenxiu, 2014. "Nonparametric quantile frontier estimation under shape restriction," European Journal of Operational Research, Elsevier, vol. 232(3), pages 671-678.
    19. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    20. Kuosmanen, Timo & Zhou, Xun, 2021. "Shadow prices and marginal abatement costs: Convex quantile regression approach," European Journal of Operational Research, Elsevier, vol. 289(2), pages 666-675.
    21. Eunji Lim, 2014. "On Convergence Rates of Convex Regression in Multiple Dimensions," INFORMS Journal on Computing, INFORMS, vol. 26(3), pages 616-628, August.
    22. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    23. repec:aen:journl:ej41-si1-kuosmanen is not listed on IDEAS
    24. Xi Chen & Qihang Lin & Bodhisattva Sen, 2020. "On Degrees of Freedom of Projection Estimators With Applications to Multivariate Nonparametric Regression," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 115(529), pages 173-186, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Maggioni, Francesca & Spinelli, Andrea, 2025. "A novel robust optimization model for nonlinear Support Vector Machine," European Journal of Operational Research, Elsevier, vol. 322(1), pages 237-253.
    2. Tao, Xiangyang & Peng, Qiaoyu, 2025. "Moral hazard in data envelopment analysis benchmarking," European Journal of Operational Research, Elsevier, vol. 327(1), pages 203-217.
    3. Wu, Xiaofei & Liang, Rongmei & Zhang, Zhimin & Cui, Zhenyu, 2025. "A unified consensus-based parallel algorithm for high-dimensional regression with combined regularizations," Computational Statistics & Data Analysis, Elsevier, vol. 203(C).
    4. Shan, Shan, 2024. "Unveiling Socioeconomic Factors Shaping Global Food Prices and Security: A Machine Learning Approach," Asian Journal of Agriculture and Development, Southeast Asian Regional Center for Graduate Study and Research in Agriculture (SEARCA), vol. 21(2), December.
    5. Huang, Ling-Wei & Shao, Yuan-Hai & Lv, Xiao-Jing & Li, Chun-Na, 2024. "Large-scale robust regression with truncated loss via majorization-minimization algorithm," European Journal of Operational Research, Elsevier, vol. 319(2), pages 494-504.
    6. Mostafa M. Radwan & Elsaid M. M. Zahran & Osama Dawoud & Ziyad Abunada & Ahmad Mousa, 2024. "Comparative Analysis of Asphalt Pavement Condition Prediction Models," Sustainability, MDPI, vol. 17(1), pages 1-22, December.
    7. Zhiqiang Liao & Zhaonan Qu, 2024. "Structured Lasso for convex nonparametric least squares: An application to Swedish electricity distribution networks," Papers 2409.01911, arXiv.org, revised Oct 2025.
    8. Ali Emrouznejad & Victor Podinovski & Vincent Charles & Chixiao Lu & Amir Moradi-Motlagh, 2025. "Rajiv Banker’s lasting impact on data envelopment analysis," Annals of Operations Research, Springer, vol. 351(2), pages 1225-1264, August.
    9. Ali Sahleh & Maziar Salahi & Sadegh Eskandari & Tahereh Khodamoradi, 2025. "Extended twin parametric margin support vector regression," OPSEARCH, Springer;Operational Research Society of India, vol. 62(2), pages 682-705, June.
    10. Xie, Jianhui & Pan, Zhewen, 2025. "Structure identification for partially linear partially concave models," European Journal of Operational Research, Elsevier, vol. 324(1), pages 142-154.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dai, Sheng & Kuosmanen, Timo & Zhou, Xun, 2023. "Generalized quantile and expectile properties for shape constrained nonparametric estimation," European Journal of Operational Research, Elsevier, vol. 310(2), pages 914-927.
    2. Dai, Sheng, 2023. "Variable selection in convex quantile regression: L1-norm or L0-norm regularization?," European Journal of Operational Research, Elsevier, vol. 305(1), pages 338-355.
    3. Dai, Sheng & Kuosmanen, Natalia & Kuosmanen, Timo & Liesiö, Juuso, 2025. "Optimal resource allocation: Convex quantile regression approach," European Journal of Operational Research, Elsevier, vol. 324(1), pages 221-230.
    4. Wen, Xiaojie & Yao, Shunbo & Sauer, Johannes, 2022. "Shadow prices and abatement cost of soil erosion in Shaanxi Province, China: Convex expectile regression approach," Ecological Economics, Elsevier, vol. 201(C).
    5. Zhiqiang Liao & Zhaonan Qu, 2024. "Structured Lasso for convex nonparametric least squares: An application to Swedish electricity distribution networks," Papers 2409.01911, arXiv.org, revised Oct 2025.
    6. Zhiqiang Liao & Sheng Dai & Eunji Lim & Timo Kuosmanen, 2024. "Overfitting Reduction in Convex Regression," Papers 2404.09528, arXiv.org, revised Oct 2024.
    7. Cristina Polo & Julián Ramajo & Alejandro Ricci‐Risquete, 2021. "A stochastic semi‐non‐parametric analysis of regional efficiency in the European Union," Regional Science Policy & Practice, Wiley Blackwell, vol. 13(1), pages 7-24, February.
    8. K. Hervé Dakpo & Yann Desjeux & Laure Latruffe, 2023. "Cost of abating excess nitrogen on wheat plots in France: An assessment with multi‐technology modelling," Journal of Agricultural Economics, Wiley Blackwell, vol. 74(3), pages 800-815, September.
    9. Lee, Chia-Yen & Wang, Ke, 2019. "Nash marginal abatement cost estimation of air pollutant emissions using the stochastic semi-nonparametric frontier," European Journal of Operational Research, Elsevier, vol. 273(1), pages 390-400.
    10. Gabriel Villa & Sebastián Lozano, 2026. "Semi-parametric estimation of the team wage frontier of European football technology," Annals of Operations Research, Springer, vol. 359(3), pages 2655-2679, April.
    11. Eunji Lim, 2025. "Estimating a Function and Its Derivatives Under a Smoothness Condition," Mathematics of Operations Research, INFORMS, vol. 50(2), pages 1112-1138, May.
    12. Kuosmanen, Timo & Zhou, Xun, 2021. "Shadow prices and marginal abatement costs: Convex quantile regression approach," European Journal of Operational Research, Elsevier, vol. 289(2), pages 666-675.
    13. Timo Kuosmanen & Yong Tan & Sheng Dai, 2023. "Performance analysis of English hospitals during the first and second waves of the coronavirus pandemic," Health Care Management Science, Springer, vol. 26(3), pages 447-460, September.
    14. Xie, Jianhui & Pan, Zhewen, 2025. "Structure identification for partially linear partially concave models," European Journal of Operational Research, Elsevier, vol. 324(1), pages 142-154.
    15. Jose M. Cordero & Cristina Polo & Daniel Santín, 2020. "Assessment of new methods for incorporating contextual variables into efficiency measures: a Monte Carlo simulation," Operational Research, Springer, vol. 20(4), pages 2245-2265, December.
    16. Tsionas, Mike G., 2023. "Joint production in stochastic non-parametric envelopment of data with firm-specific directions," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1336-1347.
    17. Ali Emrouznejad & Victor Podinovski & Vincent Charles & Chixiao Lu & Amir Moradi-Motlagh, 2025. "Rajiv Banker’s lasting impact on data envelopment analysis," Annals of Operations Research, Springer, vol. 351(2), pages 1225-1264, August.
    18. Tsionas, Mike, 2022. "Efficiency estimation using probabilistic regression trees with an application to Chilean manufacturing industries," International Journal of Production Economics, Elsevier, vol. 249(C).
    19. Kuosmanen, Timo & Johnson, Andrew, 2017. "Modeling joint production of multiple outputs in StoNED: Directional distance function approach," European Journal of Operational Research, Elsevier, vol. 262(2), pages 792-801.
    20. Maziotis, Alexandros & Sala-Garrido, Ramon & Mocholi-Arce, Manuel & Molinos-Senante, Maria, 2023. "Cost and quality of service performance in the Chilean water industry: A comparison of stochastic approaches," Structural Change and Economic Dynamics, Elsevier, vol. 67(C), pages 211-219.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:313:y:2024:i:3:p:858-870. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.