IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v261y2017i2p772-788.html
   My bibliography  Save this article

Optimization approaches to Supervised Classification

Author

Listed:
  • Pedro Duarte Silva, A.

Abstract

The Supervised Classification problem, one of the oldest and most recurrent problems in applied data analysis, has always been analyzed from many different perspectives. When the emphasis is placed on its overall goal of developing classification rules with minimal classification cost, Supervised Classification can be understood as an optimization problem. On the other hand, when the focus is in modeling the uncertainty involved in the classification of future unknown entities, it can be formulated as a statistical problem. Other perspectives that pay particular attention to pattern recognition and machine learning aspects of Supervised Classification have also a long history that has lead to influential insights and different methodologies.

Suggested Citation

  • Pedro Duarte Silva, A., 2017. "Optimization approaches to Supervised Classification," European Journal of Operational Research, Elsevier, vol. 261(2), pages 772-788.
  • Handle: RePEc:eee:ejores:v:261:y:2017:i:2:p:772-788
    DOI: 10.1016/j.ejor.2017.02.020
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S037722171730142X
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ejor.2017.02.020?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rubin, P. A., 1994. "A comment regarding polynomial discriminant functions," European Journal of Operational Research, Elsevier, vol. 72(1), pages 29-31, January.
    2. Kristin P. Bennett & Erin J. Bredensteiner, 1997. "A Parametric Optimization Method for Machine Learning," INFORMS Journal on Computing, INFORMS, vol. 9(3), pages 311-318, August.
    3. Dellepiane, Umberto & Palagi, Laura, 2015. "Using SVM to combine global heuristics for the Standard Quadratic Problem," European Journal of Operational Research, Elsevier, vol. 241(3), pages 596-605.
    4. Liu, Yufeng & Helen Zhang, Hao & Park, Cheolwoo & Ahn, Jeongyoun, 2007. "Support vector machines with adaptive Lq penalty," Computational Statistics & Data Analysis, Elsevier, vol. 51(12), pages 6380-6394, August.
    5. Martens, David & Baesens, Bart & Van Gestel, Tony & Vanthienen, Jan, 2007. "Comprehensible credit scoring models using rule extraction from support vector machines," European Journal of Operational Research, Elsevier, vol. 183(3), pages 1466-1476, December.
    6. YichaoWu, & Liu, Yufeng, 2007. "Robust Truncated Hinge Loss Support Vector Machines," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 974-983, September.
    7. Antonie Stam, 1997. "Nontraditional approaches to statistical classification: Some perspectives on L_p-norm methods," Annals of Operations Research, Springer, vol. 74(0), pages 1-36, November.
    8. Yanev, N. & Balev, S., 1999. "A combinatorial approach to the classification problem," European Journal of Operational Research, Elsevier, vol. 115(2), pages 339-350, June.
    9. J J Glen, 2005. "Mathematical programming models for piecewise-linear discriminant analysis," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 56(3), pages 331-341, March.
    10. Silva, Antonio Pedro Duarte & Stam, Antonie, 1994. "Second order mathematical programming formulations for discriminant analysis," European Journal of Operational Research, Elsevier, vol. 72(1), pages 4-22, January.
    11. A. Duarte Silva & Antonie Stam, 1997. "A mixed integer programming algorithm for minimizing the training sample misclassification cost in two-group classification," Annals of Operations Research, Springer, vol. 74(0), pages 129-157, November.
    12. Robert Pavur & Pradit Wanarat & Constantine Loucopoulos, 1997. "Examination of the classificatory performance of MIP models with secondary goals for the two-group discriminant problem," Annals of Operations Research, Springer, vol. 74(0), pages 173-189, November.
    13. Emilio Carrizosa & Belén Martín-Barragán & Frank Plastria & Dolores Romero Morales, 2007. "On the Selection of the Globally Optimal Prototype Subset for Nearest-Neighbor Classification," INFORMS Journal on Computing, INFORMS, vol. 19(3), pages 470-479, August.
    14. John W. Chinneck, 1997. "Finding a Useful Subset of Constraints for Analysis in an Infeasible Linear Program," INFORMS Journal on Computing, INFORMS, vol. 9(2), pages 164-174, May.
    15. J. Paul Brooks, 2011. "Support Vector Machines with the Ramp Loss and the Hard Margin Loss," Operations Research, INFORMS, vol. 59(2), pages 467-479, April.
    16. Minghe Sun, 2011. "A Mixed Integer Programming Model For Multiple-Class Discriminant Analysis," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 10(04), pages 589-612.
    17. Paul Rubin, 1997. "Solving mixed integer classification problems by decomposition," Annals of Operations Research, Springer, vol. 74(0), pages 51-64, November.
    18. Saïd Hanafi & Nicola Yanev, 2011. "Tabu search approaches for solving the two-group classification problem," Annals of Operations Research, Springer, vol. 183(1), pages 25-46, March.
    19. Jianqing Fan & Yang Feng & Xin Tong, 2012. "A road to classification in high dimensional space: the regularized optimal affine discriminant," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 74(4), pages 745-771, September.
    20. Adem, Jan & Gochet, Willy, 2006. "Mathematical programming based heuristics for improving LP-generated classifiers for the multiclass supervised classification problem," European Journal of Operational Research, Elsevier, vol. 168(1), pages 181-199, January.
    21. Ognian Asparoukhov & Antonie Stam, 1997. "Mathematical programming formulations for two-group classification with binary variables," Annals of Operations Research, Springer, vol. 74(0), pages 89-112, November.
    22. Gianni Codato & Matteo Fischetti, 2006. "Combinatorial Benders' Cuts for Mixed-Integer Linear Programming," Operations Research, INFORMS, vol. 54(4), pages 756-766, August.
    23. Martin-Barragan, Belen & Lillo, Rosa & Romo, Juan, 2014. "Interpretable support vector machines for functional data," European Journal of Operational Research, Elsevier, vol. 232(1), pages 146-155.
    24. Carrizosa, Emilio & Martín-Barragán, Belén & Morales, Dolores Romero, 2011. "Detecting relevant variables and interactions in supervised classification," European Journal of Operational Research, Elsevier, vol. 213(1), pages 260-269, August.
    25. Richard C. Grinold, 1972. "Mathematical Programming Methods of Pattern Classification," Management Science, INFORMS, vol. 19(3), pages 272-289, November.
    26. Unknown, 1986. "Letters," Choices: The Magazine of Food, Farm, and Resource Issues, Agricultural and Applied Economics Association, vol. 1(4), pages 1-9.
    27. J J Glen, 1999. "Integer programming methods for normalisation and variable selection in mathematical programming discriminant analysis models," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 50(10), pages 1043-1053, October.
    28. O. L. Mangasarian & M. E. Thompson, 2006. "Massive Data Classification via Unconstrained Support Vector Machines," Journal of Optimization Theory and Applications, Springer, vol. 131(3), pages 315-325, December.
    29. du Jardin, Philippe & Séverin, Eric, 2012. "Forecasting financial failure using a Kohonen map: A comparative study to improve model stability over time," European Journal of Operational Research, Elsevier, vol. 221(2), pages 378-396.
    30. A. Cassioli & D. Di Lorenzo & M. Locatelli & F. Schoen & M. Sciandrone, 2012. "Machine learning for global optimization," Computational Optimization and Applications, Springer, vol. 51(1), pages 279-303, January.
    31. A. Astorino & M. Gaudioso, 2002. "Polyhedral Separability Through Successive LP," Journal of Optimization Theory and Applications, Springer, vol. 112(2), pages 265-293, February.
    32. O. L. Mangasarian, 1965. "Linear and Nonlinear Separation of Patterns by Linear Programming," Operations Research, INFORMS, vol. 13(3), pages 444-452, June.
    33. Glen, J.J., 2006. "A comparison of standard and two-stage mathematical programming discriminant analysis methods," European Journal of Operational Research, Elsevier, vol. 171(2), pages 496-515, June.
    34. Frank Plastria & Steven De Bruyne & Emilio Carrizosa, 2010. "Alternating local search based VNS for linear classification," Annals of Operations Research, Springer, vol. 174(1), pages 121-134, February.
    35. Freed, Ned & Glover, Fred, 1981. "Simple but powerful goal programming models for discriminant problems," European Journal of Operational Research, Elsevier, vol. 7(1), pages 44-60, May.
    36. K Falangis & J J Glen, 2010. "Heuristics for feature selection in mathematical programming discriminant analysis models," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 61(5), pages 804-812, May.
    37. Olvi L. Mangasarian & W. Nick Street & William H. Wolberg, 1995. "Breast Cancer Diagnosis and Prognosis Via Linear Programming," Operations Research, INFORMS, vol. 43(4), pages 570-577, August.
    38. Dodge, Yadolah, 1987. "An introduction to L1-norm based statistical data analysis," Computational Statistics & Data Analysis, Elsevier, vol. 5(4), pages 239-253, September.
    39. J. M. Liittschwager & C. Wang, 1978. "Integer Programming Solution of a Classification Problem," Management Science, INFORMS, vol. 24(14), pages 1515-1525, October.
    40. P. Du Jardin & E. Séverin, 2012. "Forecasting financial failure using a Kohonen map: a comparative study to improve bankruptcy model over time," Post-Print hal-00801853, HAL.
    41. Lam, Kim Fung & Choo, Eng Ung & Moy, Jane W., 1996. "Minimizing deviations from the group mean: A new linear programming approach for the two-group classification problem," European Journal of Operational Research, Elsevier, vol. 88(2), pages 358-367, January.
    42. John W. Chinneck, 2001. "Fast Heuristics for the Maximum Feasible Subsystem Problem," INFORMS Journal on Computing, INFORMS, vol. 13(3), pages 210-223, August.
    43. Willy Gochet & Antonie Stam & V. Srinivasan & Shaoxiang Chen, 1997. "Multigroup Discriminant Analysis Using Linear Programming," Operations Research, INFORMS, vol. 45(2), pages 213-225, April.
    44. Duarte Silva, António Pedro, 2001. "Efficient Variable Screening for Multivariate Analysis," Journal of Multivariate Analysis, Elsevier, vol. 76(1), pages 35-62, January.
    45. Banks, William J. & Abad, Prakash L., 1994. "On the performance of linear programming heuristics applied on a quadratic transformation in the classification problem," European Journal of Operational Research, Elsevier, vol. 72(1), pages 23-28, January.
    46. J J Glen, 2008. "An additive utility mixed integer programming model for nonlinear discriminant analysis," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(11), pages 1492-1505, November.
    47. Minghe Sun, 2013. "A Multi-Class Support Vector Machine: Theory And Model," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 12(06), pages 1175-1199.
    48. Huber, Peter J., 1987. "The place of the L1-norm in robust estimation," Computational Statistics & Data Analysis, Elsevier, vol. 5(4), pages 255-262, September.
    49. Lee, Yoonkyung & Lin, Yi & Wahba, Grace, 2004. "Multicategory Support Vector Machines: Theory and Application to the Classification of Microarray Data and Satellite Radiance Data," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 67-81, January.
    50. Stein, Oliver, 2012. "How to solve a semi-infinite optimization problem," European Journal of Operational Research, Elsevier, vol. 223(2), pages 312-320.
    51. Ognian Asparouhov & Paul A. Rubin, 2004. "Oscillation Heuristics for the Two-group Classification Problem," Journal of Classification, Springer;The Classification Society, vol. 21(2), pages 255-277, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Blanquero, Rafael & Carrizosa, Emilio & Molero-Río, Cristina & Romero Morales, Dolores, 2020. "Sparsity in optimal randomized classification trees," European Journal of Operational Research, Elsevier, vol. 284(1), pages 255-272.
    2. Baldomero-Naranjo, Marta & Martínez-Merino, Luisa I. & Rodríguez-Chía, Antonio M., 2020. "Tightening big Ms in integer programming formulations for support vector machines with ramp loss," European Journal of Operational Research, Elsevier, vol. 286(1), pages 84-100.
    3. Baumann, P. & Hochbaum, D.S. & Yang, Y.T., 2019. "A comparative study of the leading machine learning techniques and two new optimization algorithms," European Journal of Operational Research, Elsevier, vol. 272(3), pages 1041-1057.
    4. Sandra Benítez-Peña & Rafael Blanquero & Emilio Carrizosa & Pepa Ramírez-Cobo, 2019. "On support vector machines under a multiple-cost scenario," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 13(3), pages 663-682, September.
    5. Laura Palagi, 2019. "Global optimization issues in deep network regression: an overview," Journal of Global Optimization, Springer, vol. 73(2), pages 239-277, February.
    6. Emilio Carrizosa & Cristina Molero-Río & Dolores Romero Morales, 2021. "Mathematical optimization in classification and regression trees," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 29(1), pages 5-33, April.
    7. Astorino, Annabella & Avolio, Matteo & Fuduli, Antonio, 2022. "A maximum-margin multisphere approach for binary Multiple Instance Learning," European Journal of Operational Research, Elsevier, vol. 299(2), pages 642-652.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zopounidis, Constantin & Doumpos, Michael, 2002. "Multicriteria classification and sorting methods: A literature review," European Journal of Operational Research, Elsevier, vol. 138(2), pages 229-246, April.
    2. Mingue Sun, 2009. "Liquidity Risk and Financial Competition: A Mixed Integer Programming Model for Multiple-Class Discriminant Analysis," Working Papers 0102, College of Business, University of Texas at San Antonio.
    3. Glen, J.J., 2006. "A comparison of standard and two-stage mathematical programming discriminant analysis methods," European Journal of Operational Research, Elsevier, vol. 171(2), pages 496-515, June.
    4. J. J. Glen, 2004. "Dichotomous categorical variable formation in mathematical programming discriminant analysis models," Naval Research Logistics (NRL), John Wiley & Sons, vol. 51(4), pages 575-596, June.
    5. Adem, Jan & Gochet, Willy, 2006. "Mathematical programming based heuristics for improving LP-generated classifiers for the multiclass supervised classification problem," European Journal of Operational Research, Elsevier, vol. 168(1), pages 181-199, January.
    6. Stam, Antonie & Ungar, David R., 1995. "RAGNU: A microcomputer package for two-group mathematical programming-based nonparametric classification," European Journal of Operational Research, Elsevier, vol. 86(2), pages 374-388, October.
    7. Loucopoulos, Constantine, 2001. "Three-group classification with unequal misclassification costs: a mathematical programming approach," Omega, Elsevier, vol. 29(3), pages 291-297, June.
    8. Mingue Sun, 2009. "Liquidity Risk and Financial Competition: A Mixed Integer Programming Model for Multiple-Class Discriminant Analysis," Working Papers 0102, College of Business, University of Texas at San Antonio.
    9. Eva K. Lee & Richard J. Gallagher & David A. Patterson, 2003. "A Linear Programming Approach to Discriminant Analysis with a Reserved-Judgment Region," INFORMS Journal on Computing, INFORMS, vol. 15(1), pages 23-41, February.
    10. J J Glen, 2005. "Mathematical programming models for piecewise-linear discriminant analysis," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 56(3), pages 331-341, March.
    11. Sueyoshi, Toshiyuki, 2006. "DEA-Discriminant Analysis: Methodological comparison among eight discriminant analysis approaches," European Journal of Operational Research, Elsevier, vol. 169(1), pages 247-272, February.
    12. Brandner, Hubertus & Lessmann, Stefan & Voß, Stefan, 2013. "A memetic approach to construct transductive discrete support vector machines," European Journal of Operational Research, Elsevier, vol. 230(3), pages 581-595.
    13. Lam, Kim Fung & Moy, Jane W., 2002. "Combining discriminant methods in solving classification problems in two-group discriminant analysis," European Journal of Operational Research, Elsevier, vol. 138(2), pages 294-301, April.
    14. Soulef Smaoui & Belaid Aouni, 2017. "Fuzzy goal programming model for classification problems," Annals of Operations Research, Springer, vol. 251(1), pages 141-160, April.
    15. Sueyoshi, Toshiyuki, 1999. "DEA-discriminant analysis in the view of goal programming," European Journal of Operational Research, Elsevier, vol. 115(3), pages 564-582, June.
    16. Michael O. Olusola & Sydney I. Onyeagu, 2020. "On the binary classification problem in discriminant analysis using linear programming methods," Operations Research and Decisions, Wroclaw University of Science and Technology, Faculty of Management, vol. 30(1), pages 119-130.
    17. Wanarat, Pradit & Pavur, Robert, 1996. "Examining the effect of second-order terms in mathematical programming approaches to the classification problem," European Journal of Operational Research, Elsevier, vol. 93(3), pages 582-601, September.
    18. Sueyoshi, Toshiyuki, 2004. "Mixed integer programming approach of extended DEA-discriminant analysis," European Journal of Operational Research, Elsevier, vol. 152(1), pages 45-55, January.
    19. J J Glen, 2008. "An additive utility mixed integer programming model for nonlinear discriminant analysis," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(11), pages 1492-1505, November.
    20. Parag Pendharkar & Marvin Troutt, 2014. "Interactive classification using data envelopment analysis," Annals of Operations Research, Springer, vol. 214(1), pages 125-141, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:261:y:2017:i:2:p:772-788. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.