Optimization Techniques for Twin Support Vector Machines in Primal Space

In this chapter, we examine the twin support vector machine (TWSVM) for binary data classification, a model originally introduced by Jayadeva et al. (IEEE Trans Pattern Anal Mach Intell 29(5):905–910, 2007). TWSVM builds on the generalized eigenvalue proximal support vector machine (GEPSVM) developed by Mangasarian et al. (IEEE Trans Pattern Anal Mach Intell 28(1):69–74, 2006). Both TWSVM and GEPSVM aim to separate data classes using two nonparallel hyperplanes, with each hyperplane positioned closer to one class while remaining farther from the other. However, their approaches differ significantly: GEPSVM relies on eigenvectors derived from generalized eigenvalue problems, while TWSVM adheres more closely to the traditional support vector machine (SVM) framework by solving two smaller quadratic programming problems (QPPs) instead of one large one, resulting in improved computational efficiency. Shao et al. (IEEE Trans Neural Netw 22(6):962–968, 1998) later introduced an enhanced version of TWSVM, called twin bounded support vector machines (TBSVMs). The primary advantage of TBSVM over TWSVM is its inclusion of structural risk minimization by adding a regularization term. Rooted in statistical learning theory, this modification enhances classification performance. Another TWSVM variant, the least squares twin support vector machine (LS-TSVM), was proposed by Kumar and Gopal (Expert Syst Appl 36(4):7535–7543, 2009). This model addresses some TWSVM limitations by solving two sets of linear equations to directly obtain two nonparallel planes. However, LS-TSVM applies the principle of empirical risk minimization rather than structural risk minimization, focusing solely on minimizing training error, which can increase susceptibility to overfitting. To address this, Xu et al. (J Inf Comput Sci 9(4):1063–1071, 2012) proposed an improved LS-TSVM version, enhancing classifier accuracy. In this chapter, we will explore various optimization techniques for twin support vector machines in the primal space.