A Comparative Study of the Fisher’s Discriminant Analysis and Support Vector Machines

The relative merits of Fisher’s Discriminant Analysis (FDA) over Support Vector Machines or vice versa, will remain a bone of contention among statisticians and the machine learning community. This line of thought may be owed to the fact that FDA is due to Fishers R. A., a statistician, whereas SVM is a credit to Vanik and his team of the machine learning. In order to give a clearer picture on the strength and weakness of both classifiers, they are compared in terms of the different theories behind each one. We also look into the ways regularization is carried out by each classifier, and further examine how FDA and SVM respond to lineartransformations. We conclude with examination of the behaviour of FDA and SVM on data, given different scenarios, and in high dimensions too. In the end, we clearly draw out the differences and similarities between the two classifiers, and further highlight features that make each classifier ideal for a given classification problem.