New Theory of Discriminant Analysis and Cancer Gene Analysis

This chapter explains the “New Theory of Discriminant Analysis after R. Fisher (Theory)” and the first success of cancer gene analysis as its application (Problem 5). The theory consists of four Optimal Linear Discriminant Functions (Optimal LDFs, OLDFs), two facts of discriminant analysis, two methods, and two statistics such as MNM and RatioSV. Section 1.1 summarises the theory and explains new results. Section 1.2 explains two facts as follows: (1) the relation of NM and LDF coefficient that solves Problem 1 (the defect of NM). (2) MNM monotonic decrease that is important for Problem5. Furthermore, we explain the reason why statisticians and machine learning researchers could not solve the cancer gene analysis since 1970. Only RIP and Revised LP-OLDF can decompose microarrays into many SMs. This fact is vital for cancer gene diagnosis. Section 1.3 introduces five severe problems of discriminant analysis. Section 1.4 introduces four OLDFs and three SVMs in addition to statistical discriminant functions. Section 1.5 explains the Matryoshka feature selection method (Method2) that solves Problem5 completely. Section 1.6 describes how to validate Method2 by two common data such as Swiss banknote data and Japanese car data those are LSD. Thus, this section indicates Method2 is useful for LSD including the common data and microarrays. Section 1.7 is the conclusion. We can explain the reason why only RIP and Revised LP-OLDF can decompose the microarray into many SMs. This reason is the answer why statisticians and machine learning researchers could not solve the cancer gene analysis since 1970.