Regression Model

The generalized maximum entropy model and minimum divergence estimation are examined in a framework of regression paradigm, which is one of the most typical applications in supervised learning. This chapter begins with the linear regression analysis, in which the theory for the least squares estimator (LSE) has been established in the nineteenth century. Under the normal distribution model, the maximum likelihood estimator (MLE) is equal to the LSE, in which the Pythagorean theoremPythagoras theorem holds via the Kullback-Leibler (KL) divergence in an elementary manner. This property is generalized that under the t-distribution modelT-distribution model, the $$\gamma $$ γ -power estimator is equal to the LSE with the power $$\gamma $$ γ adjusted to the degree of freedom of the t-distribution. Similarly, the Pythagorean theoremPythagoras theorem holds for the $$\gamma $$ γ -power divergence. Next, we consider the applications of $$\varphi $$ φ -path using the Kolmogorov-Nagumo mean. A quasi-linear modelingQuasi-linear regression model in a regression setting is introduced. Finally, we discuss a regression approach on the space of positive-definite matrices in a context of manifold learnings, in which a problem of human color perception is challenged.