Regression Analysis

Assume we measure the insulin level $$Y_1, \ldots , Y_n$$ Y 1 , … , Y n of n persons. Every person has a different weight $$X_1, \ldots , X_n$$ X 1 , … , X n . Can we somehow explain the insulin level using the weights? This is the general context of regression analysis. There are different reasons why such a question might be of interest. For instance, a scientist could be interested in understanding the mechanics behind insulin level, i.e., which factor influences the insulin level and how? Other scientists may only be interested in predicting the insulin level. One common way to achieve this is to find a way to express the conditional expectation of Y given X. Call the function $$m(X) = \mathbb {E}(Y | X)$$ m ( X ) = E ( Y | X ) the regression function. This chapter is dedicated to methods that estimate parametric forms $$m(X, \vartheta )$$ m ( X , ϑ ) under various assumptions. We start with the classical linear models that assume that $$m(X, \vartheta ) = \vartheta ^\top X$$ m ( X , ϑ ) = ϑ ⊤ X is linear in X while Y follows a normal distribution first under independence assumptions and later under certain correlation assumptions. Afterward, we allow other distributions for Y like the negative-binomial distribution which lead to the classical generalized linear models. The chapter concludes with semi-parametric models, i.e., we do not explicitly assume a distribution for Y but the regression function $$m(X, \vartheta )$$ m ( X , ϑ ) still depends on some (multi-dimensional) parameter $$\vartheta $$ ϑ .