Linear Models

Infinite exchangeability of a sequence of random variables, here denoted $$y_1,y_2,\ldots $$ y 1 , y 2 , … , is a useful simplifying assumption for illustrating many of the fundamental ideas presented in the preceding chapters. However, in many practical situations, this would be too limiting as a modelling assumption; often there will be additional available information $$x_i$$ x i pertaining to each random quantity $$y_i$$ y i which affects probabilistic beliefs about the value which $$y_i$$ y i is likely to take. In the language of statistical regressionRegression modelling, the random variables of interest $$y_1,y_2,\ldots $$ y 1 , y 2 , … are referred to as responseResponse variable variables; they are believed to have a statistical dependence on the corresponding element of the sequence of so-called covariatesCovariates or predictorsPredictors $$x_1,x_2,\ldots $$ x 1 , x 2 , … which have either been determined or observed. Regression modelling is concerned with building statistical models for the conditional distribution of each $$y_i$$ y i given $$x_i$$ x i , primarily through specifying the mean value for $$y_i$$ y i having some functional relationship to $$x_i$$ x i (referred to as the regression function). The simplest functional relationship is the linear modelLinear model. With assumed Gaussian errorsErrors in the response variable, the elegant least squares estimation equations from non-Bayesian statistical linear models extend naturally to the Bayesian case. Despite the apparent rigidity of a linearity assumption, consideration of different transformations of either the covariates or the response variable can provide a surprisingly flexible modelling framework.