Choice of Priors and Variable Selection in Bayesian Regression

This study is focused on the applications of the Bayes theory to Normal linear Regression model in choosing prior distributions for the parameters of interest and in the selection of variables for inclusion/deletion from a model-in the case of a reduced model. For the choice of prior distribution for the regression parameters β, two choices of priors were employed, these are: (i) the Non-informative (vague) prior and (ii) the conjugate prior. The vague prior is from a vague uniform distribution with parameters β and logσ2, while the conjugate prior is from a t-distribution with mean zero, variance σ and n-1 degrees of freedom. The likelihood function for the Normal distribution was used to revise this distribution in both cases to obtain the posterior distribution. This posterior distribution was found to be multivariate t-distribution for β in the case of the vague prior and the multivariate Standard t distribution in the case of the conjugate prior. The distributions breakdown their univariate cases for each βj parameter. The speed of convergence to the posterior distributions were monitored as an indication of which βj should be added or deleted from a reduce model this was done by running MCMC samples for 5000, 10000, 15000, 20000, 25000 and 30000 samples. On the variable selection method, the Stochastic Variable selection was employed. This makes use of a latent variable γ to monitor the posterior distribution of each of the parameters of interest to determine which of the independent variables should be added to the new model. At the end of the work it was realized that for an appropriate choice of posterior distribution to be obtained, an appropriate choice of prior must be used. However when the prior distribution is unknown, the vague prior distribution is a plausible choice.