Statistical Methods: Regression Analysis

Regression analysis is arguably one of the most commonly used and misused statistical techniques in business and other disciplines. In this chapter we systematically develop linear regression modeling of data. Chapter 6 on Basic inference is all the prerequisite that is required for this chapter. We start with motivating examples (Sect. 2). Section 3 deals with the methods and diagnostics for linear regression. We start with a discussion on what is regression and linear regression, in particular, and why it is important (Sect. 3.1). In Sect. 3.2, we describe the descriptive statistics and basic exploratory analysis for a data set. We are now ready to describe the linear regression model and the assumptions made to get good estimates and tests related to the parameters in the model (Sect. 3.3). Sections 3.4 and 3.5 are devoted to the development of the basic inference and interpretations of the regression output when there is only one regressor and when there are more regressors respectively. In Sect. 3.6, we take the help of the famous Anscombe (1973) data sets to demonstrate the need for further analysis. In Sect. 3.7, we develop the basic building blocks to be used in constructing the diagnostics. In Sect. 3.8, we use various residual plots to check whether there are basic departures from the assumptions and to see if some transformations on the regressors are warranted. Suppose we have developed a linear regression model using some regressors. We find that we have data on one more possible regressor. Should we bring in this variable as an additional regressor, given that the other regressors are already included? This is what is explored through the added variable plot in Sect. 3.9.