Akaike’s Information Criterion (AIC) and the Third Variance

One statistic in selecting a multiple regression equation is Mallows’ C p , which is an approximation of the error variance of the estimates given by a multiple regression equation. This error variance multiplied by n is written as $$\displaystyle\begin{array}{rcl} E{\Bigl [\sum _{i=1}^{n}{(\hat{y}_{ i} - E[y_{i}])}^{2}\Bigr ]}& =& E{\Bigl [{\bigl (\mathbf{H}(\tilde{\mathbf{y}}+\boldsymbol{\epsilon }) -\tilde{{\mathbf{y}}\bigr )}}^{t}{\bigl (\mathbf{H}(\tilde{\mathbf{y}}+\boldsymbol{\epsilon }) -\tilde{\mathbf{y}}\bigr )}\Bigr ]} \\ & =& E{\Bigl [{\bigl ((\mathbf{H} -\mathbf{I})\tilde{\mathbf{y}} +{ \mathbf{H}\boldsymbol{\epsilon }\bigr )}}^{t}{\bigl ((\mathbf{H} -\mathbf{I})\tilde{\mathbf{y}} + \mathbf{H}\boldsymbol{\epsilon }\bigr )}\Bigr ]} \\ & =& \tilde{{\mathbf{y}}}^{t}(\mathbf{I} -\mathbf{H})\tilde{\mathbf{y}} + E{[\boldsymbol{\epsilon }}^{t}\mathbf{H}\boldsymbol{\epsilon }] \\ & =& \tilde{{\mathbf{y}}}^{t}(\mathbf{I} -\mathbf{H})\tilde{\mathbf{y}} + {(q + 1)\sigma }^{2}, {}\end{array}$$ where H is a hat matrix (Eq. ( 4.13 ) (page 164)) and $$\tilde{\mathbf{y}}$$ are true values, that is, $$\tilde{y}_{i} = E[y_{i}]$$ holds. The relation H 2 = H (Eq. ( 4.28 ) (page 167)) is used here, along with Eq. ( 4.34 ) (page 174). In deriving Eq. ( 4.32 ) (page 174), $$\mathbf{H}\tilde{\mathbf{y}} =\tilde{ \mathbf{y}}$$ was assumed. However, this relationship is not assumed here.