On the Harm that Pretesting Does
AbstractData in econometrics are, as a rule, non-experimental and hence we have to use the same data set to select the model and also to estimate the parameters in the selected model.In standard applied econometrics practice, however, one reports zero bias and some variance of the (pretest) estimators conditional on the selected model.In this paper we find the unconditional moments of the pretest estimator, taking full account of the fact that model selection and estimation are an integrated procedure.We derive the bias, variance, and mean squared error of the pretest estimator, and show what the error is in not reporting the correct moments.This error can be very substantial.We also show that there can be large differences in underreporting between different model selection procedures.Finally, we ask how the underreporting error increases when the number of auxiliary regressors increases.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by Tilburg University, Center for Economic Research in its series Discussion Paper with number 2001-37.
Date of creation: 2001
Date of revision:
Contact details of provider:
Web page: http://center.uvt.nl
econometric models; testing; estimation;
This paper has been announced in the following NEP Reports:
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Kevin Hoover & Stephen J. Perez, 2003.
"Data Mining Reconsidered: Encompassing And The General-To-Specific Approach To Specification Search,"
9727, University of California, Davis, Department of Economics.
- Kevin D. Hoover & Stephen J. Perez, 1999. "Data mining reconsidered: encompassing and the general-to-specific approach to specification search," Econometrics Journal, Royal Economic Society, vol. 2(2), pages 167-191.
- Kevin D. Hoover & Stephen J. Perez, . "Data Mining Reconsidered: Encompassing And The General-To-Specific Approach To Specification Search," Department of Economics 97-27, California Davis - Department of Economics.
- Lovell, Michael C, 1983. "Data Mining," The Review of Economics and Statistics, MIT Press, vol. 65(1), pages 1-12, February.
- Judge, G.G. & Bock, M.E., 1983. "Biased estimation," Handbook of Econometrics, in: Z. Griliches† & M. D. Intriligator (ed.), Handbook of Econometrics, edition 1, volume 1, chapter 10, pages 599-649 Elsevier.
- Mittelhammer, R.C., 1984. "Restricted least squares, pre-test, ols and stein rule estimators: Risk comparisons under model misspecification," Journal of Econometrics, Elsevier, vol. 25(1-2), pages 151-164.
- Hannes Leeb & Benedikt M. Poetscher, 2000.
"The Finite-Sample Distribution of Post-Model-Selection Estimators, and Uniform Versus Non-Uniform Approximations,"
- Leeb, Hannes & P tscher, Benedikt M., 2003. "The Finite-Sample Distribution Of Post-Model-Selection Estimators And Uniform Versus Nonuniform Approximations," Econometric Theory, Cambridge University Press, vol. 19(01), pages 100-142, February.
- Hendry, David F., 2001. "Achievements and challenges in econometric methodology," Journal of Econometrics, Elsevier, vol. 100(1), pages 7-10, January.
- Feldstein, Martin S, 1973. "Multicollinearity and the Mean Square Error of Alternative Estimators," Econometrica, Econometric Society, vol. 41(2), pages 337-46, March.
- repec:att:wimass:9417 is not listed on IDEAS
- Lo, Andrew W & MacKinlay, A Craig, 1990.
"Data-Snooping Biases in Tests of Financial Asset Pricing Models,"
Review of Financial Studies,
Society for Financial Studies, vol. 3(3), pages 431-67.
- Andrew W. Lo & A. Craig MacKinlay, 1991. "Data-Snooping Biases in Tests of Financial Asset Pricing Models," NBER Working Papers 3001, National Bureau of Economic Research, Inc.
- Lo, Andrew W. (Andrew Wen-Chuan) & MacKinlay, Archie Craig, 1955-, 1989. "Data-snooping biases in tests of financial asset pricing models," Working papers 3020-89., Massachusetts Institute of Technology (MIT), Sloan School of Management.
- Giles, Judith A & Giles, David E A, 1993. " Pre-test Estimation and Testing in Econometrics: Recent Developments," Journal of Economic Surveys, Wiley Blackwell, vol. 7(2), pages 145-97, June.
- Roehrig, C.S., 1984. "Optimal critical regions for pre-test estimators using a Bayes risk criterion," Journal of Econometrics, Elsevier, vol. 25(1-2), pages 3-14.
- Kenneth D. West, 1994.
"Asymptotic Inference About Predictive Ability,"
- Jan R. Magnus & J. Durbin, 1999. "Estimation of Regression Coefficients of Interest When Other Regression Coefficients Are of No Interest," Econometrica, Econometric Society, vol. 67(3), pages 639-644, May.
- Pötscher, B.M., 1991. "Effects of Model Selection on Inference," Econometric Theory, Cambridge University Press, vol. 7(02), pages 163-185, June.
- Diebold, Francis X & Mariano, Roberto S, 2002.
"Comparing Predictive Accuracy,"
Journal of Business & Economic Statistics,
American Statistical Association, vol. 20(1), pages 134-44, January.
- Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
- Zaman, A., 1984. "Avoiding model selection by the use of shrinkage techniques," Journal of Econometrics, Elsevier, vol. 25(1-2), pages 73-85.
- Thomson, Michael & Schmidt, Peter, 1982. "A Note on the Comparison of the Mean Square Error of Inequality Constrained Least Squares and Other Related Estimators," The Review of Economics and Statistics, MIT Press, vol. 64(1), pages 174-76, February.
- repec:cup:etheor:v:7:y:1991:i:2:p:163-85 is not listed on IDEAS
- Chotikapanich, D. & Griffiths, W.E. & Rao, D.S.P., 2001.
"Averaging Income Distributions,"
Department of Economics - Working Papers Series
798, The University of Melbourne.
- Chua, C.L. & Griffiths, W.E. & O'Donnell, C.J., 2001.
"Bayesian Model Averaging in Consumer Demand Systems with Inequality Constraints,"
Department of Economics - Working Papers Series
806, The University of Melbourne.
- C. L Chua & W. E. Griffiths & C. J O'Donnell, 2001. "Bayesian Model Averaging in Consumer Demand Systems with Inequality Constraints," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 49(3), pages 269-291, November.
- Danilov, D.L. & Magnus, J.R., 2002.
"Forecast Accuracy after Pretesting with an Application to the Stock Market,"
2002-76, Tilburg University, Center for Economic Research.
- Jan R. Magnus & Dmitry Danilov, 2004. "Forecast accuracy after pretesting with an application to the stock market," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(4), pages 251-274.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Richard Broekman).
If references are entirely missing, you can add them using this form.