On the Harm that Pretesting Does
AbstractData in econometrics are, as a rule, non-experimental and hence we have to use the same data set to select the model and also to estimate the parameters in the selected model.In standard applied econometrics practice, however, one reports zero bias and some variance of the (pretest) estimators conditional on the selected model.In this paper we find the unconditional moments of the pretest estimator, taking full account of the fact that model selection and estimation are an integrated procedure.We derive the bias, variance, and mean squared error of the pretest estimator, and show what the error is in not reporting the correct moments.This error can be very substantial.We also show that there can be large differences in underreporting between different model selection procedures.Finally, we ask how the underreporting error increases when the number of auxiliary regressors increases.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by Tilburg University, Center for Economic Research in its series Discussion Paper with number 2001-37.
Date of creation: 2001
Date of revision:
Contact details of provider:
Web page: http://center.uvt.nl
econometric models; testing; estimation;
This paper has been announced in the following NEP Reports:
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Roehrig, C.S., 1984. "Optimal critical regions for pre-test estimators using a Bayes risk criterion," Journal of Econometrics, Elsevier, vol. 25(1-2), pages 3-14.
- Judge, G.G. & Bock, M.E., 1983. "Biased estimation," Handbook of Econometrics, in: Z. Griliches† & M. D. Intriligator (ed.), Handbook of Econometrics, edition 1, volume 1, chapter 10, pages 599-649 Elsevier.
- West, Kenneth D, 1996.
"Asymptotic Inference about Predictive Ability,"
Econometric Society, vol. 64(5), pages 1067-84, September.
- Lovell, Michael C, 1983. "Data Mining," The Review of Economics and Statistics, MIT Press, vol. 65(1), pages 1-12, February.
- Pötscher, B.M., 1991. "Effects of Model Selection on Inference," Econometric Theory, Cambridge University Press, vol. 7(02), pages 163-185, June.
- Hannes Leeb & Benedikt M. Poetscher, 2000.
"The Finite-Sample Distribution of Post-Model-Selection Estimators, and Uniform Versus Non-Uniform Approximations,"
- Leeb, Hannes & P tscher, Benedikt M., 2003. "The Finite-Sample Distribution Of Post-Model-Selection Estimators And Uniform Versus Nonuniform Approximations," Econometric Theory, Cambridge University Press, vol. 19(01), pages 100-142, February.
- Kevin Hoover & Stephen J. Perez, 2003.
"Data Mining Reconsidered: Encompassing And The General-To-Specific Approach To Specification Search,"
9727, University of California, Davis, Department of Economics.
- Kevin D. Hoover & Stephen J. Perez, 1999. "Data mining reconsidered: encompassing and the general-to-specific approach to specification search," Econometrics Journal, Royal Economic Society, vol. 2(2), pages 167-191.
- Kevin D. Hoover & Stephen J. Perez, . "Data Mining Reconsidered: Encompassing And The General-To-Specific Approach To Specification Search," Department of Economics 97-27, California Davis - Department of Economics.
- Francis X. Diebold & Robert S. Mariano, 1994.
"Comparing Predictive Accuracy,"
NBER Technical Working Papers
0169, National Bureau of Economic Research, Inc.
- Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-44, January.
- Diebold, Francis X & Mariano, Roberto S, 1995. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(3), pages 253-63, July.
- Hendry, David F., 2001. "Achievements and challenges in econometric methodology," Journal of Econometrics, Elsevier, vol. 100(1), pages 7-10, January.
- Giles, Judith A & Giles, David E A, 1993. " Pre-test Estimation and Testing in Econometrics: Recent Developments," Journal of Economic Surveys, Wiley Blackwell, vol. 7(2), pages 145-97, June.
- repec:cup:etheor:v:7:y:1991:i:2:p:163-85 is not listed on IDEAS
- Lo, Andrew W. (Andrew Wen-Chuan) & MacKinlay, Archie Craig, 1955-, 1989.
"Data-snooping biases in tests of financial asset pricing models,"
3020-89., Massachusetts Institute of Technology (MIT), Sloan School of Management.
- Lo, Andrew W & MacKinlay, A Craig, 1990. "Data-Snooping Biases in Tests of Financial Asset Pricing Models," Review of Financial Studies, Society for Financial Studies, vol. 3(3), pages 431-67.
- Andrew W. Lo & A. Craig MacKinlay, 1991. "Data-Snooping Biases in Tests of Financial Asset Pricing Models," NBER Working Papers 3001, National Bureau of Economic Research, Inc.
- Thomson, Michael & Schmidt, Peter, 1982. "A Note on the Comparison of the Mean Square Error of Inequality Constrained Least Squares and Other Related Estimators," The Review of Economics and Statistics, MIT Press, vol. 64(1), pages 174-76, February.
- Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
- Feldstein, Martin S, 1973. "Multicollinearity and the Mean Square Error of Alternative Estimators," Econometrica, Econometric Society, vol. 41(2), pages 337-46, March.
- Jan R. Magnus & J. Durbin, 1999. "Estimation of Regression Coefficients of Interest When Other Regression Coefficients Are of No Interest," Econometrica, Econometric Society, vol. 67(3), pages 639-644, May.
- Zaman, A., 1984. "Avoiding model selection by the use of shrinkage techniques," Journal of Econometrics, Elsevier, vol. 25(1-2), pages 73-85.
- Mittelhammer, R.C., 1984. "Restricted least squares, pre-test, ols and stein rule estimators: Risk comparisons under model misspecification," Journal of Econometrics, Elsevier, vol. 25(1-2), pages 151-164.
- Jan R. Magnus & Dmitry Danilov, 2004.
"Forecast accuracy after pretesting with an application to the stock market,"
Journal of Forecasting,
John Wiley & Sons, Ltd., vol. 23(4), pages 251-274.
- Danilov, D.L. & Magnus, J.R., 2002. "Forecast Accuracy after Pretesting with an Application to the Stock Market," Discussion Paper 2002-76, Tilburg University, Center for Economic Research.
- William E. Griffiths & Duangkamon Chotikapanich & D. S. Prasada Rao, 2005.
"Averaging Income Distributions,"
Bulletin of Economic Research,
Wiley Blackwell, vol. 57(4), pages 347-367, October.
- Chua, C.L. & Griffiths, W.E. & O'Donnell, C.J., 2001.
"Bayesian Model Averaging in Consumer Demand Systems with Inequality Constraints,"
Department of Economics - Working Papers Series
806, The University of Melbourne.
- C. L Chua & W. E. Griffiths & C. J O'Donnell, 2001. "Bayesian Model Averaging in Consumer Demand Systems with Inequality Constraints," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 49(3), pages 269-291, November.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Richard Broekman).
If references are entirely missing, you can add them using this form.