Advanced Search
MyIDEAS: Login

On the Harm that Pretesting Does

Contents:

Author Info

  • Danilov, D.L.
  • Magnus, J.R.

    (Tilburg University, Center for Economic Research)

Abstract

Data in econometrics are, as a rule, non-experimental and hence we have to use the same data set to select the model and also to estimate the parameters in the selected model.In standard applied econometrics practice, however, one reports zero bias and some variance of the (pretest) estimators conditional on the selected model.In this paper we find the unconditional moments of the pretest estimator, taking full account of the fact that model selection and estimation are an integrated procedure.We derive the bias, variance, and mean squared error of the pretest estimator, and show what the error is in not reporting the correct moments.This error can be very substantial.We also show that there can be large differences in underreporting between different model selection procedures.Finally, we ask how the underreporting error increases when the number of auxiliary regressors increases.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://arno.uvt.nl/show.cgi?fid=4229
Download Restriction: no

Bibliographic Info

Paper provided by Tilburg University, Center for Economic Research in its series Discussion Paper with number 2001-37.

as in new window
Length:
Date of creation: 2001
Date of revision:
Handle: RePEc:dgr:kubcen:200137

Contact details of provider:
Web page: http://center.uvt.nl

Related research

Keywords: econometric models; testing; estimation;

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Lo, Andrew W & MacKinlay, A Craig, 1990. "Data-Snooping Biases in Tests of Financial Asset Pricing Models," Review of Financial Studies, Society for Financial Studies, vol. 3(3), pages 431-67.
  2. Hendry, David F., 2001. "Achievements and challenges in econometric methodology," Journal of Econometrics, Elsevier, vol. 100(1), pages 7-10, January.
  3. Mittelhammer, R.C., 1984. "Restricted least squares, pre-test, ols and stein rule estimators: Risk comparisons under model misspecification," Journal of Econometrics, Elsevier, vol. 25(1-2), pages 151-164.
  4. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
  5. Lovell, Michael C, 1983. "Data Mining," The Review of Economics and Statistics, MIT Press, vol. 65(1), pages 1-12, February.
  6. Feldstein, Martin S, 1973. "Multicollinearity and the Mean Square Error of Alternative Estimators," Econometrica, Econometric Society, vol. 41(2), pages 337-46, March.
  7. Pötscher, B.M., 1991. "Effects of Model Selection on Inference," Econometric Theory, Cambridge University Press, vol. 7(02), pages 163-185, June.
  8. Giles, Judith A & Giles, David E A, 1993. " Pre-test Estimation and Testing in Econometrics: Recent Developments," Journal of Economic Surveys, Wiley Blackwell, vol. 7(2), pages 145-97, June.
  9. Judge, G.G. & Bock, M.E., 1983. "Biased estimation," Handbook of Econometrics, in: Z. Griliches† & M. D. Intriligator (ed.), Handbook of Econometrics, edition 1, volume 1, chapter 10, pages 599-649 Elsevier.
  10. Zaman, A., 1984. "Avoiding model selection by the use of shrinkage techniques," Journal of Econometrics, Elsevier, vol. 25(1-2), pages 73-85.
  11. Leeb, Hannes & P tscher, Benedikt M., 2003. "The Finite-Sample Distribution Of Post-Model-Selection Estimators And Uniform Versus Nonuniform Approximations," Econometric Theory, Cambridge University Press, vol. 19(01), pages 100-142, February.
  12. Kenneth D. West, 1994. "Asymptotic Inference About Predictive Ability," Macroeconomics 9410002, EconWPA.
  13. Kevin D. Hoover & Stephen J. Perez, 1999. "Data mining reconsidered: encompassing and the general-to-specific approach to specification search," Econometrics Journal, Royal Economic Society, vol. 2(2), pages 167-191.
  14. repec:cup:etheor:v:7:y:1991:i:2:p:163-85 is not listed on IDEAS
  15. Thomson, Michael & Schmidt, Peter, 1982. "A Note on the Comparison of the Mean Square Error of Inequality Constrained Least Squares and Other Related Estimators," The Review of Economics and Statistics, MIT Press, vol. 64(1), pages 174-76, February.
  16. Francis X. Diebold & Robert S. Mariano, 1994. "Comparing Predictive Accuracy," NBER Technical Working Papers 0169, National Bureau of Economic Research, Inc.
  17. Jan R. Magnus & J. Durbin, 1999. "Estimation of Regression Coefficients of Interest When Other Regression Coefficients Are of No Interest," Econometrica, Econometric Society, vol. 67(3), pages 639-644, May.
  18. Roehrig, C.S., 1984. "Optimal critical regions for pre-test estimators using a Bayes risk criterion," Journal of Econometrics, Elsevier, vol. 25(1-2), pages 3-14.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. William E. Griffiths & Duangkamon Chotikapanich & D. S. Prasada Rao, 2005. "Averaging Income Distributions," Bulletin of Economic Research, Wiley Blackwell, vol. 57(4), pages 347-367, October.
  2. Danilov, D.L. & Magnus, J.R., 2002. "Forecast Accuracy after Pretesting with an Application to the Stock Market," Discussion Paper 2002-76, Tilburg University, Center for Economic Research.
  3. C. L Chua & W. E. Griffiths & C. J O'Donnell, 2001. "Bayesian Model Averaging in Consumer Demand Systems with Inequality Constraints," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 49(3), pages 269-291, November.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:dgr:kubcen:200137. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Richard Broekman).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.