IDEAS home Printed from https://ideas.repec.org/a/jof/jforec/v23y2004i4p251-274.html
   My bibliography  Save this article

Forecast accuracy after pretesting with an application to the stock market

Author

Listed:
  • Jan R. Magnus

    (CentER, Tilburg University, The Netherlands)

  • Dmitry Danilov

    (Eurandom, Eindhoven University of Technology, The Netherlands)

Abstract

In econometrics, as a rule, the same data set is used to select the model and, conditional on the selected model, to forecast. However, one typically reports the properties of the (conditional) forecast, ignoring the fact that its properties are affected by the model selection (pretesting). This is wrong, and in this paper we show that the error can be substantial. We obtain explicit expressions for this error. To illustrate the theory we consider a regression approach to stock market forecasting, and show that the standard predictions ignoring pretesting are much less robust than naive econometrics might suggest. We also propose a forecast procedure based on the 'neutral Laplace estimator', which leads to an improvement over standard model selection procedures. Copyright © 2004 John Wiley & Sons, Ltd.

Suggested Citation

  • Jan R. Magnus & Dmitry Danilov, 2004. "Forecast accuracy after pretesting with an application to the stock market," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(4), pages 251-274.
  • Handle: RePEc:jof:jforec:v:23:y:2004:i:4:p:251-274
    DOI: 10.1002/for.916
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1002/for.916
    File Function: Link to full text; subscription required
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Fama, Eugene F & French, Kenneth R, 1992. " The Cross-Section of Expected Stock Returns," Journal of Finance, American Finance Association, vol. 47(2), pages 427-465, June.
    2. Karim M. Abadir & Jan R. Magnus, 2002. "Notation in econometrics: a proposal for a standard," Econometrics Journal, Royal Economic Society, vol. 5(1), pages 76-90, June.
    3. Paye, Bradley S. & Timmermann, Allan, 2002. "How Stable are Financial Prediction Models? Evidence from US and International Stock Market Data," University of California at San Diego, Economics Working Paper Series qt74v515fr, Department of Economics, UC San Diego.
    4. Jan R. Magnus & J. Durbin, 1999. "Estimation of Regression Coefficients of Interest When Other Regression Coefficients Are of No Interest," Econometrica, Econometric Society, vol. 67(3), pages 639-644, May.
    5. Ryan Sullivan & Allan Timmermann & Halbert White, 1999. "Data-Snooping, Technical Trading Rule Performance, and the Bootstrap," Journal of Finance, American Finance Association, vol. 54(5), pages 1647-1691, October.
    6. Fama, Eugene F. & French, Kenneth R., 1989. "Business conditions and expected returns on stocks and bonds," Journal of Financial Economics, Elsevier, vol. 25(1), pages 23-49, November.
    7. Danilov, D.L. & Magnus, J.R., 2002. "Estimation of the Mean of a Univariate Normal Distribution When the Variance is not Known," Discussion Paper 2002-77, Tilburg University, Center for Economic Research.
    8. Lo, Andrew W & MacKinlay, A Craig, 1990. "Data-Snooping Biases in Tests of Financial Asset Pricing Models," Review of Financial Studies, Society for Financial Studies, vol. 3(3), pages 431-467.
    9. Pesaran, M Hashem & Timmermann, Allan, 1995. " Predictability of Stock Returns: Robustness and Economic Significance," Journal of Finance, American Finance Association, vol. 50(4), pages 1201-1228, September.
    10. French, Kenneth R. & Schwert, G. William & Stambaugh, Robert F., 1987. "Expected stock returns and volatility," Journal of Financial Economics, Elsevier, vol. 19(1), pages 3-29, September.
    11. Fama, Eugene F & MacBeth, James D, 1973. "Risk, Return, and Equilibrium: Empirical Tests," Journal of Political Economy, University of Chicago Press, vol. 81(3), pages 607-636, May-June.
    12. Jan R. Magnus, 2002. "Estimation of the mean of a univariate normal distribution with known variance," Econometrics Journal, Royal Economic Society, vol. 5(1), pages 225-236, June.
    13. Foster, F Douglas & Smith, Tom & Whaley, Robert E, 1997. " Assessing Goodness-of-Fit of Asset Pricing Models: The Distribution of the Maximal R-Squared," Journal of Finance, American Finance Association, vol. 52(2), pages 591-607, June.
    14. Chen, Nai-Fu & Roll, Richard & Ross, Stephen A, 1986. "Economic Forces and the Stock Market," The Journal of Business, University of Chicago Press, vol. 59(3), pages 383-403, July.
    15. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
    16. Danilov, D.L. & Magnus, J.R., 2001. "On the Harm that Pretesting Does," Discussion Paper 2001-37, Tilburg University, Center for Economic Research.
    17. Balvers, Ronald J & Cosimano, Thomas F & McDonald, Bill, 1990. " Predicting Stock Returns in an Efficient Market," Journal of Finance, American Finance Association, vol. 45(4), pages 1109-1128, September.
    18. Danilov, Dmitry & Magnus, J.R.Jan R., 2004. "On the harm that ignoring pretesting can cause," Journal of Econometrics, Elsevier, vol. 122(1), pages 27-46, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Magnus, Jan & Peresetsky, Anatoly, 2010. "The price of Moscow apartments," Applied Econometrics, Publishing House "SINERGIA PRESS", vol. 17(1), pages 89-105.
    2. Mouchart, Michel & Rombouts, Jeroen V.K., 2005. "Clustered panel data models: an efficient approach for nowcasting from poor data," International Journal of Forecasting, Elsevier, vol. 21(3), pages 577-594.
    3. Joshua Gallin & Randal Verbrugge, 2007. "Improving the CPI’s Age-Bias Adjustment: Leverage, Disaggregation and Model Averaging," Working Papers 411, U.S. Bureau of Labor Statistics.
    4. Ouysse, Rachida, 2006. "Consistent variable selection in large panels when factors are observable," Journal of Multivariate Analysis, Elsevier, vol. 97(4), pages 946-984, April.
    5. Magnus, Jan R. & Wan, Alan T.K. & Zhang, Xinyu, 2011. "Weighted average least squares estimation with nonspherical disturbances and an application to the Hong Kong housing market," Computational Statistics & Data Analysis, Elsevier, vol. 55(3), pages 1331-1341, March.

    More about this item

    JEL classification:

    • C20 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - General
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • G10 - Financial Economics - - General Financial Markets - - - General (includes Measurement and Data)

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jof:jforec:v:23:y:2004:i:4:p:251-274. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Wiley-Blackwell Digital Licensing) or (Christopher F. Baum). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/2966 .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.