The illusion of predictability: How regression statistics mislead experts
Does the manner in which results are presented in empirical studies affect perceptions of the predictability of the outcomes? Noting the predominant role of linear regression analysis in empirical economics, we asked 257 academic economists to make probabilistic inferences based on different presentations of the outputs of this statistical tool. The questions concerned the distribution of the dependent variable, conditional on known values of the independent variable. The answers based on the presentation mode that is standard in the literature demonstrated an illusion of predictability; the outcomes were perceived to be more predictable than could be justified by the model. In particular, many respondents failed to take the error term into account. Adding graphs did not improve the inference. Paradoxically, the respondents were more accurate when only graphs were provided (i.e., no regression statistics). The implications of our study suggest, inter alia, the need to reconsider the way in which empirical results are presented, and the possible provision of easy-to-use simulation tools that would enable readers of empirical papers to make accurate inferences.
Volume (Year): 28 (2012)
Issue (Month): 3 ()
|Contact details of provider:|| Web page: http://www.elsevier.com/locate/ijforecast|
References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Stephen T. Ziliak & Deirdre N. McCloskey, 2004.
"Size Matters: The Standard Error of Regressions in the American Economic Review,"
Econ Journal Watch,
Econ Journal Watch, vol. 1(2), pages 331-358, August.
- Ziliak, Stephen T. & McCloskey, Deirdre N., 2004. "Size matters: the standard error of regressions in the American Economic Review," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 33(5), pages 527-546, November.
- Deirdre N. McCloskey & Stephen T. Ziliak, 1996. "The Standard Error of Regressions," Journal of Economic Literature, American Economic Association, vol. 34(1), pages 97-114, March.
- Michael C. Jensen, 1968. "The Performance Of Mutual Funds In The Period 1945–1964," Journal of Finance, American Finance Association, vol. 23(2), pages 389-416, 05.
- Camerer, Colin F & Hogarth, Robin M, 1999. "The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework," Journal of Risk and Uncertainty, Springer, vol. 19(1-3), pages 7-42, December.
- Camerer, Colin F. & Hogarth, Robin M., 1999. "The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework," Working Papers 1059, California Institute of Technology, Division of the Humanities and Social Sciences.
- Lawrence, Michael & Makridakis, Spyros, 1989. "Factors affecting judgmental forecasts and confidence intervals," Organizational Behavior and Human Decision Processes, Elsevier, vol. 43(2), pages 172-187, April.
- Kahneman, Daniel & Tversky, Amos, 1979. "Prospect Theory: An Analysis of Decision under Risk," Econometrica, Econometric Society, vol. 47(2), pages 263-291, March.
- Amos Tversky & Daniel Kahneman, 1979. "Prospect Theory: An Analysis of Decision under Risk," Levine's Working Paper Archive 7656, David K. Levine.
- Simon, Herbert A, 1978. "Rationality as Process and as Product of Thought," American Economic Review, American Economic Association, vol. 68(2), pages 1-16, May.
- Baltagi, Badi H., 2007. "Worldwide Econometrics Rankings: 1989 2005," Econometric Theory, Cambridge University Press, vol. 23(05), pages 952-1012, October.
- Badi H. Baltagi, 2007. "Worldwide Econometrics Rankings: 1989-2005," Center for Policy Research Working Papers 94, Center for Policy Research, Maxwell School, Syracuse University.
- David F. Hendry & Bent Nielsen, 2007. "Preface to Econometric Modeling: A Likelihood Approach," Introductory Chapters,in: Econometric Modeling: A Likelihood Approach Princeton University Press.
- Zellner, Arnold, 2004. "To test or not to test and if so, how?: Comments on "size matters"," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 33(5), pages 581-586, November.
- Armstrong, J. Scott, 2007. "Significance tests harm progress in forecasting," International Journal of Forecasting, Elsevier, vol. 23(2), pages 321-327.
- Carhart, Mark M, 1997. " On Persistence in Mutual Fund Performance," Journal of Finance, American Finance Association, vol. 52(1), pages 57-82, March. Full references (including those not matched with items on IDEAS)