The illusion of predictability: How regression statistics mislead experts
AbstractDoes the manner in which results are presented in empirical studies affect perceptions of the predictability of the outcomes? Noting the predominant role of linear regression analysis in empirical economics, we asked 257 academic economists to make probabilistic inferences based on different presentations of the outputs of this statistical tool. The questions concerned the distribution of the dependent variable, conditional on known values of the independent variable. The answers based on the presentation mode that is standard in the literature demonstrated an illusion of predictability; the outcomes were perceived to be more predictable than could be justified by the model. In particular, many respondents failed to take the error term into account. Adding graphs did not improve the inference. Paradoxically, the respondents were more accurate when only graphs were provided (i.e., no regression statistics). The implications of our study suggest, inter alia, the need to reconsider the way in which empirical results are presented, and the possible provision of easy-to-use simulation tools that would enable readers of empirical papers to make accurate inferences.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoArticle provided by Elsevier in its journal International Journal of Forecasting.
Volume (Year): 28 (2012)
Issue (Month): 3 ()
Contact details of provider:
Web page: http://www.elsevier.com/locate/ijforecast
Regression; Presentation formats; Probabilistic inference; Prediction; Graphics; Uncertainty;
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Simon, Herbert A, 1978. "Rationality as Process and as Product of Thought," American Economic Review, American Economic Association, vol. 68(2), pages 1-16, May.
- Stephen T. Ziliak & Deirdre N. McCloskey, 2004. "Size Matters: The Standard Error of Regressions in the American Economic Review," Econ Journal Watch, Econ Journal Watch, vol. 1(2), pages 331-358, August.
- Kahneman, Daniel & Tversky, Amos, 1979.
"Prospect Theory: An Analysis of Decision under Risk,"
Econometric Society, vol. 47(2), pages 263-91, March.
- Amos Tversky & Daniel Kahneman, 1979. "Prospect Theory: An Analysis of Decision under Risk," Levine's Working Paper Archive 7656, David K. Levine.
- Carhart, Mark M, 1997. " On Persistence in Mutual Fund Performance," Journal of Finance, American Finance Association, vol. 52(1), pages 57-82, March.
- Lawrence, Michael & Makridakis, Spyros, 1989. "Factors affecting judgmental forecasts and confidence intervals," Organizational Behavior and Human Decision Processes, Elsevier, vol. 43(2), pages 172-187, April.
- Camerer, Colin F. & Hogarth, Robin M., 1999.
"The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework,"
1059, California Institute of Technology, Division of the Humanities and Social Sciences.
- Camerer, Colin F & Hogarth, Robin M, 1999. "The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework," Journal of Risk and Uncertainty, Springer, vol. 19(1-3), pages 7-42, December.
- Badi H. Baltagi, 2007.
"Worldwide Econometrics Rankings: 1989-2005,"
Center for Policy Research Working Papers
94, Center for Policy Research, Maxwell School, Syracuse University.
- Zellner, Arnold, 2004. "To test or not to test and if so, how?: Comments on "size matters"," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 33(5), pages 581-586, November.
- Ziliak, Stephen T. & McCloskey, Deirdre N., 2004. "Size matters: the standard error of regressions in the American Economic Review," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 33(5), pages 527-546, November.
- David F. Hendry & Bent Nielsen, 2007.
"Preface to Econometric Modeling: A Likelihood Approach
[Econometric Modeling: A Likelihood Approach]," Introductory Chapters, Princeton University Press.
- Deirdre N. McCloskey & Stephen T. Ziliak, 1996. "The Standard Error of Regressions," Journal of Economic Literature, American Economic Association, vol. 34(1), pages 97-114, March.
- Armstrong, J. Scott, 2007. "Significance tests harm progress in forecasting," International Journal of Forecasting, Elsevier, vol. 23(2), pages 321-327.
- Michael C. Jensen, 1968. "The Performance Of Mutual Funds In The Period 1945–1964," Journal of Finance, American Finance Association, vol. 23(2), pages 389-416, 05.
- Deirdre N. McCloskey & Stephen T. Ziliak, 2012. "Statistical Significance in the New Tom and the Old Tom: A Reply to Thomas Mayer," Econ Journal Watch, Econ Journal Watch, vol. 9(3), pages 298-308, September.
- Armstrong, J. Scott & Green, Kesten C. & Graefe, Andreas, 2014. "Golden Rule of Forecasting: Be conservative," MPRA Paper 53579, University Library of Munich, Germany.
- Arch G. Woodside & Man-Ling Chang & Cheng-Feng Cheng, 2012. "Government Regulations of Business, Corruption, Reforms, and the Economic Growth of Nations," International Journal of Business and Economics, College of Business, and College of Finance, Feng Chia University, Taichung, Taiwan, vol. 11(2), pages 127-142, December.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei).
If references are entirely missing, you can add them using this form.