The illusion of predictability: How regression statistics mislead experts
AbstractDoes the manner in which results are presented in empirical studies affect perceptions of the predictability of the outcomes? Noting the predominant role of linear regression analysis in empirical economics, we asked 257 academic economists to make probabilistic inferences based on different presentations of the outputs of this statistical tool. The questions concerned the distribution of the dependent variable, conditional on known values of the independent variable. The answers based on the presentation mode that is standard in the literature demonstrated an illusion of predictability; the outcomes were perceived to be more predictable than could be justified by the model. In particular, many respondents failed to take the error term into account. Adding graphs did not improve the inference. Paradoxically, the respondents were more accurate when only graphs were provided (i.e., no regression statistics). The implications of our study suggest, inter alia, the need to reconsider the way in which empirical results are presented, and the possible provision of easy-to-use simulation tools that would enable readers of empirical papers to make accurate inferences.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.
Bibliographic InfoArticle provided by Elsevier in its journal International Journal of Forecasting.
Volume (Year): 28 (2012)
Issue (Month): 3 ()
Contact details of provider:
Web page: http://www.elsevier.com/locate/ijforecast
Regression; Presentation formats; Probabilistic inference; Prediction; Graphics; Uncertainty;
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Camerer, Colin F. & Hogarth, Robin M., 1999.
"The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework,"
1059, California Institute of Technology, Division of the Humanities and Social Sciences.
- Camerer, Colin F & Hogarth, Robin M, 1999. "The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework," Journal of Risk and Uncertainty, Springer, vol. 19(1-3), pages 7-42, December.
- Kahneman, Daniel & Tversky, Amos, 1979.
"Prospect Theory: An Analysis of Decision under Risk,"
Econometric Society, vol. 47(2), pages 263-91, March.
- Amos Tversky & Daniel Kahneman, 1979. "Prospect Theory: An Analysis of Decision under Risk," Levine's Working Paper Archive 7656, David K. Levine.
- Stephen T. Ziliak & Deirdre N. McCloskey, 2004. "Size Matters: The Standard Error of Regressions in the American Economic Review," Econ Journal Watch, Econ Journal Watch, vol. 1(2), pages 331-358, August.
- Lawrence, Michael & Makridakis, Spyros, 1989. "Factors affecting judgmental forecasts and confidence intervals," Organizational Behavior and Human Decision Processes, Elsevier, vol. 43(2), pages 172-187, April.
- David F. Hendry & Bent Nielsen, 2007.
"Preface to Econometric Modeling: A Likelihood Approach
[Econometric Modeling: A Likelihood Approach]," Introductory Chapters, Princeton University Press.
- Armstrong, J. Scott, 2007. "Significance tests harm progress in forecasting," International Journal of Forecasting, Elsevier, vol. 23(2), pages 321-327.
- Carhart, Mark M, 1997. " On Persistence in Mutual Fund Performance," Journal of Finance, American Finance Association, vol. 52(1), pages 57-82, March.
- Ziliak, Stephen T. & McCloskey, Deirdre N., 2004. "Size matters: the standard error of regressions in the American Economic Review," The Journal of Socio-Economics, Elsevier, vol. 33(5), pages 527-546, November.
- Zellner, Arnold, 2004. "To test or not to test and if so, how?: Comments on "size matters"," The Journal of Socio-Economics, Elsevier, vol. 33(5), pages 581-586, November.
- Baltagi, Badi H., 2007.
"Worldwide Econometrics Rankings: 1989 2005,"
Cambridge University Press, vol. 23(05), pages 952-1012, October.
- Deirdre N. McCloskey & Stephen T. Ziliak, 1996. "The Standard Error of Regressions," Journal of Economic Literature, American Economic Association, vol. 34(1), pages 97-114, March.
- Michael C. Jensen, 1968. "The Performance Of Mutual Funds In The Period 1945–1964," Journal of Finance, American Finance Association, vol. 23(2), pages 389-416, 05.
- Simon, Herbert A, 1978. "Rationality as Process and as Product of Thought," American Economic Review, American Economic Association, vol. 68(2), pages 1-16, May.
- Arch G. Woodside & Man-Ling Chang & Cheng-Feng Cheng, 2012. "Government Regulations of Business, Corruption, Reforms, and the Economic Growth of Nations," International Journal of Business and Economics, College of Business, and College of Finance, Feng Chia University, Taichung, Taiwan, vol. 11(2), pages 127-142, December.
- Deirdre N. McCloskey & Stephen T. Ziliak, 2012. "Statistical Significance in the New Tom and the Old Tom: A Reply to Thomas Mayer," Econ Journal Watch, Econ Journal Watch, vol. 9(3), pages 298-308, September.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Wendy Shamier).
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If references are entirely missing, you can add them using this form.
If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.
Please note that corrections may take a couple of weeks to filter through the various RePEc services.