IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v28y2012i3p695-711.html
   My bibliography  Save this article

The illusion of predictability: How regression statistics mislead experts

Author

Listed:
  • Soyer, Emre
  • Hogarth, Robin M.

Abstract

Does the manner in which results are presented in empirical studies affect perceptions of the predictability of the outcomes? Noting the predominant role of linear regression analysis in empirical economics, we asked 257 academic economists to make probabilistic inferences based on different presentations of the outputs of this statistical tool. The questions concerned the distribution of the dependent variable, conditional on known values of the independent variable. The answers based on the presentation mode that is standard in the literature demonstrated an illusion of predictability; the outcomes were perceived to be more predictable than could be justified by the model. In particular, many respondents failed to take the error term into account. Adding graphs did not improve the inference. Paradoxically, the respondents were more accurate when only graphs were provided (i.e., no regression statistics). The implications of our study suggest, inter alia, the need to reconsider the way in which empirical results are presented, and the possible provision of easy-to-use simulation tools that would enable readers of empirical papers to make accurate inferences.

Suggested Citation

  • Soyer, Emre & Hogarth, Robin M., 2012. "The illusion of predictability: How regression statistics mislead experts," International Journal of Forecasting, Elsevier, vol. 28(3), pages 695-711.
    • Handle: RePEc:eee:intfor:v:28:y:2012:i:3:p:695-711
    DOI: 10.1016/j.ijforecast.2012.02.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207012000258
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Deirdre N. McCloskey & Stephen T. Ziliak, 1996. "The Standard Error of Regressions," Journal of Economic Literature, American Economic Association, vol. 34(1), pages 97-114, March.
    2. Stephen T. Ziliak & Deirdre N. McCloskey, 2004. "Size Matters: The Standard Error of Regressions in the American Economic Review," Econ Journal Watch, Econ Journal Watch, vol. 1(2), pages 331-358, August.
    3. Armstrong, J. Scott, 2007. "Significance Tests Harm Progress in Forecasting," MPRA Paper 81664, University Library of Munich, Germany.
    4. Michael C. Jensen, 1968. "The Performance Of Mutual Funds In The Period 1945–1964," Journal of Finance, American Finance Association, vol. 23(2), pages 389-416, May.
    5. Camerer, Colin F & Hogarth, Robin M, 1999. "The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework," Journal of Risk and Uncertainty, Springer, vol. 19(1-3), pages 7-42, December.
    6. Lawrence, Michael & Makridakis, Spyros, 1989. "Factors affecting judgmental forecasts and confidence intervals," Organizational Behavior and Human Decision Processes, Elsevier, vol. 43(2), pages 172-187, April.
    7. Kahneman, Daniel & Tversky, Amos, 1979. "Prospect Theory: An Analysis of Decision under Risk," Econometrica, Econometric Society, vol. 47(2), pages 263-291, March.
    8. Simon, Herbert A, 1978. "Rationality as Process and as Product of Thought," American Economic Review, American Economic Association, vol. 68(2), pages 1-16, May.
    9. Baltagi, Badi H., 2007. "Worldwide Econometrics Rankings: 1989 2005," Econometric Theory, Cambridge University Press, vol. 23(05), pages 952-1012, October.
    10. David F. Hendry & Bent Nielsen, 2007. "Preface to Econometric Modeling: A Likelihood Approach," Introductory Chapters,in: Econometric Modeling: A Likelihood Approach Princeton University Press.
    11. Zellner, Arnold, 2004. "To test or not to test and if so, how?: Comments on "size matters"," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 33(5), pages 581-586, November.
    12. Armstrong, J. Scott, 2007. "Significance tests harm progress in forecasting," International Journal of Forecasting, Elsevier, vol. 23(2), pages 321-327.
    13. Carhart, Mark M, 1997. " On Persistence in Mutual Fund Performance," Journal of Finance, American Finance Association, vol. 52(1), pages 57-82, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hogarth, Robin M. & Soyer, Emre, 2015. "Communicating forecasts: The simplicity of simulated experience," Journal of Business Research, Elsevier, vol. 68(8), pages 1800-1809.
    2. Kim, Jae H. & Ji, Philip Inyeob, 2015. "Significance testing in empirical finance: A critical review and assessment," Journal of Empirical Finance, Elsevier, vol. 34(C), pages 1-14.
    3. Armstrong, J. Scott & Green, Kesten C. & Graefe, Andreas, 2015. "Golden rule of forecasting: Be conservative," Journal of Business Research, Elsevier, vol. 68(8), pages 1717-1731.
    4. Green, Kesten C. & Armstrong, J. Scott, 2015. "Simple versus complex forecasting: The evidence," Journal of Business Research, Elsevier, vol. 68(8), pages 1678-1685.
    5. Steffen Roth & Jari Kaivo-Oja, 2016. "Is the future a political economy? Functional analysis of three leading foresight and futures studies journals," Post-Print hal-01475083, HAL.
    6. Soyer, Emre & Hogarth, Robin M., 2015. "The golden rule of forecasting: Objections, refinements, and enhancements," Journal of Business Research, Elsevier, vol. 68(8), pages 1702-1704.
    7. Kim, Jae, 2015. "How to Choose the Level of Significance: A Pedagogical Note," MPRA Paper 66373, University Library of Munich, Germany.
    8. Arch G. Woodside & Man-Ling Chang & Cheng-Feng Cheng, 2012. "Government Regulations of Business, Corruption, Reforms, and the Economic Growth of Nations," International Journal of Business and Economics, College of Business and College of Finance, Feng Chia University, Taichung, Taiwan, vol. 11(2), pages 127-142, December.
    9. Ang, Huat Bin (Andy) & Woodside, Arch G., 2017. "Is Bart Simpson offering sage advice? A case-based general theory of managers' core self-evaluations and job satisfaction," Journal of Business Research, Elsevier, vol. 74(C), pages 11-37.
    10. Ren, Shengce & Tsai, Huei-Ting & Eisingerich, Andreas B., 2016. "Case-based asymmetric modeling of firms with high versus low outcomes in implementing changes in direction," Journal of Business Research, Elsevier, vol. 69(2), pages 500-507.
    11. Green, Kesten C. & Armstrong, J. Scott & Graefe, Andreas, 2015. "Golden rule of forecasting rearticulated: Forecast unto others as you would have them forecast unto you," Journal of Business Research, Elsevier, vol. 68(8), pages 1768-1771.
    12. Deirdre N. McCloskey & Stephen T. Ziliak, 2012. "Statistical Significance in the New Tom and the Old Tom: A Reply to Thomas Mayer," Econ Journal Watch, Econ Journal Watch, vol. 9(3), pages 298-308, September.
    13. Armstrong, J. Scott, 2011. "Illusions in Regression Analysis," MPRA Paper 81663, University Library of Munich, Germany.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:28:y:2012:i:3:p:695-711. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Dana Niculescu). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.