IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/22959.html
   My bibliography  Save this paper

Box Office Buzz: Does Social Media Data Steal the Show from Model Uncertainty When Forecasting for Hollywood?

Author

Listed:
  • Steven Lehrer
  • Tian Xie

Abstract

Substantial excitement currently exists in industry regarding the potential of using analytic tools to measure sentiment in social media messages to help predict individual reactions to a new product, including movies. However, the majority of models subsequently used for forecasting exercises do not allow for model uncertainty. Using data on the universe of Twitter messages, we use an algorithm that calculates the sentiment regarding each film prior to, and after its release date via emotional valence to understand whether these opinions affect box office opening and retail movie unit (DVD and Blu-Ray) sales. Our results contrasting eleven different empirical strategies from econometrics and penalization methods indicate that accounting for model uncertainty can lead to large gains in forecast accuracy. While penalization methods do not outperform model averaging on forecast accuracy, evidence indicates they perform just as well at the variable selection stage. Last, incorporating social media data is shown to greatly improve forecast accuracy for box-office opening and retail movie unit sales.

Suggested Citation

  • Steven Lehrer & Tian Xie, 2016. "Box Office Buzz: Does Social Media Data Steal the Show from Model Uncertainty When Forecasting for Hollywood?," NBER Working Papers 22959, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:22959
    Note: TWP
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w22959.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Julia Campos & David F. Hendry & Hans‐Martin Krolzig, 2003. "Consistent Model Selection by an Automatic Gets Approach," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 65(s1), pages 803-819, December.
    2. Amemiya, Takeshi, 1980. "Selection of Regressors," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 21(2), pages 331-354, June.
    3. Xie, Tian, 2015. "Prediction model averaging estimator," Economics Letters, Elsevier, vol. 131(C), pages 5-8.
    4. David F. Hendry & Bent Nielsen, 2007. "Preface to Econometric Modeling: A Likelihood Approach," Introductory Chapters, in: Econometric Modeling: A Likelihood Approach, Princeton University Press.
    5. Enrico Moretti, 2011. "Social Learning and Peer Effects in Consumption: Evidence from Movie Sales," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 78(1), pages 356-393.
    6. David F. Hendry & Bent Nielsen, 2007. "The Bernoulli model, from Econometric Modeling: A Likelihood Approach," Introductory Chapters, in: Econometric Modeling: A Likelihood Approach, Princeton University Press.
    7. Zhang, Xinyu & Wan, Alan T.K. & Zou, Guohua, 2013. "Model averaging by jackknife criterion in models with dependent data," Journal of Econometrics, Elsevier, vol. 174(2), pages 82-94.
    8. Wan, Alan T.K. & Zhang, Xinyu & Zou, Guohua, 2010. "Least squares model averaging by Mallows criterion," Journal of Econometrics, Elsevier, vol. 156(2), pages 277-283, June.
    9. Hansen, Bruce E. & Racine, Jeffrey S., 2012. "Jackknife model averaging," Journal of Econometrics, Elsevier, vol. 167(1), pages 38-46.
    10. Bruce E. Hansen, 2007. "Least Squares Model Averaging," Econometrica, Econometric Society, vol. 75(4), pages 1175-1189, July.
    11. Pradeep K. Chintagunta & Shyam Gopinath & Sriram Venkataraman, 2010. "The Effects of Online User Reviews on Movie Box Office Performance: Accounting for Sequential Rollout and Aggregation Across Local Markets," Marketing Science, INFORMS, vol. 29(5), pages 944-957, 09-10.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Carstensen, Kai & Bachmann, Rüdiger & Schneider, Martin & Lautenbacher, Stefan, 2018. "Uncertainty is Change," VfS Annual Conference 2018 (Freiburg, Breisgau): Digital Economy 181572, Verein für Socialpolitik / German Economic Association.
    2. Steven F. Lehrer & Tian Xie, 2022. "The Bigger Picture: Combining Econometrics with Analytics Improves Forecasts of Movie Success," Management Science, INFORMS, vol. 68(1), pages 189-210, January.
    3. Sun, Yuying & Hong, Yongmiao & Wang, Shouyang & Zhang, Xinyu, 2023. "Penalized time-varying model averaging," Journal of Econometrics, Elsevier, vol. 235(2), pages 1355-1377.
    4. Steven Lehrer & Tian Xie & Tao Zeng, 2021. "Does High-Frequency Social Media Data Improve Forecasts of Low-Frequency Consumer Confidence Measures? [Regression Models with Mixed Sampling Frequencies]," Journal of Financial Econometrics, Oxford University Press, vol. 19(5), pages 910-933.
    5. Hui Xiao & Yiguo Sun, 2019. "On Tuning Parameter Selection in Model Selection and Model Averaging: A Monte Carlo Study," JRFM, MDPI, vol. 12(3), pages 1-16, June.
    6. Zhao, Shangwei & Xie, Tian & Ai, Xin & Yang, Guangren & Zhang, Xinyu, 2023. "Correcting sample selection bias with model averaging for consumer demand forecasting," Economic Modelling, Elsevier, vol. 123(C).
    7. Jordi McKenzie, 2023. "The economics of movies (revisited): A survey of recent literature," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 480-525, April.
    8. Qiu, Yue & Wang, Zongrun & Xie, Tian & Zhang, Xinyu, 2021. "Forecasting Bitcoin realized volatility by exploiting measurement error under model uncertainty," Journal of Empirical Finance, Elsevier, vol. 62(C), pages 179-201.
    9. Qiu, Yue & Zheng, Yuchen, 2023. "Improving box office projections through sentiment analysis: Insights from regularization-based forecast combinations," Economic Modelling, Elsevier, vol. 125(C).
    10. Jianchun Fang & Wanshan Wu & Zhou Lu & Eunho Cho, 2019. "Using Baidu Index To Nowcast Mobile Phone Sales In China," The Singapore Economic Review (SER), World Scientific Publishing Co. Pte. Ltd., vol. 64(01), pages 83-96, March.
    11. Gabriel Okasa & Kenneth A. Younge, 2022. "Sample Fit Reliability," Papers 2209.06631, arXiv.org.
    12. Lehrer, Steven & Xie, Tian & Zhang, Xinyu, 2021. "Social media sentiment, model uncertainty, and volatility forecasting," Economic Modelling, Elsevier, vol. 102(C).
    13. Zhentao Shi & Liangjun Su & Tian Xie, 2020. "L2-Relaxation: With Applications to Forecast Combination and Portfolio Analysis," Papers 2010.09477, arXiv.org, revised Aug 2022.
    14. Xie, Tian, 2017. "Heteroscedasticity-robust model screening: A useful toolkit for model averaging in big data analytics," Economics Letters, Elsevier, vol. 151(C), pages 119-122.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Steven F. Lehrer & Tian Xie, 2022. "The Bigger Picture: Combining Econometrics with Analytics Improves Forecasts of Movie Success," Management Science, INFORMS, vol. 68(1), pages 189-210, January.
    2. Liao, Jun & Zou, Guohua, 2020. "Corrected Mallows criterion for model averaging," Computational Statistics & Data Analysis, Elsevier, vol. 144(C).
    3. Zhao, Shangwei & Xie, Tian & Ai, Xin & Yang, Guangren & Zhang, Xinyu, 2023. "Correcting sample selection bias with model averaging for consumer demand forecasting," Economic Modelling, Elsevier, vol. 123(C).
    4. Zhao, Shangwei & Zhang, Xinyu & Gao, Yichen, 2016. "Model averaging with averaging covariance matrix," Economics Letters, Elsevier, vol. 145(C), pages 214-217.
    5. Xie, Tian, 2017. "Heteroscedasticity-robust model screening: A useful toolkit for model averaging in big data analytics," Economics Letters, Elsevier, vol. 151(C), pages 119-122.
    6. Tian Xie, 2019. "Forecast Bitcoin Volatility with Least Squares Model Averaging," Econometrics, MDPI, vol. 7(3), pages 1-20, September.
    7. Haili Zhang & Guohua Zou, 2020. "Cross-Validation Model Averaging for Generalized Functional Linear Model," Econometrics, MDPI, vol. 8(1), pages 1-35, February.
    8. Kenichiro McAlinn & Kosaku Takanashi, 2019. "Mean-shift least squares model averaging," Papers 1912.01194, arXiv.org.
    9. Xie, Tian, 2015. "Prediction model averaging estimator," Economics Letters, Elsevier, vol. 131(C), pages 5-8.
    10. Yuting Wei & Qihua Wang & Wei Liu, 2021. "Model averaging for linear models with responses missing at random," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 73(3), pages 535-553, June.
    11. Yan Gao & Xinyu Zhang & Shouyang Wang & Terence Tai-leung Chong & Guohua Zou, 2019. "Frequentist model averaging for threshold models," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 71(2), pages 275-306, April.
    12. Rongjie Jiang & Liming Wang & Yang Bai, 2021. "Optimal model averaging estimator for semi-functional partially linear models," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 84(2), pages 167-194, February.
    13. Chu-An Liu & Biing-Shen Kuo & Wen-Jen Tsay, 2017. "Autoregressive Spectral Averaging Estimator," IEAS Working Paper : academic research 17-A013, Institute of Economics, Academia Sinica, Taipei, Taiwan.
    14. Shangwei Zhao & Jun Liao & Dalei Yu, 2020. "Model averaging estimator in ridge regression and its large sample properties," Statistical Papers, Springer, vol. 61(4), pages 1719-1739, August.
    15. Qingfeng Liu & Andrey L. Vasnev, 2019. "A Combination Method for Averaging OLS and GLS Estimators," Econometrics, MDPI, vol. 7(3), pages 1-12, September.
    16. Hongwei Zhang & Qiang He & Ben Jacobsen & Fuwei Jiang, 2020. "Forecasting stock returns with model uncertainty and parameter instability," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(5), pages 629-644, August.
    17. Alena Skolkova, 2023. "Model Averaging with Ridge Regularization," CERGE-EI Working Papers wp758, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    18. Wan, Alan T.K. & Zhang, Xinyu & Wang, Shouyang, 2014. "Frequentist model averaging for multinomial and ordered logit models," International Journal of Forecasting, Elsevier, vol. 30(1), pages 118-128.
    19. Yin-Wong Cheung & Wenhao Wang, 2020. "A Jackknife Model Averaging Analysis of RMB Misalignment Estimates," Journal of International Commerce, Economics and Policy (JICEP), World Scientific Publishing Co. Pte. Ltd., vol. 11(02), pages 1-45, June.
    20. Lu, Xun & Su, Liangjun, 2015. "Jackknife model averaging for quantile regressions," Journal of Econometrics, Elsevier, vol. 188(1), pages 40-58.

    More about this item

    JEL classification:

    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • M21 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Business Economics - - - Business Economics

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:22959. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.