IDEAS home Printed from https://ideas.repec.org/a/eee/jfinec/v144y2022i1p273-297.html
   My bibliography  Save this article

A picture is worth a thousand words: Measuring investor sentiment by combining machine learning and photos from news

Author

Listed:
  • Obaid, Khaled
  • Pukthuanthong, Kuntara

Abstract

By applying machine learning to the accurate and cost-effective classification of photos based on sentiment, we introduce a daily market-level investor sentiment index (Photo Pessimism) obtained from a large sample of news photos. Consistent with behavioral models, Photo Pessimism predicts market return reversals and trading volume. The relation is strongest among stocks with high limits to arbitrage and during periods of elevated fear. We examine whether Photo Pessimism and pessimism embedded in news text act as complements or substitutes for each other in predicting stock returns and find evidence that the two are substitutes.

Suggested Citation

  • Obaid, Khaled & Pukthuanthong, Kuntara, 2022. "A picture is worth a thousand words: Measuring investor sentiment by combining machine learning and photos from news," Journal of Financial Economics, Elsevier, vol. 144(1), pages 273-297.
    • Handle: RePEc:eee:jfinec:v:144:y:2022:i:1:p:273-297
    DOI: 10.1016/j.jfineco.2021.06.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304405X21002683
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jfineco.2021.06.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ivo Welch & Amit Goyal, 2008. "A Comprehensive Look at The Empirical Performance of Equity Premium Prediction," Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1455-1508, July.
    2. Guanhao Feng & Stefano Giglio & Dacheng Xiu, 2020. "Taming the Factor Zoo: A Test of New Factors," Journal of Finance, American Finance Association, vol. 75(3), pages 1327-1370, June.
    3. David Hirshleifer & Tyler Shumway, 2003. "Good Day Sunshine: Stock Returns and the Weather," Journal of Finance, American Finance Association, vol. 58(3), pages 1009-1032, June.
    4. Malcolm Baker & Jeffrey Wurgler, 2006. "Investor Sentiment and the Cross‐Section of Stock Returns," Journal of Finance, American Finance Association, vol. 61(4), pages 1645-1680, August.
    5. Zhi Da & Joseph Engelberg & Pengjie Gao, 2015. "Editor's Choice The Sum of All FEARS Investor Sentiment and Asset Prices," Review of Financial Studies, Society for Financial Studies, vol. 28(1), pages 1-32.
    6. Calomiris, Charles W. & Mamaysky, Harry, 2019. "How news and its context drive risk and returns around the world," Journal of Financial Economics, Elsevier, vol. 133(2), pages 299-336.
    7. De Long, J Bradford & Andrei Shleifer & Lawrence H. Summers & Robert J. Waldmann, 1990. "Noise Trader Risk in Financial Markets," Journal of Political Economy, University of Chicago Press, vol. 98(4), pages 703-738, August.
    8. John Y. Campbell & Sanford J. Grossman & Jiang Wang, 1993. "Trading Volume and Serial Correlation in Stock Returns," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 108(4), pages 905-939.
    9. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
    10. Robert Connolly & Chris Stivers, 2003. "Momentum and Reversals in Equity‐Index Returns During Periods of Abnormal Turnover and Return Dispersion," Journal of Finance, American Finance Association, vol. 58(4), pages 1521-1556, August.
    11. Sendhil Mullainathan & Andrei Shleifer, 2005. "The Market for News," American Economic Review, American Economic Association, vol. 95(4), pages 1031-1053, September.
    12. David Hirshleifer, 2001. "Investor Psychology and Asset Pricing," Journal of Finance, American Finance Association, vol. 56(4), pages 1533-1597, August.
    13. Bryan Kelly & Seth Pruitt, 2013. "Market Expectations in the Cross-Section of Present Values," Journal of Finance, American Finance Association, vol. 68(5), pages 1721-1756, October.
    14. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    15. Fama, Eugene F. & French, Kenneth R., 2015. "A five-factor asset pricing model," Journal of Financial Economics, Elsevier, vol. 116(1), pages 1-22.
    16. Yongqiang Chu & David Hirshleifer & Liang Ma, 2020. "The Causal Effect of Limits to Arbitrage on Asset Pricing Anomalies," Journal of Finance, American Finance Association, vol. 75(5), pages 2631-2672, October.
    17. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    18. Matthias M M Buehlmaier & Toni M Whited, 2018. "Are Financial Constraints Priced? Evidence from Textual Analysis," Review of Financial Studies, Society for Financial Studies, vol. 31(7), pages 2693-2728.
    19. Matthew Gentzkow & Jesse M. Shapiro, 2010. "What Drives Media Slant? Evidence From U.S. Daily Newspapers," Econometrica, Econometric Society, vol. 78(1), pages 35-71, January.
    20. John Y. Campbell & Samuel B. Thompson, 2008. "Predicting Excess Stock Returns Out of Sample: Can Anything Beat the Historical Average?," Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1509-1531, July.
    21. John R. Graham & Campbell R. Harvey & Manju Puri, 2017. "A Corporate Beauty Contest," Management Science, INFORMS, vol. 63(9), pages 3044-3056, September.
    22. Nicolaj Siggelkow, 2002. "Misperceiving Interactions Among Complements and Substitutes: Organizational Consequences," Management Science, INFORMS, vol. 48(7), pages 900-916, July.
    23. Elizabeth Blankespoor & Bradley E. Hendricks & Gregory S. Miller, 2017. "Perceptions and Price: Evidence from CEO Presentations at IPO Roadshows," Journal of Accounting Research, Wiley Blackwell, vol. 55(2), pages 275-327, May.
    24. Guofu Zhou, 2018. "Measuring Investor Sentiment," Annual Review of Financial Economics, Annual Reviews, vol. 10(1), pages 239-259, November.
    25. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    26. John H. Cochrane, 2011. "Presidential Address: Discount Rates," Journal of Finance, American Finance Association, vol. 66(4), pages 1047-1108, August.
    27. Jeffrey Wurgler & Ekaterina Zhuravskaya, 2002. "Does Arbitrage Flatten Demand Curves for Stocks?," The Journal of Business, University of Chicago Press, vol. 75(4), pages 583-608, October.
    28. Carhart, Mark M, 1997. "On Persistence in Mutual Fund Performance," Journal of Finance, American Finance Association, vol. 52(1), pages 57-82, March.
    29. Diego García, 2013. "Sentiment during Recessions," Journal of Finance, American Finance Association, vol. 68(3), pages 1267-1300, June.
    30. Cox, Don R & Peterson, David R, 1994. "Stock Returns Following Large One-Day Declines: Evidence on Short-Term Reversals and Longer-Term Performance," Journal of Finance, American Finance Association, vol. 49(1), pages 255-267, March.
    31. Zhi Da & Qianqiu Liu & Ernst Schaumburg, 2014. "A Closer Look at the Short-Term Return Reversal," Management Science, INFORMS, vol. 60(3), pages 658-674, March.
    32. Manela, Asaf & Moreira, Alan, 2017. "News implied volatility and disaster concerns," Journal of Financial Economics, Elsevier, vol. 123(1), pages 137-162.
    33. Shleifer, Andrei & Vishny, Robert W, 1997. "The Limits of Arbitrage," Journal of Finance, American Finance Association, vol. 52(1), pages 35-55, March.
    34. Alex Edmans & Diego García & Øyvind Norli, 2007. "Sports Sentiment and Stock Returns," Journal of Finance, American Finance Association, vol. 62(4), pages 1967-1998, August.
    35. Jiang, Fuwei & Lee, Joshua & Martin, Xiumin & Zhou, Guofu, 2019. "Manager sentiment and stock returns," Journal of Financial Economics, Elsevier, vol. 132(1), pages 126-149.
    36. Sendhil Mullainathan & Jann Spiess, 2017. "Machine Learning: An Applied Econometric Approach," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 87-106, Spring.
    37. Rapach, David E. & Ringgenberg, Matthew C. & Zhou, Guofu, 2016. "Short interest and aggregate stock returns," Journal of Financial Economics, Elsevier, vol. 121(1), pages 46-65.
    38. Fama, Eugene F. & French, Kenneth R., 1993. "Common risk factors in the returns on stocks and bonds," Journal of Financial Economics, Elsevier, vol. 33(1), pages 3-56, February.
    39. Jeffrey Pontiff, 1996. "Costly Arbitrage: Evidence from Closed-End Funds," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 111(4), pages 1135-1151.
    40. Paul C. Tetlock, 2007. "Giving Content to Investor Sentiment: The Role of Media in the Stock Market," Journal of Finance, American Finance Association, vol. 62(3), pages 1139-1168, June.
    41. Serhiy Kozak & Stefan Nagel & Shrihari Santosh, 2018. "Interpreting Factor Models," Journal of Finance, American Finance Association, vol. 73(3), pages 1183-1223, June.
    42. Matthew Spiegel, 2008. "Forecasting the Equity Premium: Where We Stand Today," Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1453-1454, July.
    43. Sendhil Mullainathan & Andrei Shleifer, 2005. "The Market for News," American Economic Review, American Economic Association, vol. 95(4), pages 1031-1053, September.
    44. Tim Loughran & Bill Mcdonald, 2011. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks," Journal of Finance, American Finance Association, vol. 66(1), pages 35-65, February.
    45. Hailiang Chen & Prabuddha De & Yu (Jeffrey) Hu & Byoung-Hyoun Hwang, 2014. "Wisdom of Crowds: The Value of Stock Opinions Transmitted Through Social Media," Review of Financial Studies, Society for Financial Studies, vol. 27(5), pages 1367-1403.
    46. Gallant, A Ronald & Rossi, Peter E & Tauchen, George, 1992. "Stock Prices and Volume," Review of Financial Studies, Society for Financial Studies, vol. 5(2), pages 199-242.
    47. Jefferson Duarte & Stephan Siegel & Lance Young, 2012. "Trust and Credit: The Role of Appearance in Peer-to-peer Lending," Review of Financial Studies, Society for Financial Studies, vol. 25(8), pages 2455-2484.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Helena Chuliá & Sabuhi Khalili & Jorge M. Uribe, 2024. "Monitoring time-varying systemic risk in sovereign debt and currency markets with generative AI," IREA Working Papers 202402, University of Barcelona, Research Institute of Applied Economics, revised Feb 2024.
    2. Hadhri, Sinda, 2023. "Do cryptocurrencies feel the music?," International Review of Financial Analysis, Elsevier, vol. 89(C).
    3. Sudarshan Kumar & Sobhesh Kumar Agarwalla & Jayanth R. Varma & Vineet Virmani, 2023. "Harvesting the volatility smile in a large emerging market: A Dynamic Nelson–Siegel approach," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 43(11), pages 1615-1644, November.
    4. Liang, Chao & Xu, Yongan & Wang, Jianqiong & Yang, Mo, 2022. "Whether dimensionality reduction techniques can improve the ability of sentiment proxies to predict stock market returns," International Review of Financial Analysis, Elsevier, vol. 82(C).
    5. Hoang, Daniel & Wiegratz, Kevin, 2022. "Machine learning methods in finance: Recent applications and prospects," Working Paper Series in Economics 158, Karlsruhe Institute of Technology (KIT), Department of Economics and Management.
    6. Xiaofeng Quan & Cheng Xiang & Donghui Li & Kelvin Jui Keng Tan, 2023. "To see is to believe: Corporate site visits and mutual fund herding," Financial Management, Financial Management Association International, vol. 52(4), pages 711-740, December.
    7. Sourav Prasad & Sabyasachi Mohapatra & Molla Ramizur Rahman & Amit Puniyani, 2022. "Investor Sentiment Index: A Systematic Review," IJFS, MDPI, vol. 11(1), pages 1-27, December.
    8. Mestiri, Sami, 2023. "How to use machine learning in finance," MPRA Paper 120045, University Library of Munich, Germany.
    9. Santi, Caterina, 2023. "Investor climate sentiment and financial markets," International Review of Financial Analysis, Elsevier, vol. 86(C).
    10. Tran, Vu Le, 2023. "Sentiment and covariance characteristics," International Review of Financial Analysis, Elsevier, vol. 86(C).
    11. John Hua Fan & Sebastian Binnewies & Sanuri De Silva, 2023. "Wisdom of crowds and commodity pricing," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 43(8), pages 1040-1068, August.
    12. Mohammad Arief Rajendra & Sekar Utami Setiastuti, 2023. "Climate Policy Uncertainty and the Demand for Renewable Energy in the United States of America: Evidence from a Non-Linear Threshold Autoregressive Model," Gadjah Mada Economics Working Paper Series 202312012, Department of Economics, Faculty of Economics and Business, Universitas Gadjah Mada.
    13. Durand, Robert B. & Khuu, Joyce & Smales, Lee A., 2023. "Lost in translation. When sentiment metrics for one market are derived from two different languages," Journal of Behavioral and Experimental Finance, Elsevier, vol. 39(C).
    14. Ben Jabeur, Sami & Serret, Vanessa, 2023. "Bankruptcy prediction using fuzzy convolutional neural networks," Research in International Business and Finance, Elsevier, vol. 64(C).
    15. Huynh, Nhan & Phan, Hoa, 2023. "Emotions in the crypto market: Do photos really speak?," Finance Research Letters, Elsevier, vol. 55(PB).
    16. Mestiri, Sami, 2024. "Financial applications of machine learning using R software," MPRA Paper 119998, University Library of Munich, Germany.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guofu Zhou, 2018. "Measuring Investor Sentiment," Annual Review of Financial Economics, Annual Reviews, vol. 10(1), pages 239-259, November.
    2. Chen, Jian & Tang, Guohao & Yao, Jiaquan & Zhou, Guofu, 2023. "Employee sentiment and stock returns," Journal of Economic Dynamics and Control, Elsevier, vol. 149(C).
    3. Zongwu Cai & Pixiong Chen, 2022. "New Online Investor Sentiment and Asset Returns," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 202216, University of Kansas, Department of Economics, revised Nov 2022.
    4. Santi, Caterina, 2023. "Investor climate sentiment and financial markets," International Review of Financial Analysis, Elsevier, vol. 86(C).
    5. Cakici, Nusret & Zaremba, Adam, 2022. "Salience theory and the cross-section of stock returns: International and further evidence," Journal of Financial Economics, Elsevier, vol. 146(2), pages 689-725.
    6. Lutzenberger, Fabian T., 2014. "The predictability of aggregate returns on commodity futures," Review of Financial Economics, Elsevier, vol. 23(3), pages 120-130.
    7. Jiang, Fuwei & Lee, Joshua & Martin, Xiumin & Zhou, Guofu, 2019. "Manager sentiment and stock returns," Journal of Financial Economics, Elsevier, vol. 132(1), pages 126-149.
    8. Daniele Ballinari & Simon Behrendt, 2021. "How to gauge investor behavior? A comparison of online investor sentiment measures," Digital Finance, Springer, vol. 3(2), pages 169-204, June.
    9. Radeef Chundakkadan, 2021. "Light a lamp and look at the stock market," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 7(1), pages 1-21, December.
    10. Xi Dong & Yan Li & David E. Rapach & Guofu Zhou, 2022. "Anomalies and the Expected Market Return," Journal of Finance, American Finance Association, vol. 77(1), pages 639-681, February.
    11. Alejandro Bernales & Marcela Valenzuela & Ilknur Zer, 2023. "Effects of Information Overload on Financial Markets: How Much Is Too Much?," International Finance Discussion Papers 1372, Board of Governors of the Federal Reserve System (U.S.).
    12. Fernandez-Perez, Adrian & Garel, Alexandre & Indriawan, Ivan, 2020. "Music sentiment and stock returns," Economics Letters, Elsevier, vol. 192(C).
    13. Huang, Dashan & Li, Jiangyuan & Wang, Liyao, 2021. "Are disagreements agreeable? Evidence from information aggregation," Journal of Financial Economics, Elsevier, vol. 141(1), pages 83-101.
    14. Barroso, Pedro & Detzel, Andrew, 2021. "Do limits to arbitrage explain the benefits of volatility-managed portfolios?," Journal of Financial Economics, Elsevier, vol. 140(3), pages 744-767.
    15. Lin, Qi & Lin, Xi, 2021. "Cash conversion cycle and aggregate stock returns," Journal of Financial Markets, Elsevier, vol. 52(C).
    16. Sun, Licheng & Najand, Mohammad & Shen, Jiancheng, 2016. "Stock return predictability and investor sentiment: A high-frequency perspective," Journal of Banking & Finance, Elsevier, vol. 73(C), pages 147-164.
    17. Renault, Thomas, 2017. "Intraday online investor sentiment and return patterns in the U.S. stock market," Journal of Banking & Finance, Elsevier, vol. 84(C), pages 25-40.
    18. Ma, Tian & Leong, Wen Jun & Jiang, Fuwei, 2023. "A latent factor model for the Chinese stock market," International Review of Financial Analysis, Elsevier, vol. 87(C).
    19. Ang, Tze Chuan ‘Chewie’ & Lam, F.Y. Eric C. & Wei, K.C. John, 2020. "Mispricing firm-level productivity," Journal of Empirical Finance, Elsevier, vol. 58(C), pages 139-163.
    20. Szymon Lis, 2022. "Investor Sentiment in Asset Pricing Models: A Review," Working Papers 2022-14, Faculty of Economic Sciences, University of Warsaw.

    More about this item

    Keywords

    Investor sentiment; Behavioral finance; Return predictability; Machine learning; Deep learning; Big data;
    All these keywords.

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • G10 - Financial Economics - - General Financial Markets - - - General (includes Measurement and Data)
    • G17 - Financial Economics - - General Financial Markets - - - Financial Forecasting and Simulation

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jfinec:v:144:y:2022:i:1:p:273-297. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/505576 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.