IDEAS home Printed from https://ideas.repec.org/p/cam/camdae/2577.html
   My bibliography  Save this paper

Supervising Sentiment Models: Market Signals or Human Expertise?

Author

Listed:
  • Babolmorad, N.
  • Massoud, N.

Abstract

We build a framework to examine how the training regime-rather than model architecture-drives the performance of financial sentiment models. Using firm-level news and parsimonious classifiers, we compare three supervision regimes: human-only, hybrid, and market-only (fully automated). The framework opens the "black box" of sentiment modeling by tracing how supervision shapes each component of the classifier. Across extensive tests, the hybrid regime consistently outperforms fully automated training in explaining variation in stock returns and trading volume, enhancing interpretability and economic relevance. Human input improves sentiment inference, offering new insights into information processing and price formation in financial markets.

Suggested Citation

  • Babolmorad, N. & Massoud, N., 2025. "Supervising Sentiment Models: Market Signals or Human Expertise?," Cambridge Working Papers in Economics 2577, Faculty of Economics, University of Cambridge.
  • Handle: RePEc:cam:camdae:2577
    as

    Download full text from publisher

    File URL: https://www.econ.cam.ac.uk/sites/default/files/publication-cwpe-pdfs/cwpe2577.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Luca Barbaglia & Sergio Consoli & Sebastiano Manzan, 2023. "Forecasting with Economic News," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 41(3), pages 708-719, July.
    2. Sanjiv R. Das & Mike Y. Chen, 2007. "Yahoo! for Amazon: Sentiment Extraction from Small Talk on the Web," Management Science, INFORMS, vol. 53(9), pages 1375-1388, September.
    3. Glasserman, Paul & Mamaysky, Harry, 2019. "Does Unusual News Forecast Market Stress?," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 54(5), pages 1937-1974, October.
    4. Filippou, Ilias & Taylor, Mark P. & Wang, Zigan, 2024. "Media Sentiment and Currency Reversals," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 59(3), pages 1401-1429, May.
    5. Cao, Sean & Jiang, Wei & Wang, Junbo & Yang, Baozhong, 2024. "From Man vs. Machine to Man + Machine: The art and AI of stock analyses," Journal of Financial Economics, Elsevier, vol. 160(C).
    6. Hirshleifer, David & Teoh, Siew Hong, 2003. "Limited attention, information disclosure, and financial reporting," Journal of Accounting and Economics, Elsevier, vol. 36(1-3), pages 337-386, December.
    7. Liu, Baixiao & McConnell, John J., 2013. "The role of the media in corporate governance: Do the media influence managers' capital allocation decisions?," Journal of Financial Economics, Elsevier, vol. 110(1), pages 1-17.
    8. Morgan, I G, 1976. "Stock Prices and Heteroscedasticity," The Journal of Business, University of Chicago Press, vol. 49(4), pages 496-508, October.
    9. García, Diego & Hu, Xiaowen & Rohrer, Maximilian, 2023. "The colour of finance words," Journal of Financial Economics, Elsevier, vol. 147(3), pages 525-549.
    10. Matthew Gentzkow & Jesse M. Shapiro, 2010. "What Drives Media Slant? Evidence From U.S. Daily Newspapers," Econometrica, Econometric Society, vol. 78(1), pages 35-71, January.
    11. Shapiro, Adam Hale & Sudhof, Moritz & Wilson, Daniel J., 2022. "Measuring news sentiment," Journal of Econometrics, Elsevier, vol. 228(2), pages 221-243.
    12. David Hirshleifer & Sonya S. Lim & Siew Hong Teoh, 2011. "Limited Investor Attention and Stock Market Misreactions to Accounting Information," The Review of Asset Pricing Studies, Society for Financial Studies, vol. 1(1), pages 35-73.
    13. Isil Erel & Léa H Stern & Chenhao Tan & Michael S Weisbach, 2021. "Selecting Directors Using Machine Learning," NBER Chapters, in: Big Data: Long-Term Implications for Financial Markets and Firms, pages 3226-3264, National Bureau of Economic Research, Inc.
    14. Jeon, Yoontae & McCurdy, Thomas H. & Zhao, Xiaofei, 2022. "News as sources of jumps in stock returns: Evidence from 21 million news articles for 9000 companies," Journal of Financial Economics, Elsevier, vol. 145(2), pages 1-17.
    15. Grossman, Sanford J & Stiglitz, Joseph E, 1980. "On the Impossibility of Informationally Efficient Markets," American Economic Review, American Economic Association, vol. 70(3), pages 393-408, June.
    16. Veronesi, Pietro, 1999. "Stock Market Overreaction to Bad News in Good Times: A Rational Expectations Equilibrium Model," The Review of Financial Studies, Society for Financial Studies, vol. 12(5), pages 975-1007.
    17. Pekka Malo & Ankur Sinha & Pekka Korhonen & Jyrki Wallenius & Pyry Takala, 2014. "Good debt or bad debt: Detecting semantic orientations in economic texts," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(4), pages 782-796, April.
    18. Miller, Edward M, 1977. "Risk, Uncertainty, and Divergence of Opinion," Journal of Finance, American Finance Association, vol. 32(4), pages 1151-1168, September.
    19. Leif Anders Thorsrud, 2020. "Words are the New Numbers: A Newsy Coincident Index of the Business Cycle," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(2), pages 393-409, April.
    20. Harris, Milton & Raviv, Artur, 1993. "Differences of Opinion Make a Horse Race," The Review of Financial Studies, Society for Financial Studies, vol. 6(3), pages 473-506.
    21. Zijia Du & Alan Guoming Huang & Russ Wermers & Wenfeng Wu, 2022. "Language and Domain Specificity: A Chinese Financial Sentiment Dictionary [The effects of analyst-country institutions on biased research: Evidence from target prices]," Review of Finance, European Finance Association, vol. 26(3), pages 673-719.
    22. Harrison Hong & Jeremy C. Stein, 1999. "A Unified Theory of Underreaction, Momentum Trading, and Overreaction in Asset Markets," Journal of Finance, American Finance Association, vol. 54(6), pages 2143-2184, December.
    23. De Long, J Bradford & Andrei Shleifer & Lawrence H. Summers & Robert J. Waldmann, 1990. "Noise Trader Risk in Financial Markets," Journal of Political Economy, University of Chicago Press, vol. 98(4), pages 703-738, August.
    24. Tim Loughran & Bill Mcdonald, 2011. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks," Journal of Finance, American Finance Association, vol. 66(1), pages 35-65, February.
    25. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    26. Larry G. Epstein & Martin Schneider, 2008. "Ambiguity, Information Quality, and Asset Pricing," Journal of Finance, American Finance Association, vol. 63(1), pages 197-228, February.
    27. Harris, Lawrence E & Gurel, Eitan, 1986. "Price and Volume Effects Associated with Changes in the S&P 500 List: New Evidence for the Existence of Price Pressures," Journal of Finance, American Finance Association, vol. 41(4), pages 815-829, September.
    28. Paul C. Tetlock & Maytal Saar‐Tsechansky & Sofus Macskassy, 2008. "More Than Words: Quantifying Language to Measure Firms' Fundamentals," Journal of Finance, American Finance Association, vol. 63(3), pages 1437-1467, June.
    29. Kyle, Albert S, 1985. "Continuous Auctions and Insider Trading," Econometrica, Econometric Society, vol. 53(6), pages 1315-1335, November.
    30. Diego García, 2013. "Sentiment during Recessions," Journal of Finance, American Finance Association, vol. 68(3), pages 1267-1300, June.
    31. Shuaiyu Chen & T. Clifton Green & Huseyin Gulen & Dexin Zhou, 2024. "What Does ChatGPT Make of Historical Stock Returns? Extrapolation and Miscalibration in LLM Stock Return Forecasts," Papers 2409.11540, arXiv.org.
    32. Manela, Asaf & Moreira, Alan, 2017. "News implied volatility and disaster concerns," Journal of Financial Economics, Elsevier, vol. 123(1), pages 137-162.
    33. Paul C. Tetlock, 2007. "Giving Content to Investor Sentiment: The Role of Media in the Stock Market," Journal of Finance, American Finance Association, vol. 62(3), pages 1139-1168, June.
    34. Jules H van Binsbergen & Xiao Han & Alejandro Lopez-Lira, 2023. "Man versus Machine Learning: The Term Structure of Earnings Expectations and Conditional Biases," The Review of Financial Studies, Society for Financial Studies, vol. 36(6), pages 2361-2396.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tom Marty & Bruce Vanstone & Tobias Hahn, 2020. "News media analytics in finance: a survey," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 60(2), pages 1385-1434, June.
    2. Daniele Ballinari & Simon Behrendt, 2021. "How to gauge investor behavior? A comparison of online investor sentiment measures," Digital Finance, Springer, vol. 3(2), pages 169-204, June.
    3. Blankespoor, Elizabeth & deHaan, Ed & Marinovic, Iván, 2020. "Disclosure processing costs, investors’ information choice, and equity market outcomes: A review," Journal of Accounting and Economics, Elsevier, vol. 70(2).
    4. Andres Algaba & David Ardia & Keven Bluteau & Samuel Borms & Kris Boudt, 2020. "Econometrics Meets Sentiment: An Overview Of Methodology And Applications," Journal of Economic Surveys, Wiley Blackwell, vol. 34(3), pages 512-547, July.
    5. Chen, Cathy Yi-Hsuan & Fengler, Matthias R. & Härdle, Wolfgang Karl & Liu, Yanchu, 2022. "Media-expressed tone, option characteristics, and stock return predictability," Journal of Economic Dynamics and Control, Elsevier, vol. 134(C).
    6. Marie Bessec & Julien Fouquau, 2024. "A Green Wave in Media: A Change of Tack in Stock Markets," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 86(5), pages 1026-1057, October.
    7. Renault, Thomas, 2017. "Intraday online investor sentiment and return patterns in the U.S. stock market," Journal of Banking & Finance, Elsevier, vol. 84(C), pages 25-40.
    8. David Hirshleife, 2015. "Behavioral Finance," Annual Review of Financial Economics, Annual Reviews, vol. 7(1), pages 133-159, December.
    9. Szymon Lis, 2022. "Investor Sentiment in Asset Pricing Models: A Review," Working Papers 2022-14, Faculty of Economic Sciences, University of Warsaw.
    10. Ji Sun & Yi Zhou & Jiaguo (George) Wang & Jie (Michael) Guo, 2020. "Influence of media coverage and sentiment on seasoned equity offerings," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 60(S1), pages 557-585, April.
    11. Tim Loughran & Bill Mcdonald, 2016. "Textual Analysis in Accounting and Finance: A Survey," Journal of Accounting Research, John Wiley & Sons, Ltd., vol. 54(4), pages 1187-1230, September.
    12. Ahmad, Khurshid & Han, JingGuang & Hutson, Elaine & Kearney, Colm & Liu, Sha, 2016. "Media-expressed negative tone and firm-level stock returns," Journal of Corporate Finance, Elsevier, vol. 37(C), pages 152-172.
    13. Szymon Lis, 2024. "Investor Sentiment in Asset Pricing Models: A Review of Empirical Evidence," Papers 2411.13180, arXiv.org.
    14. Jian Chen & Guohao Tang & Guofu Zhou & Wu Zhu, 2025. "ChatGPT and Deepseek: Can They Predict the Stock Market and Macroeconomy?," Papers 2502.10008, arXiv.org.
    15. Wu, Chen-Hui & Lin, Chan-Jane, 2017. "The impact of media coverage on investor trading behavior and stock returns," Pacific-Basin Finance Journal, Elsevier, vol. 43(C), pages 151-172.
    16. Chia-Wei Chen & Christos Pantzalis & Jung Chul Park, 2013. "Press Coverage And Stock Price Deviation From Fundamental Value," Journal of Financial Research, Southern Finance Association;Southwestern Finance Association, vol. 36(2), pages 175-214, June.
    17. Kirtac, Kemal & Germano, Guido, 2024. "Sentiment trading with large language models," Finance Research Letters, Elsevier, vol. 62(PB).
    18. Obaid, Khaled & Pukthuanthong, Kuntara, 2022. "A picture is worth a thousand words: Measuring investor sentiment by combining machine learning and photos from news," Journal of Financial Economics, Elsevier, vol. 144(1), pages 273-297.
    19. Sun, Licheng & Najand, Mohammad & Shen, Jiancheng, 2016. "Stock return predictability and investor sentiment: A high-frequency perspective," Journal of Banking & Finance, Elsevier, vol. 73(C), pages 147-164.
    20. Vegard Høghaug Larsen & Leif Anders Thorsrud, 2022. "Asset returns, news topics, and media effects," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 838-868, July.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • G02 - Financial Economics - - General - - - Behavioral Finance: Underlying Principles
    • G11 - Financial Economics - - General Financial Markets - - - Portfolio Choice; Investment Decisions
    • G12 - Financial Economics - - General Financial Markets - - - Asset Pricing; Trading Volume; Bond Interest Rates
    • G14 - Financial Economics - - General Financial Markets - - - Information and Market Efficiency; Event Studies; Insider Trading

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cam:camdae:2577. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Jake Dyer (email available below). General contact details of provider: https://www.econ.cam.ac.uk/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.