IDEAS home Printed from https://ideas.repec.org/p/cam/camdae/2577.html

Supervising Sentiment Models: Market Signals or Human Expertise?

Author

Listed:
  • Babolmorad, N.
  • Massoud, N.

Abstract

We build a framework to examine how the training regime-rather than model architecture-drives the performance of financial sentiment models. Using firm-level news and parsimonious classifiers, we compare three supervision regimes: human-only, hybrid, and market-only (fully automated). The framework opens the "black box" of sentiment modeling by tracing how supervision shapes each component of the classifier. Across extensive tests, the hybrid regime consistently outperforms fully automated training in explaining variation in stock returns and trading volume, enhancing interpretability and economic relevance. Human input improves sentiment inference, offering new insights into information processing and price formation in financial markets.

Suggested Citation

  • Babolmorad, N. & Massoud, N., 2025. "Supervising Sentiment Models: Market Signals or Human Expertise?," Cambridge Working Papers in Economics 2577, Faculty of Economics, University of Cambridge.
  • Handle: RePEc:cam:camdae:2577
    as

    Download full text from publisher

    File URL: https://www.econ.cam.ac.uk/sites/default/files/publication-cwpe-pdfs/cwpe2577.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zijia Du & Alan Guoming Huang & Russ Wermers & Wenfeng Wu, 2022. "Language and Domain Specificity: A Chinese Financial Sentiment Dictionary [The effects of analyst-country institutions on biased research: Evidence from target prices]," Review of Finance, European Finance Association, vol. 26(3), pages 673-719.
    2. Matthew Gentzkow & Jesse M. Shapiro, 2010. "What Drives Media Slant? Evidence From U.S. Daily Newspapers," Econometrica, Econometric Society, vol. 78(1), pages 35-71, January.
    3. Paul C. Tetlock & Maytal Saar‐Tsechansky & Sofus Macskassy, 2008. "More Than Words: Quantifying Language to Measure Firms' Fundamentals," Journal of Finance, American Finance Association, vol. 63(3), pages 1437-1467, June.
    4. Miller, Edward M, 1977. "Risk, Uncertainty, and Divergence of Opinion," Journal of Finance, American Finance Association, vol. 32(4), pages 1151-1168, September.
    5. Harrison Hong & Jeremy C. Stein, 1999. "A Unified Theory of Underreaction, Momentum Trading, and Overreaction in Asset Markets," Journal of Finance, American Finance Association, vol. 54(6), pages 2143-2184, December.
    6. De Long, J Bradford & Andrei Shleifer & Lawrence H. Summers & Robert J. Waldmann, 1990. "Noise Trader Risk in Financial Markets," Journal of Political Economy, University of Chicago Press, vol. 98(4), pages 703-738, August.
    7. Diego García, 2013. "Sentiment during Recessions," Journal of Finance, American Finance Association, vol. 68(3), pages 1267-1300, June.
    8. Morgan, I G, 1976. "Stock Prices and Heteroscedasticity," The Journal of Business, University of Chicago Press, vol. 49(4), pages 496-508, October.
    9. Luca Barbaglia & Sergio Consoli & Sebastiano Manzan, 2023. "Forecasting with Economic News," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 41(3), pages 708-719, July.
    10. García, Diego & Hu, Xiaowen & Rohrer, Maximilian, 2023. "The colour of finance words," Journal of Financial Economics, Elsevier, vol. 147(3), pages 525-549.
    11. Jules H van Binsbergen & Xiao Han & Alejandro Lopez-Lira, 2023. "Man versus Machine Learning: The Term Structure of Earnings Expectations and Conditional Biases," The Review of Financial Studies, Society for Financial Studies, vol. 36(6), pages 2361-2396.
    12. Sanjiv R. Das & Mike Y. Chen, 2007. "Yahoo! for Amazon: Sentiment Extraction from Small Talk on the Web," Management Science, INFORMS, vol. 53(9), pages 1375-1388, September.
    13. Shapiro, Adam Hale & Sudhof, Moritz & Wilson, Daniel J., 2022. "Measuring news sentiment," Journal of Econometrics, Elsevier, vol. 228(2), pages 221-243.
    14. Cao, Sean & Jiang, Wei & Wang, Junbo & Yang, Baozhong, 2024. "From Man vs. Machine to Man + Machine: The art and AI of stock analyses," Journal of Financial Economics, Elsevier, vol. 160(C).
    15. Kyle, Albert S, 1985. "Continuous Auctions and Insider Trading," Econometrica, Econometric Society, vol. 53(6), pages 1315-1335, November.
    16. Grossman, Sanford J & Stiglitz, Joseph E, 1980. "On the Impossibility of Informationally Efficient Markets," American Economic Review, American Economic Association, vol. 70(3), pages 393-408, June.
    17. Hirshleifer, David & Teoh, Siew Hong, 2003. "Limited attention, information disclosure, and financial reporting," Journal of Accounting and Economics, Elsevier, vol. 36(1-3), pages 337-386, December.
    18. Shuaiyu Chen & T. Clifton Green & Huseyin Gulen & Dexin Zhou, 2024. "What Does ChatGPT Make of Historical Stock Returns? Extrapolation and Miscalibration in LLM Stock Return Forecasts," Papers 2409.11540, arXiv.org.
    19. Larry G. Epstein & Martin Schneider, 2008. "Ambiguity, Information Quality, and Asset Pricing," Journal of Finance, American Finance Association, vol. 63(1), pages 197-228, February.
    20. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    21. Glasserman, Paul & Mamaysky, Harry, 2019. "Does Unusual News Forecast Market Stress?," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 54(5), pages 1937-1974, October.
    22. Tim Loughran & Bill Mcdonald, 2011. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks," Journal of Finance, American Finance Association, vol. 66(1), pages 35-65, February.
    23. Liu, Baixiao & McConnell, John J., 2013. "The role of the media in corporate governance: Do the media influence managers' capital allocation decisions?," Journal of Financial Economics, Elsevier, vol. 110(1), pages 1-17.
    24. Leif Anders Thorsrud, 2020. "Words are the New Numbers: A Newsy Coincident Index of the Business Cycle," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(2), pages 393-409, April.
    25. Filippou, Ilias & Taylor, Mark P. & Wang, Zigan, 2024. "Media Sentiment and Currency Reversals," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 59(3), pages 1401-1429, May.
    26. Harris, Lawrence E & Gurel, Eitan, 1986. "Price and Volume Effects Associated with Changes in the S&P 500 List: New Evidence for the Existence of Price Pressures," Journal of Finance, American Finance Association, vol. 41(4), pages 815-829, September.
    27. Manela, Asaf & Moreira, Alan, 2017. "News implied volatility and disaster concerns," Journal of Financial Economics, Elsevier, vol. 123(1), pages 137-162.
    28. Veronesi, Pietro, 1999. "Stock Market Overreaction to Bad News in Good Times: A Rational Expectations Equilibrium Model," The Review of Financial Studies, Society for Financial Studies, vol. 12(5), pages 975-1007.
    29. Harris, Milton & Raviv, Artur, 1993. "Differences of Opinion Make a Horse Race," The Review of Financial Studies, Society for Financial Studies, vol. 6(3), pages 473-506.
    30. Paul C. Tetlock, 2007. "Giving Content to Investor Sentiment: The Role of Media in the Stock Market," Journal of Finance, American Finance Association, vol. 62(3), pages 1139-1168, June.
    31. Jeon, Yoontae & McCurdy, Thomas H. & Zhao, Xiaofei, 2022. "News as sources of jumps in stock returns: Evidence from 21 million news articles for 9000 companies," Journal of Financial Economics, Elsevier, vol. 145(2), pages 1-17.
    32. David Hirshleifer & Sonya S. Lim & Siew Hong Teoh, 2011. "Limited Investor Attention and Stock Market Misreactions to Accounting Information," The Review of Asset Pricing Studies, Society for Financial Studies, vol. 1(1), pages 35-73.
    33. Isil Erel & Léa H Stern & Chenhao Tan & Michael S Weisbach, 2021. "Selecting Directors Using Machine Learning," NBER Chapters, in: Big Data: Long-Term Implications for Financial Markets and Firms, pages 3226-3264, National Bureau of Economic Research, Inc.
    34. Pekka Malo & Ankur Sinha & Pekka Korhonen & Jyrki Wallenius & Pyry Takala, 2014. "Good debt or bad debt: Detecting semantic orientations in economic texts," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(4), pages 782-796, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tom Marty & Bruce Vanstone & Tobias Hahn, 2020. "News media analytics in finance: a survey," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 60(2), pages 1385-1434, June.
    2. Blankespoor, Elizabeth & deHaan, Ed & Marinovic, Iván, 2020. "Disclosure processing costs, investors’ information choice, and equity market outcomes: A review," Journal of Accounting and Economics, Elsevier, vol. 70(2).
    3. Andres Algaba & David Ardia & Keven Bluteau & Samuel Borms & Kris Boudt, 2020. "Econometrics Meets Sentiment: An Overview Of Methodology And Applications," Journal of Economic Surveys, Wiley Blackwell, vol. 34(3), pages 512-547, July.
    4. Daniele Ballinari & Simon Behrendt, 2021. "How to gauge investor behavior? A comparison of online investor sentiment measures," Digital Finance, Springer, vol. 3(2), pages 169-204, June.
    5. Renault, Thomas, 2017. "Intraday online investor sentiment and return patterns in the U.S. stock market," Journal of Banking & Finance, Elsevier, vol. 84(C), pages 25-40.
    6. Szymon Lis, 2022. "Investor Sentiment in Asset Pricing Models: A Review," Working Papers 2022-14, Faculty of Economic Sciences, University of Warsaw.
    7. David Hirshleife, 2015. "Behavioral Finance," Annual Review of Financial Economics, Annual Reviews, vol. 7(1), pages 133-159, December.
    8. Marie Bessec & Julien Fouquau, 2024. "A Green Wave in Media: A Change of Tack in Stock Markets," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 86(5), pages 1026-1057, October.
    9. Ji Sun & Yi Zhou & Jiaguo (George) Wang & Jie (Michael) Guo, 2020. "Influence of media coverage and sentiment on seasoned equity offerings," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 60(S1), pages 557-585, April.
    10. Tim Loughran & Bill Mcdonald, 2016. "Textual Analysis in Accounting and Finance: A Survey," Journal of Accounting Research, John Wiley & Sons, Ltd., vol. 54(4), pages 1187-1230, September.
    11. Szymon Lis, 2024. "Investor Sentiment in Asset Pricing Models: A Review of Empirical Evidence," Papers 2411.13180, arXiv.org.
    12. Jian Chen & Guohao Tang & Guofu Zhou & Wu Zhu, 2025. "ChatGPT and Deepseek: Can They Predict the Stock Market and Macroeconomy?," Papers 2502.10008, arXiv.org.
    13. Ahmad, Khurshid & Han, JingGuang & Hutson, Elaine & Kearney, Colm & Liu, Sha, 2016. "Media-expressed negative tone and firm-level stock returns," Journal of Corporate Finance, Elsevier, vol. 37(C), pages 152-172.
    14. Wu, Chen-Hui & Lin, Chan-Jane, 2017. "The impact of media coverage on investor trading behavior and stock returns," Pacific-Basin Finance Journal, Elsevier, vol. 43(C), pages 151-172.
    15. Chia-Wei Chen & Christos Pantzalis & Jung Chul Park, 2013. "Press Coverage And Stock Price Deviation From Fundamental Value," Journal of Financial Research, Southern Finance Association;Southwestern Finance Association, vol. 36(2), pages 175-214, June.
    16. Chen, Cathy Yi-Hsuan & Fengler, Matthias R. & Härdle, Wolfgang Karl & Liu, Yanchu, 2022. "Media-expressed tone, option characteristics, and stock return predictability," Journal of Economic Dynamics and Control, Elsevier, vol. 134(C).
    17. Obaid, Khaled & Pukthuanthong, Kuntara, 2022. "A picture is worth a thousand words: Measuring investor sentiment by combining machine learning and photos from news," Journal of Financial Economics, Elsevier, vol. 144(1), pages 273-297.
    18. Sun, Licheng & Najand, Mohammad & Shen, Jiancheng, 2016. "Stock return predictability and investor sentiment: A high-frequency perspective," Journal of Banking & Finance, Elsevier, vol. 73(C), pages 147-164.
    19. Zhang, Xuetong & Zhang, Weiguo, 2023. "Information asymmetry, sentiment interactions, and asset price," The North American Journal of Economics and Finance, Elsevier, vol. 67(C).
    20. Aysan, Ahmet Faruk & Caporin, Massimiliano & Cepni, Oguzhan, 2024. "Not all words are equal: Sentiment and jumps in the cryptocurrency market," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 91(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • G02 - Financial Economics - - General - - - Behavioral Finance: Underlying Principles
    • G11 - Financial Economics - - General Financial Markets - - - Portfolio Choice; Investment Decisions
    • G12 - Financial Economics - - General Financial Markets - - - Asset Pricing; Trading Volume; Bond Interest Rates
    • G14 - Financial Economics - - General Financial Markets - - - Information and Market Efficiency; Event Studies; Insider Trading

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cam:camdae:2577. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Jake Dyer (email available below). General contact details of provider: https://www.econ.cam.ac.uk/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.