IDEAS home Printed from https://ideas.repec.org/a/eee/finlet/v67y2024ipas1544612324008675.html
   My bibliography  Save this article

Unveiling tone manipulation in MD&A: Evidence from ChatGPT experiments

Author

Listed:
  • Song, Piaopeng
  • Lu, Hanglin
  • Zhang, Yongjie

Abstract

This research uses ChatGPT-3.5 and ChatGPT-4 to investigate tone manipulation in the management discussion and analysis of annual reports of Chinese-listed companies. We find that the quantification of emotional content in text using financial BERT model and dictionary approach is inconsistent due to two kinds of manipulation: "Expression Manipulation" and "Word Manipulation". Based on ChatGPT we verify the manipulative behavior of complex expressions and sentiment word substitutions. Our research suggests that using ChatGPT with appropriate cue words can help alleviate tone manipulation in financial texts, with GPT-4 having a better effect.

Suggested Citation

  • Song, Piaopeng & Lu, Hanglin & Zhang, Yongjie, 2024. "Unveiling tone manipulation in MD&A: Evidence from ChatGPT experiments," Finance Research Letters, Elsevier, vol. 67(PA).
    • Handle: RePEc:eee:finlet:v:67:y:2024:i:pa:s1544612324008675
    DOI: 10.1016/j.frl.2024.105837
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1544612324008675
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.frl.2024.105837?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mark H. Lang & Russell J. Lundholm, 2000. "Voluntary Disclosure and Equity Offerings: Reducing Information Asymmetry or Hyping the Stock?," Contemporary Accounting Research, John Wiley & Sons, vol. 17(4), pages 623-662, December.
    2. Leippold, Markus, 2023. "Sentiment spin: Attacking financial sentiment with GPT-3," Finance Research Letters, Elsevier, vol. 55(PB).
    3. Dowling, Michael & Lucey, Brian, 2023. "ChatGPT for (Finance) research: The Bananarama Conjecture," Finance Research Letters, Elsevier, vol. 53(C).
    4. Markus Leippold, 2023. "Sentiment Spin: Attacking Financial Sentiment with GPT-3," Swiss Finance Institute Research Paper Series 23-11, Swiss Finance Institute.
    5. Shuyu Zhang & Walter Aerts & Dunli Zhang & Zishan Chen, 2022. "Positive tone and initial coin offering," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 62(2), pages 2237-2266, June.
    6. Pekka Malo & Ankur Sinha & Pekka Korhonen & Jyrki Wallenius & Pyry Takala, 2014. "Good debt or bad debt: Detecting semantic orientations in economic texts," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(4), pages 782-796, April.
    7. Oehler, Andreas & Horn, Matthias, 2024. "Does ChatGPT provide better advice than robo-advisors?," Finance Research Letters, Elsevier, vol. 60(C).
    8. Wei, Tian & Wu, Han & Chu, Gang, 2023. "Is ChatGPT competent? Heterogeneity in the cognitive schemas of financial auditors and robots," International Review of Economics & Finance, Elsevier, vol. 88(C), pages 1389-1396.
    9. Tim Loughran & Bill Mcdonald, 2011. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks," Journal of Finance, American Finance Association, vol. 66(1), pages 35-65, February.
    10. Maciej P. Polak & Dane Morgan, 2024. "Extracting accurate materials data from research papers with conversational language models and prompt engineering," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Julian Junyan Wang & Victor Xiaoqi Wang, 2025. "Assessing Consistency and Reproducibility in the Outputs of Large Language Models: Evidence Across Diverse Finance and Accounting Tasks," Papers 2503.16974, arXiv.org, revised Mar 2025.
    2. Yuqi Nie & Yaxuan Kong & Xiaowen Dong & John M. Mulvey & H. Vincent Poor & Qingsong Wen & Stefan Zohren, 2024. "A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges," Papers 2406.11903, arXiv.org.
    3. Smales, Lee A., 2023. "Classification of RBA monetary policy announcements using ChatGPT," Finance Research Letters, Elsevier, vol. 58(PC).
    4. Li Xian Liu & Zhiyue Sun & Kunpeng Xu & Chao Chen, 2024. "AI-Driven Financial Analysis: Exploring ChatGPT’s Capabilities and Challenges," IJFS, MDPI, vol. 12(3), pages 1-35, June.
    5. Sui, Cong & Wang, Shuhan & Zheng, Wei, 2024. "Sentiment as a shipping market predictor: Testing market-specific language models," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 189(C).
    6. Kirtac, Kemal & Germano, Guido, 2024. "Sentiment trading with large language models," Finance Research Letters, Elsevier, vol. 62(PB).
    7. Chen, Cathy Yi-Hsuan & Fengler, Matthias R. & Härdle, Wolfgang Karl & Liu, Yanchu, 2022. "Media-expressed tone, option characteristics, and stock return predictability," Journal of Economic Dynamics and Control, Elsevier, vol. 134(C).
    8. Bingler, Julia Anna & Kraus, Mathias & Leippold, Markus & Webersinke, Nicolas, 2024. "How cheap talk in climate disclosures relates to climate initiatives, corporate emissions, and reputation risk," Journal of Banking & Finance, Elsevier, vol. 164(C).
    9. Alonso-Robisco, Andres & Carbó, José Manuel, 2023. "Analysis of CBDC narrative by central banks using large language models," Finance Research Letters, Elsevier, vol. 58(PC).
    10. Amel-Zadeh, Amir & Meeks, Geoff, 2019. "Bidder earnings forecasts in mergers and acquisitions," Journal of Corporate Finance, Elsevier, vol. 58(C), pages 373-392.
    11. Durnev, Art & Mangen, Claudine, 2020. "The spillover effects of MD&A disclosures for real investment: The role of industry competition," Journal of Accounting and Economics, Elsevier, vol. 70(1).
    12. Johan, Sofia & Zhang, Yelin, 2020. "Quality revealing versus overstating in equity crowdfunding," Journal of Corporate Finance, Elsevier, vol. 65(C).
    13. Price, S. McKay & Doran, James S. & Peterson, David R. & Bliss, Barbara A., 2012. "Earnings conference calls and stock returns: The incremental informativeness of textual tone," Journal of Banking & Finance, Elsevier, vol. 36(4), pages 992-1011.
    14. De Franco, Gus & Shohfi, Thomas & Xu, Da & Zhu, Zhiwei (Vivi), 2023. "Fixed income conference calls," Journal of Accounting and Economics, Elsevier, vol. 75(1).
    15. Ankur Sinha & Satishwar Kedas & Rishu Kumar & Pekka Malo, 2022. "SEntFiN 1.0: Entity‐aware sentiment analysis for financial news," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(9), pages 1314-1335, September.
    16. Abdollahi, Hooman & Fjesme, Sturla L. & Sirnes, Espen, 2024. "Measuring market volatility connectedness to media sentiment," The North American Journal of Economics and Finance, Elsevier, vol. 71(C).
    17. Saverio Bozzolan & Charles Cho & Giovanna Michelon, 2015. "Impression Management and Organizational Audiences: The Fiat Group Case," Journal of Business Ethics, Springer, vol. 126(1), pages 143-165, January.
    18. Dong, Mengming Michael & Stratopoulos, Theophanis C. & Wang, Victor Xiaoqi, 2024. "A scoping review of ChatGPT research in accounting and finance," International Journal of Accounting Information Systems, Elsevier, vol. 55(C).
    19. Najah Attig & Wenyao Hu & Mohammad M. Rahaman & Ashraf Al Zaman, 2023. "Overselling corporate social responsibility," Financial Management, Financial Management Association International, vol. 52(3), pages 573-610, September.
    20. Olamilekan Shobayo & Sidikat Adeyemi-Longe & Olusogo Popoola & Bayode Ogunleye, 2024. "Innovative Sentiment Analysis and Prediction of Stock Price Using FinBERT, GPT-4 and Logistic Regression: A Data-Driven Approach," Papers 2412.06837, arXiv.org.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:finlet:v:67:y:2024:i:pa:s1544612324008675. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/frl .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.