IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2601.04160.html

All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation Detection

Author

Listed:
  • Yuechen Jiang
  • Zhiwei Liu
  • Yupeng Cao
  • Yueru He
  • Ziyang Xu
  • Chen Xu
  • Zhiyang Deng
  • Prayag Tiwari
  • Xi Chen
  • Alejandro Lopez-Lira
  • Jimin Huang
  • Junichi Tsujii
  • Sophia Ananiadou

Abstract

We introduce RFC Bench, a benchmark for evaluating large language models on financial misinformation under realistic news. RFC Bench operates at the paragraph level and captures the contextual complexity of financial news where meaning emerges from dispersed cues. The benchmark defines two complementary tasks: reference free misinformation detection and comparison based diagnosis using paired original perturbed inputs. Experiments reveal a consistent pattern: performance is substantially stronger when comparative context is available, while reference free settings expose significant weaknesses, including unstable predictions and elevated invalid outputs. These results indicate that current models struggle to maintain coherent belief states without external grounding. By highlighting this gap, RFC Bench provides a structured testbed for studying reference free reasoning and advancing more reliable financial misinformation detection in real world settings.

Suggested Citation

  • Yuechen Jiang & Zhiwei Liu & Yupeng Cao & Yueru He & Ziyang Xu & Chen Xu & Zhiyang Deng & Prayag Tiwari & Xi Chen & Alejandro Lopez-Lira & Jimin Huang & Junichi Tsujii & Sophia Ananiadou, 2026. "All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation Detection," Papers 2601.04160, arXiv.org, revised Jan 2026.
    • Handle: RePEc:arx:papers:2601.04160
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2601.04160
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Weilong Fu, 2025. "The New Quant: A Survey of Large Language Models in Financial Prediction and Trading," Papers 2510.05533, arXiv.org.
    2. Kahan, Dan M. & Peters, Ellen & Dawson, Erica Cantrell & Slovic, Paul, 2017. "Motivated numeracy and enlightened self-government," Behavioural Public Policy, Cambridge University Press, vol. 1(1), pages 54-86, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. William Darity & David McMillon, 2026. "The Matter of White Racism as Self-Sabotage: A Stratification Economics Perspective," NBER Chapters, in: The Economics of Race and Stratification, National Bureau of Economic Research, Inc.
    2. Ali, Ayesha & Qazi, Ihsan Ayyub, 2023. "Countering misinformation on social media through educational interventions: Evidence from a randomized experiment in Pakistan," Journal of Development Economics, Elsevier, vol. 163(C).
    3. Welsch, Heinz, 2021. "How climate-friendly behavior relates to moral identity and identity-protective cognition: Evidence from the European social surveys," Ecological Economics, Elsevier, vol. 185(C).
    4. Vanessa C. Burbano, 2021. "The Demotivating Effects of Communicating a Social-Political Stance: Field Experimental Evidence from an Online Labor Market Platform," Management Science, INFORMS, vol. 67(2), pages 1004-1025, February.
    5. Heinz Welsch, 2022. "Correction to: What shapes cognitions of climate change in Europe? Ideology, morality, and the role of educational attainment," Journal of Environmental Studies and Sciences, Springer;Association of Environmental Studies and Sciences, vol. 12(2), pages 396-397, June.
    6. Brandts, Jordi & Busom, Isabel & Lopez-Mayan, Cristina & Panadés, Judith, 2022. "Dispelling misconceptions about economics," Journal of Economic Psychology, Elsevier, vol. 88(C).
    7. 'Alvaro Romaniega, 2021. "On the probability of the Condorcet Jury Theorem or the Miracle of Aggregation," Papers 2108.00733, arXiv.org, revised Jun 2022.
    8. Timmons, Shane & Lunn, Pete, 2022. "Public understanding of climate change and support for mitigation," Research Series, Economic and Social Research Institute (ESRI), number RS135.
    9. Mohamed Mostagir & James Siderius, 2022. "Learning in a Post-Truth World," Management Science, INFORMS, vol. 68(4), pages 2860-2868, April.
    10. Farhart, Christina E. & Struby, Ethan, 2026. "Inflation expectations and political polarization: Evidence from the cooperative election study," Journal of Macroeconomics, Elsevier, vol. 87(C).
    11. Carr-Harris, Andrew & Lang, Corey, 2019. "Sustainability and tourism: the effect of the United States’ first offshore wind farm on the vacation rental market," Resource and Energy Economics, Elsevier, vol. 57(C), pages 51-67.
    12. Sheheryar Banuri & Stefan Dercon & Varun Gauri, 2019. "Biased Policy Professionals," The World Bank Economic Review, World Bank, vol. 33(2), pages 310-327.
    13. Adrian Kwek & Luke Peh & Josef Tan & Jin Xing Lee, 2023. "Distractions, analytical thinking and falling for fake news: A survey of psychological factors," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 10(1), pages 1-12, December.
    14. Robert M. Ross & David G. Rand & Gordon Pennycook, 2021. "Beyond “fake news†: Analytic thinking and the detection of false and hyperpartisan news headlines," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 16(2), pages 484-504, March.
    15. Meifen Wu & Ruyin Long & Shuhan Yang & Xinru Wang & Hong Chen, 2022. "Evolution of the Knowledge Mapping of Climate Change Communication Research: Basic Status, Research Hotspots, and Prospects," IJERPH, MDPI, vol. 19(18), pages 1-18, September.
    16. Becky L. Choma & David Sumantry & Yaniv Hanoch, 2019. "Right-wing ideology and numeracy: A perception of greater ability, but poorer performance," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 14(4), pages 412-422, July.
    17. Luis Pérez-González, 2020. "‘Is climate science taking over the science?’: A corpus-based study of competing stances on bias, dogma and expertise in the blogosphere," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 7(1), pages 1-16, December.
    18. Robin Bayes, 2022. "Moral Convictions and Threats to Science," The ANNALS of the American Academy of Political and Social Science, , vol. 700(1), pages 86-96, March.
    19. Abraham Aldama & Cristina Bicchieri & Jana Freundt & Barbara Mellers & Ellen Peters, 2021. "How perceptions of autonomy relate to beliefs about inequality and fairness," PLOS ONE, Public Library of Science, vol. 16(1), pages 1-16, January.
    20. M. Aenne Schoop & Marco Verweij & Ulrich Kühnen & Shenghua Luan, 2020. "Political disagreement in the classroom: testing cultural theory through structured observation," Quality & Quantity: International Journal of Methodology, Springer, vol. 54(2), pages 623-643, April.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2601.04160. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.