IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2606.08285.html

Beyond Agent Architecture: Execution Assumptions and Reproducibility in LLM-Based Trading Systems

Author

Listed:
  • Junyi Yao
  • Zihao Zheng

Abstract

Large language models (LLMs) and agentic systems are increasingly proposed for financial trading, yet their reported performance remains difficult to compare because studies vary in data provenance, temporal split discipline, execution timing, turnover treatment, and transaction-cost modeling. This article presents a targeted topical review and reproducibility audit of execution realism in LLM-based trading research. A coded evidence matrix covering 30 trade-relevant primary studies is used to assess point-in-time controls, split transparency, held-out evaluation, cost and turnover treatment, execution semantics, universe definition, and artifact release. Across the audited sample, architecture reporting is generally clearer than the evaluation assumptions needed to judge whether a trading result is economically interpretable or reproducible. A 10-equity worked example is included only as a methodological scaffold to illustrate how explicit friction and timing choices can materially compress active-strategy results. The main conclusion is that the next useful step for LLM trading research is not only better agent design, but also clearer reporting standards for execution realism, reproducibility, and evaluation comparability.

Suggested Citation

  • Junyi Yao & Zihao Zheng, 2026. "Beyond Agent Architecture: Execution Assumptions and Reproducibility in LLM-Based Trading Systems," Papers 2606.08285, arXiv.org.
  • Handle: RePEc:arx:papers:2606.08285
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2606.08285
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Haohang Li & Yupeng Cao & Yangyang Yu & Shashidhar Reddy Javaji & Zhiyang Deng & Yueru He & Yuechen Jiang & Zining Zhu & Koduvayur Subbalakshmi & Guojun Xiong & Jimin Huang & Lingfei Qian & Xueqing Pe, 2024. "INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent," Papers 2412.18174, arXiv.org.
    2. Terence Lim & Kumar Muthuraman & Michael Sury, 2026. "QRAFTI: An Agentic Framework for Empirical Research in Quantitative Finance," Papers 2604.18500, arXiv.org.
    3. Thanos Konstantinidis & Giorgos Iacovides & Mingxue Xu & Tony G. Constantinides & Danilo Mandic, 2024. "FinLlama: Financial Sentiment Classification for Algorithmic Trading Applications," Papers 2403.12285, arXiv.org.
    4. Alejandro Lopez-Lira, 2025. "Can Large Language Models Trade? Testing Financial Theories with LLM Agents in Market Simulations," Papers 2504.10789, arXiv.org.
    5. Dat Mai, 2024. "StockGPT: A GenAI Model for Stock Prediction and Trading," Papers 2404.05101, arXiv.org, revised Oct 2024.
    6. Saizhuo Wang & Hang Yuan & Lionel M. Ni & Jian Guo, 2024. "QuantAgent: Seeking Holy Grail in Trading by Self-Improving Large Language Model," Papers 2402.03755, arXiv.org.
    7. Chong Zhang & Xinyi Liu & Zhongmou Zhang & Mingyu Jin & Lingyao Li & Zhenting Wang & Wenyue Hua & Dong Shu & Suiyuan Zhu & Xiaobo Jin & Sujian Li & Mengnan Du & Yongfeng Zhang, 2024. "When AI Meets Finance (StockAgent): Large Language Model-based Stock Trading in Simulated Real-world Environments," Papers 2407.18957, arXiv.org, revised Jun 2026.
    8. Yuxuan Zhao & Sijia Chen & Ningxin Su, 2026. "PortBench: A Correlation-Aware, Full-Pipeline Benchmark for LLM-Driven Portfolio Management," Papers 2605.27887, arXiv.org, revised Jun 2026.
    9. Jean Lee & Nicholas Stevens & Soyeon Caren Han & Minseok Song, 2024. "A Survey of Large Language Models in Finance (FinLLMs)," Papers 2402.02315, arXiv.org.
    10. Adam Darmanin & Vince Vella, 2025. "Language Model Guided Reinforcement Learning in Quantitative Trading," Papers 2508.02366, arXiv.org, revised Oct 2025.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Xavier Fonseca, 2026. "Look-Ahead-Freedom as Temporal Non-Interference: A Verifiable Correctness Property for Backtesting and Agentic Trading Pipelines," Papers 2607.04958, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Han Ding & Yinheng Li & Junhao Wang & Hang Chen & Doudou Guo & Yunbai Zhang, 2024. "Large Language Model Agent in Financial Trading: A Survey," Papers 2408.06361, arXiv.org, revised Mar 2026.
    2. Zhi Yang & Lingfeng Zeng & Fangqi Lou & Qi Qi & Wei Zhang & Zhenyu Wu & Zhenxiong Yu & Jun Han & Zhiheng Jin & Lejie Zhang & Xiaoming Huang & Xiaolong Liang & Zheng Wei & Junbo Zou & Dongpo Cheng & Zh, 2026. "UniFinEval: Towards Unified Evaluation of Financial Multimodal Models across Text, Images and Videos," Papers 2601.22162, arXiv.org.
    3. Mostapha Benhenda, 2026. "Look-Ahead-Bench: a Standardized Benchmark of Look-ahead Bias in Point-in-Time LLMs for Finance," Papers 2601.13770, arXiv.org.
    4. Benjamin Coriat & Eric Benhamou, 2025. "HARLF: Hierarchical Reinforcement Learning and Lightweight LLM-Driven Sentiment Integration for Financial Portfolio Optimization," Papers 2507.18560, arXiv.org.
    5. Liyuan Chen & Shuoling Liu & Jiangpeng Yan & Xiaoyu Wang & Henglin Liu & Chuang Li & Kecheng Jiao & Jixuan Ying & Yang Veronica Liu & Qiang Yang & Xiu Li, 2025. "Advancing Financial Engineering with Foundation Models: Progress, Applications, and Challenges," Papers 2507.18577, arXiv.org, revised Dec 2025.
    6. Tirulo, Aschalew & Yadav, Monika & Lolamo, Mathewos & Chauhan, Siddhartha & Siano, Pierluigi & Shafie-khah, Miadreza, 2026. "Beyond automation: Unveiling the potential of agentic intelligence," Renewable and Sustainable Energy Reviews, Elsevier, vol. 226(PA).
    7. Rubén Fernández-Fuertes, 2025. "Monetary Policy Shocks: A New Hope. Large Language Models and Central Bank Communication," BAFFI CAREFIN Working Papers 25257, BAFFI CAREFIN, Centre for Applied Research on International Markets Banking Finance and Regulation, Universita' Bocconi, Milano, Italy.
    8. Hamidou Tembine & Manzoor Ahmed Khan & Issa Bamia, 2024. "Mean-Field-Type Transformers," Mathematics, MDPI, vol. 12(22), pages 1-51, November.
    9. Matthew Francis Dixon, 2026. "Model Validation of Agentic AI Systems: A POMDP-Based Framework for Belief-State, Forecast, and Policy Validation," Papers 2606.17383, arXiv.org.
    10. Zichen Chen & Jiaao Chen & Jianda Chen & Misha Sra, 2025. "Standard Benchmarks Fail -- Auditing LLM Agents in Finance Must Prioritize Risk," Papers 2502.15865, arXiv.org, revised Jun 2025.
    11. Kassiani Papasotiriou & Srijan Sood & Shayleen Reynolds & Tucker Balch, 2024. "AI in Investment Analysis: LLMs for Equity Stock Ratings," Papers 2411.00856, arXiv.org.
    12. Zhenyu Gao & Wenxi Jiang & Yutong Yan, 2026. "Debiasing LLMs by Fine-tuning," Papers 2604.02921, arXiv.org, revised May 2026.
    13. Junhua Liu, 2024. "A Survey of Financial AI: Architectures, Advances and Open Challenges," Papers 2411.12747, arXiv.org.
    14. Filippo Gusella & Eugenio Vicario, 2025. "Generative Agents and Expectations: Do LLMs Align with Heterogeneous Agent Models?," Working Papers - Economics wp2025_18.rdf, Universita' degli Studi di Firenze, Dipartimento di Scienze per l'Economia e l'Impresa.
    15. Aadi Singhi, 2025. "An Adaptive Multi Agent Bitcoin Trading System," Papers 2510.08068, arXiv.org, revised Nov 2025.
    16. Simeon Allmendinger & Lukas Bonenberger & Kathrin Endres & Dominik Fetzer & Henner Gimpel & Niklas Kühl, 2026. "Multi-agent AI," Electronic Markets, Springer;IIM University of St. Gallen, vol. 36(1), pages 1-18, December.
    17. Ryuji Hashimoto & Takehiro Takayanagi & Masahiro Suzuki & Kiyoshi Izumi, 2026. "LLM agents reveal how human bias shapes path-dependent market dynamics," Journal of Computational Social Science, Springer, vol. 9(2), pages 1-26, May.
    18. Haoyi Zhang & Tianyi Zhu, 2025. "Neither Consent nor Property: A Policy Lab for Data Law," Papers 2510.26727, arXiv.org, revised Apr 2026.
    19. Anne Lundgaard Hansen & Seung Jung Lee, 2025. "Financial Stability Implications of Generative AI: Taming the Animal Spirits," Papers 2510.01451, arXiv.org.
    20. Antonino Castelli & Paolo Giudici & Alessandro Piergallini, 2025. "Building crypto portfolios with agentic AI," Papers 2507.20468, arXiv.org.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2606.08285. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.