IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2311.13326.html
   My bibliography  Save this paper

Curriculum Learning and Imitation Learning for Model-free Control on Financial Time-series

Author

Listed:
  • Woosung Koh
  • Insu Choi
  • Yuntae Jang
  • Gimin Kang
  • Woo Chang Kim

Abstract

Curriculum learning and imitation learning have been leveraged extensively in the robotics domain. However, minimal research has been done on leveraging these ideas on control tasks over highly stochastic time-series data. Here, we theoretically and empirically explore these approaches in a representative control task over complex time-series data. We implement the fundamental ideas of curriculum learning via data augmentation, while imitation learning is implemented via policy distillation from an oracle. Our findings reveal that curriculum learning should be considered a novel direction in improving control-task performance over complex time-series. Our ample random-seed out-sample empirics and ablation studies are highly encouraging for curriculum learning for time-series control. These findings are especially encouraging as we tune all overlapping hyperparameters on the baseline -- giving an advantage to the baseline. On the other hand, we find that imitation learning should be used with caution.

Suggested Citation

  • Woosung Koh & Insu Choi & Yuntae Jang & Gimin Kang & Woo Chang Kim, 2023. "Curriculum Learning and Imitation Learning for Model-free Control on Financial Time-series," Papers 2311.13326, arXiv.org, revised Jan 2024.
  • Handle: RePEc:arx:papers:2311.13326
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2311.13326
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ben Hambly & Renyuan Xu & Huining Yang, 2021. "Recent Advances in Reinforcement Learning in Finance," Papers 2112.04553, arXiv.org, revised Feb 2023.
    2. Ben Hambly & Renyuan Xu & Huining Yang, 2023. "Recent advances in reinforcement learning in finance," Mathematical Finance, Wiley Blackwell, vol. 33(3), pages 437-503, July.
    3. Yunan Ye & Hengzhi Pei & Boxin Wang & Pin-Yu Chen & Yada Zhu & Jun Xiao & Bo Li, 2020. "Reinforcement-Learning based Portfolio Management with Augmented Asset Movement Prediction States," Papers 2002.05780, arXiv.org.
    4. Thomas M. Cover, 1991. "Universal Portfolios," Mathematical Finance, Wiley Blackwell, vol. 1(1), pages 1-29, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shuo Sun & Rundong Wang & Bo An, 2021. "Reinforcement Learning for Quantitative Trading," Papers 2109.13851, arXiv.org.
    2. Xiangyu Cui & Xun Li & Yun Shi & Si Zhao, 2023. "Discrete-Time Mean-Variance Strategy Based on Reinforcement Learning," Papers 2312.15385, arXiv.org.
    3. Xianhua Peng & Chenyin Gong & Xue Dong He, 2023. "Reinforcement Learning for Financial Index Tracking," Papers 2308.02820, arXiv.org.
    4. Reilly Pickard & Yuri Lawryshyn, 2023. "Deep Reinforcement Learning for Dynamic Stock Option Hedging: A Review," Mathematics, MDPI, vol. 11(24), pages 1-19, December.
    5. Foster, Dean P. & Vohra, Rakesh, 1999. "Regret in the On-Line Decision Problem," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 7-35, October.
    6. Eric Benhamou & David Saltiel & Sandrine Ungari & Abhishek Mukhopadhyay & Jamal Atif, 2020. "AAMDRL: Augmented Asset Management with Deep Reinforcement Learning," Papers 2010.08497, arXiv.org.
    7. Frensi Zejnullahu & Maurice Moser & Joerg Osterrieder, 2022. "Applications of Reinforcement Learning in Finance -- Trading with a Double Deep Q-Network," Papers 2206.14267, arXiv.org.
    8. Seung-Hyun Moon & Yong-Hyuk Kim & Byung-Ro Moon, 2019. "Empirical investigation of state-of-the-art mean reversion strategies for equity markets," Papers 1909.04327, arXiv.org.
    9. Man Yiu Tsang & Tony Sit & Hoi Ying Wong, 2022. "Adaptive Robust Online Portfolio Selection," Papers 2206.01064, arXiv.org.
    10. Luo, Yong & Zhu, Bo & Tang, Yong, 2014. "Simulated annealing algorithm for optimal capital growth," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 408(C), pages 10-18.
    11. Fabio Stella & Alfonso Ventura, 2011. "Defensive online portfolio selection," International Journal of Financial Markets and Derivatives, Inderscience Enterprises Ltd, vol. 2(1/2), pages 88-105.
    12. Erhan Bayraktar & Donghan Kim & Abhishek Tilva, 2023. "Quantifying dimensional change in stochastic portfolio theory," Papers 2303.00858, arXiv.org, revised Apr 2023.
    13. Ormos, Mihály & Urbán, András & Zoltán, Tamás, 2009. "Logoptimális portfóliók empirikus vizsgálata [Empirical analysis of log-optimal portfolios]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(1), pages 1-18.
    14. Jarrod Wilcox, 2020. "Better portfolios with higher moments," Journal of Asset Management, Palgrave Macmillan, vol. 21(7), pages 569-580, December.
    15. Dmitry B. Rokhlin, 2021. "Relative utility bounds for empirically optimal portfolios," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 93(3), pages 437-462, June.
    16. Vladimir V'yugin, 2014. "Log-Optimal Portfolio Selection Using the Blackwell Approachability Theorem," Papers 1410.5996, arXiv.org, revised Jun 2015.
    17. R'emi J'ez'equel & Dmitrii M. Ostrovskii & Pierre Gaillard, 2022. "Efficient and Near-Optimal Online Portfolio Selection," Papers 2209.13932, arXiv.org.
    18. Miquel Noguer i Alonso & Sonam Srivastava, 2020. "Deep Reinforcement Learning for Asset Allocation in US Equities," Papers 2010.04404, arXiv.org.
    19. Malo Huard & Rémy Garnier & Gilles Stoltz, 2020. "Hierarchical robust aggregation of sales forecasts at aggregated levels in e-commerce, based on exponential smoothing and Holt's linear trend method," Working Papers hal-02794320, HAL.
    20. Christopher Dance & Alexei Gaivoronski, 2012. "Stochastic optimization for real time service capacity allocation under random service demand," Annals of Operations Research, Springer, vol. 193(1), pages 221-253, March.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2311.13326. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.