IDEAS home Printed from https://ideas.repec.org/p/hal/journl/hal-03184841.html
   My bibliography  Save this paper

Recurrent Dictionary Learning for State-Space Models with an Application in Stock Forecasting

Author

Listed:
  • Shalini Sharma

    (IIIT-Delhi - Indraprastha Institute of Information Technology [New Delhi])

  • Víctor Elvira

    (School of Mathematics - University of Edinburgh - Edin. - University of Edinburgh)

  • Emilie Chouzenoux

    (OPIS - OPtimisation Imagerie et Santé - Inria Saclay - Ile de France - Inria - Institut National de Recherche en Informatique et en Automatique - CVN - Centre de vision numérique - Inria - Institut National de Recherche en Informatique et en Automatique - CentraleSupélec - Université Paris-Saclay)

  • Angshul Majumdar

    (IIIT-Delhi - Indraprastha Institute of Information Technology [New Delhi])

Abstract

In this work, we introduce a new modeling and inferential tool for dynamical processing of time series. The approach is called recurrent dictionary learning (RDL). The proposed model reads as a linear Gaussian Markovian state-space model involving two linear operators, the state evolution and the observation matrices, that we assumed to be unknown. These two unknown operators (that can be seen interpreted as dictionaries) and the sequence of hidden states are jointly learnt via an expectation-maximization algorithm. The RDL model gathers several advantages, namely online processing, probabilistic inference, and a high model expressiveness which is usually typical of neural networks. RDL is particularly well suited for stock forecasting. Its performance is illustrated on two problems: next day forecasting (regression problem) and next day trading (classification problem), given past stock market observations. Experimental results show that our proposed method excels over state-of-the-art stock analysis models such as CNN-TA, MFNN, and LSTM.

Suggested Citation

  • Shalini Sharma & Víctor Elvira & Emilie Chouzenoux & Angshul Majumdar, 2021. "Recurrent Dictionary Learning for State-Space Models with an Application in Stock Forecasting," Post-Print hal-03184841, HAL.
  • Handle: RePEc:hal:journl:hal-03184841
    Note: View the original document on HAL open archive server: https://hal.science/hal-03184841
    as

    Download full text from publisher

    File URL: https://hal.science/hal-03184841/document
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ng, Jason & Forbes, Catherine S. & Martin, Gael M. & McCabe, Brendan P.M., 2013. "Non-parametric estimation of forecast distributions in non-Gaussian, non-linear state space models," International Journal of Forecasting, Elsevier, vol. 29(3), pages 411-430.
    2. repec:dau:papers:123456789/6830 is not listed on IDEAS
    3. Neha Saini & Anil Kumar Mittal, 2014. "Forecasting Volatility in Indian Stock Market using State Space Models," Journal of Statistical and Econometric Methods, SCIENPRESS Ltd, vol. 3(1), pages 1-8.
    4. repec:cup:cbooks:9780521835954 is not listed on IDEAS
    5. repec:dau:papers:123456789/871 is not listed on IDEAS
    6. R. H. Shumway & D. S. Stoffer, 1982. "An Approach To Time Series Smoothing And Forecasting Using The Em Algorithm," Journal of Time Series Analysis, Wiley Blackwell, vol. 3(4), pages 253-264, July.
    7. Harvey,Andrew C., 1991. "Forecasting, Structural Time Series Models and the Kalman Filter," Cambridge Books, Cambridge University Press, number 9780521405737, October.
    8. Pai, Ping-Feng & Lin, Chih-Sheng, 2005. "A hybrid ARIMA and support vector machines model in stock price forecasting," Omega, Elsevier, vol. 33(6), pages 497-505, December.
    9. Stephany Griffith-Jones & Stephen Spratt, 2003. "Basel II and Developing Countries: Diversification and Portfolio Effects," FMG Discussion Papers dp437, Financial Markets Group.
    10. Andreea-Cristina PETRICĂ & Stelian STANCU & Alexandru TINDECHE, 2016. "Limitation of ARIMA models in financial and monetary economics," Theoretical and Applied Economics, Asociatia Generala a Economistilor din Romania / Editura Economica, vol. 0(4(609), W), pages 19-42, Winter.
    11. Griffith-Jones, Stephany & Segoviano, Miguel Angel & Spratt, Stephen, 2003. "Basel II and developing countries: diversification and portfolio effects," LSE Research Online Documents on Economics 24824, London School of Economics and Political Science, LSE Library.
    12. Fischer, Thomas & Krauss, Christopher, 2018. "Deep learning with long short-term memory networks for financial market predictions," European Journal of Operational Research, Elsevier, vol. 270(2), pages 654-669.
    13. Pandher, Gurupdesh S, 2002. "Forecasting Multivariate Time Series with Linear Restrictions Using Constrained Structural State-Space Models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 21(4), pages 281-300, July.
    14. Smith, J.Q. & Santos, Antonio A.F., 2006. "Second-Order Filter Distribution Approximations for Financial Time Series With Extreme Outliers," Journal of Business & Economic Statistics, American Statistical Association, vol. 24, pages 329-337, July.
    15. Manish Kumar & M. Thenmozhi, 2014. "Forecasting stock index returns using ARIMA-SVM, ARIMA-ANN, and ARIMA-random forest hybrid models," International Journal of Banking, Accounting and Finance, Inderscience Enterprises Ltd, vol. 5(3), pages 284-308.
    16. Joseph McCarthy & Mohammad Najand, 1993. "State space modeling of price and volume dependence: Evidence from currency futures," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 13(4), pages 335-344, June.
    17. Delphine Lautier & Alireza Javaheri & Alain Galli, 2003. "Filtering in finance," Post-Print halshs-00153006, HAL.
    18. Andreea-Cristina PETRICĂ & Stelian STANCU & Alexandru TINDECHE, 2016. "Limitation of ARIMA models in financial and monetary economics," Theoretical and Applied Economics, Asociatia Generala a Economistilor din Romania - AGER, vol. 0(4(609), W), pages 19-42, Winter.
    19. Fama, Eugene F, 1970. "Efficient Capital Markets: A Review of Theory and Empirical Work," Journal of Finance, American Finance Association, vol. 25(2), pages 383-417, May.
    20. Roberto Casarin & Carmine Trecroci, 2006. "Business Cycle and Stock Market Volatility: A Particle Filter Approach," Working Papers ubs0603, University of Brescia, Department of Economics.
    21. Dev Shah & Haruna Isah & Farhana Zulkernine, 2019. "Stock Market Analysis: A Review and Taxonomy of Prediction Techniques," IJFS, MDPI, vol. 7(2), pages 1-22, May.
    22. Masaya Abe & Hideki Nakayama, 2018. "Deep Learning for Forecasting Stock Returns in the Cross-Section," Papers 1801.01777, arXiv.org, revised Jun 2018.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shalini Sharma & Angshul Majumdar & Emilie Chouzenoux & Victor Elvira, 2023. "Deep State-Space Model for Predicting Cryptocurrency Price," Papers 2311.14731, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kamaladdin Fataliyev & Aneesh Chivukula & Mukesh Prasad & Wei Liu, 2021. "Stock Market Analysis with Text Data: A Review," Papers 2106.12985, arXiv.org, revised Jul 2021.
    2. Frédy Pokou & Jules Sadefo Kamdem & François Benhmad, 2024. "Hybridization of ARIMA with Learning Models for Forecasting of Stock Market Time Series," Computational Economics, Springer;Society for Computational Economics, vol. 63(4), pages 1349-1399, April.
    3. Tobias Hartl & Roland Jucknewitz, 2022. "Approximate state space modelling of unobserved fractional components," Econometric Reviews, Taylor & Francis Journals, vol. 41(1), pages 75-98, January.
    4. Kentaro Imajo & Kentaro Minami & Katsuya Ito & Kei Nakagawa, 2020. "Deep Portfolio Optimization via Distributional Prediction of Residual Factors," Papers 2012.07245, arXiv.org.
    5. Bartosz Bieganowski & Robert Ślepaczuk, 2024. "Supervised Autoencoder MLP for Financial Time Series Forecasting," Working Papers 2024-03, Faculty of Economic Sciences, University of Warsaw.
    6. Hakan Pabuccu & Adrian Barbu, 2023. "Feature Selection with Annealing for Forecasting Financial Time Series," Papers 2303.02223, arXiv.org, revised Feb 2024.
    7. Alexander Jakob Dautel & Wolfgang Karl Härdle & Stefan Lessmann & Hsin-Vonn Seow, 2020. "Forex exchange rate forecasting using deep recurrent neural networks," Digital Finance, Springer, vol. 2(1), pages 69-96, September.
    8. Manahov, Viktor & Hudson, Robert & Gebka, Bartosz, 2014. "Does high frequency trading affect technical analysis and market efficiency? And if so, how?," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 28(C), pages 131-157.
    9. Ehsan Hoseinzade & Saman Haratizadeh & Arash Khoeini, 2019. "U-CNNpred: A Universal CNN-based Predictor for Stock Markets," Papers 1911.12540, arXiv.org.
    10. Iris Chen & Yogeshwar D Kelkar & Yu Gu & Jie Zhou & Xing Qiu & Hulin Wu, 2017. "High-dimensional linear state space models for dynamic microbial interaction networks," PLOS ONE, Public Library of Science, vol. 12(11), pages 1-20, November.
    11. Rzayev, Khaladdin & Ibikunle, Gbenga, 2019. "A state-space modeling of the information content of trading volume," Journal of Financial Markets, Elsevier, vol. 46(C).
    12. Fabian Waldow & Matthias Schnaubelt & Christopher Krauss & Thomas Günter Fischer, 2021. "Machine Learning in Futures Markets," JRFM, MDPI, vol. 14(3), pages 1-14, March.
    13. Kizilaslan, Recep & Freund, Steven & Iseri, Ali, 2016. "A data analytic approach to forecasting daily stock returns in an emerging marketAuthor-Name: Oztekin, Asil," European Journal of Operational Research, Elsevier, vol. 253(3), pages 697-710.
    14. Chlebus Marcin & Dyczko Michał & Woźniak Michał, 2021. "Nvidia's Stock Returns Prediction Using Machine Learning Techniques for Time Series Forecasting Problem," Central European Economic Journal, Sciendo, vol. 8(55), pages 44-62, January.
    15. Nikolaos Zirogiannis & Yorghos Tripodis, 2013. "A Generalized Dynamic Factor Model for Panel Data: Estimation with a Two-Cycle Conditional Expectation-Maximization Algorithm," Working Papers 2013-1, University of Massachusetts Amherst, Department of Resource Economics.
    16. Omer Berat Sezer & Mehmet Ugur Gudelek & Ahmet Murat Ozbayoglu, 2019. "Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019," Papers 1911.13288, arXiv.org.
    17. Söhnke M. Bartram & Jürgen Branke & Mehrshad Motahari, 2020. "Artificial intelligence in asset management," Working Papers 20202001, Cambridge Judge Business School, University of Cambridge.
    18. Green, Lawrence & Sung, Ming-Chien & Ma, Tiejun & Johnson, Johnnie E. V., 2019. "To what extent can new web-based technology improve forecasts? Assessing the economic value of information derived from Virtual Globes and its rate of diffusion in a financial market," European Journal of Operational Research, Elsevier, vol. 278(1), pages 226-239.
    19. Keer Yang & Guanqun Zhang & Chuan Bi & Qiang Guan & Hailu Xu & Shuai Xu, 2023. "Improving CNN-base Stock Trading By Considering Data Heterogeneity and Burst," Papers 2303.09407, arXiv.org.
    20. Jasleen Kaur & Khushdeep Dharni, 2022. "Application and performance of data mining techniques in stock market: A review," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 29(4), pages 219-241, October.

    More about this item

    Keywords

    Stock Forecasting; Recurrent dictionary learning; Kalman filter; expectation-minimization; dynamical modeling; uncertainty quantification;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:hal-03184841. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.