IDEAS home Printed from https://ideas.repec.org/p/hal/journl/hal-03184841.html
   My bibliography  Save this paper

Recurrent Dictionary Learning for State-Space Models with an Application in Stock Forecasting

Author

Listed:
  • Shalini Sharma

    (IIIT-Delhi - Indraprastha Institute of Information Technology [New Delhi])

  • Víctor Elvira

    (School of Mathematics - University of Edinburgh - Edin. - University of Edinburgh)

  • Emilie Chouzenoux

    (OPIS - OPtimisation Imagerie et Santé - Inria Saclay - Ile de France - Inria - Institut National de Recherche en Informatique et en Automatique - CVN - Centre de vision numérique - Inria - Institut National de Recherche en Informatique et en Automatique - CentraleSupélec - Université Paris-Saclay)

  • Angshul Majumdar

    (IIIT-Delhi - Indraprastha Institute of Information Technology [New Delhi])

Abstract

In this work, we introduce a new modeling and inferential tool for dynamical processing of time series. The approach is called recurrent dictionary learning (RDL). The proposed model reads as a linear Gaussian Markovian state-space model involving two linear operators, the state evolution and the observation matrices, that we assumed to be unknown. These two unknown operators (that can be seen interpreted as dictionaries) and the sequence of hidden states are jointly learnt via an expectation-maximization algorithm. The RDL model gathers several advantages, namely online processing, probabilistic inference, and a high model expressiveness which is usually typical of neural networks. RDL is particularly well suited for stock forecasting. Its performance is illustrated on two problems: next day forecasting (regression problem) and next day trading (classification problem), given past stock market observations. Experimental results show that our proposed method excels over state-of-the-art stock analysis models such as CNN-TA, MFNN, and LSTM.

Suggested Citation

  • Shalini Sharma & Víctor Elvira & Emilie Chouzenoux & Angshul Majumdar, 2021. "Recurrent Dictionary Learning for State-Space Models with an Application in Stock Forecasting," Post-Print hal-03184841, HAL.
  • Handle: RePEc:hal:journl:hal-03184841
    Note: View the original document on HAL open archive server: https://hal.science/hal-03184841
    as

    Download full text from publisher

    File URL: https://hal.science/hal-03184841/document
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ng, Jason & Forbes, Catherine S. & Martin, Gael M. & McCabe, Brendan P.M., 2013. "Non-parametric estimation of forecast distributions in non-Gaussian, non-linear state space models," International Journal of Forecasting, Elsevier, vol. 29(3), pages 411-430.
    2. Neha Saini & Anil Kumar Mittal, 2014. "Forecasting Volatility in Indian Stock Market using State Space Models," Journal of Statistical and Econometric Methods, SCIENPRESS Ltd, vol. 3(1), pages 1-8.
    3. repec:dau:papers:123456789/871 is not listed on IDEAS
    4. R. H. Shumway & D. S. Stoffer, 1982. "An Approach To Time Series Smoothing And Forecasting Using The Em Algorithm," Journal of Time Series Analysis, Wiley Blackwell, vol. 3(4), pages 253-264, July.
    5. Harvey,Andrew C., 1991. "Forecasting, Structural Time Series Models and the Kalman Filter," Cambridge Books, Cambridge University Press, number 9780521405737.
    6. Stephany Griffith-Jones & Stephen Spratt, 2003. "Basel II and Developing Countries: Diversification and Portfolio Effects," FMG Discussion Papers dp437, Financial Markets Group.
    7. Smith, J.Q. & Santos, Antonio A.F., 2006. "Second-Order Filter Distribution Approximations for Financial Time Series With Extreme Outliers," Journal of Business & Economic Statistics, American Statistical Association, vol. 24, pages 329-337, July.
    8. Manish Kumar & M. Thenmozhi, 2014. "Forecasting stock index returns using ARIMA-SVM, ARIMA-ANN, and ARIMA-random forest hybrid models," International Journal of Banking, Accounting and Finance, Inderscience Enterprises Ltd, vol. 5(3), pages 284-308.
    9. Joseph McCarthy & Mohammad Najand, 1993. "State space modeling of price and volume dependence: Evidence from currency futures," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 13(4), pages 335-344, June.
    10. Fama, Eugene F, 1970. "Efficient Capital Markets: A Review of Theory and Empirical Work," Journal of Finance, American Finance Association, vol. 25(2), pages 383-417, May.
    11. repec:dau:papers:123456789/6830 is not listed on IDEAS
    12. Harvey,Andrew & Koopman,Siem Jan & Shephard,Neil (ed.), 2004. "State Space and Unobserved Component Models," Cambridge Books, Cambridge University Press, number 9780521835954.
    13. Pai, Ping-Feng & Lin, Chih-Sheng, 2005. "A hybrid ARIMA and support vector machines model in stock price forecasting," Omega, Elsevier, vol. 33(6), pages 497-505, December.
    14. Andreea-Cristina PETRICĂ & Stelian STANCU & Alexandru TINDECHE, 2016. "Limitation of ARIMA models in financial and monetary economics," Theoretical and Applied Economics, Asociatia Generala a Economistilor din Romania - AGER, vol. 0(4(609), W), pages 19-42, Winter.
    15. Griffith-Jones, Stephany & Segoviano, Miguel Angel & Spratt, Stephen, 2003. "Basel II and developing countries: diversification and portfolio effects," LSE Research Online Documents on Economics 24824, London School of Economics and Political Science, LSE Library.
    16. Fischer, Thomas & Krauss, Christopher, 2018. "Deep learning with long short-term memory networks for financial market predictions," European Journal of Operational Research, Elsevier, vol. 270(2), pages 654-669.
    17. Pandher, Gurupdesh S, 2002. "Forecasting Multivariate Time Series with Linear Restrictions Using Constrained Structural State-Space Models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 21(4), pages 281-300, July.
    18. Delphine Lautier & Alireza Javaheri & Alain Galli, 2003. "Filtering in finance," Post-Print halshs-00153006, HAL.
    19. Andreea-Cristina PETRICĂ & Stelian STANCU & Alexandru TINDECHE, 2016. "Limitation of ARIMA models in financial and monetary economics," Theoretical and Applied Economics, Asociatia Generala a Economistilor din Romania - AGER, vol. 0(4(609), W), pages 19-42, Winter.
    20. Roberto Casarin & Carmine Trecroci, 2006. "Business Cycle and Stock Market Volatility: A Particle Filter Approach," Working Papers ubs0603, University of Brescia, Department of Economics.
    21. Dev Shah & Haruna Isah & Farhana Zulkernine, 2019. "Stock Market Analysis: A Review and Taxonomy of Prediction Techniques," IJFS, MDPI, vol. 7(2), pages 1-22, May.
    22. Masaya Abe & Hideki Nakayama, 2018. "Deep Learning for Forecasting Stock Returns in the Cross-Section," Papers 1801.01777, arXiv.org, revised Jun 2018.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shalini Sharma & Angshul Majumdar & Emilie Chouzenoux & Victor Elvira, 2023. "Deep State-Space Model for Predicting Cryptocurrency Price," Papers 2311.14731, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kamaladdin Fataliyev & Aneesh Chivukula & Mukesh Prasad & Wei Liu, 2021. "Stock Market Analysis with Text Data: A Review," Papers 2106.12985, arXiv.org, revised Jul 2021.
    2. Zirogiannis, Nikolaos & Tripodis, Yorghos, 2013. "A Generalized Dynamic Factor Model for Panel Data: Estimation with a Two-Cycle Conditional Expectation-Maximization Algorithm," Working Paper Series 142752, University of Massachusetts, Amherst, Department of Resource Economics.
    3. Tobias Hartl & Roland Jucknewitz, 2022. "Approximate state space modelling of unobserved fractional components," Econometric Reviews, Taylor & Francis Journals, vol. 41(1), pages 75-98, January.
    4. Saqib Farid & Rubeena Tashfeen & Tahseen Mohsan & Arsal Burhan, 2023. "Forecasting stock prices using a data mining method: Evidence from emerging market," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 28(2), pages 1911-1917, April.
    5. Kentaro Imajo & Kentaro Minami & Katsuya Ito & Kei Nakagawa, 2020. "Deep Portfolio Optimization via Distributional Prediction of Residual Factors," Papers 2012.07245, arXiv.org.
    6. Alexander Tsyplakov, 2011. "An introduction to state space modeling (in Russian)," Quantile, Quantile, issue 9, pages 1-24, July.
    7. Tej Bahadur Shahi & Ashish Shrestha & Arjun Neupane & William Guo, 2020. "Stock Price Forecasting with Deep Learning: A Comparative Study," Mathematics, MDPI, vol. 8(9), pages 1-15, August.
    8. Amin Aminimehr & Ali Raoofi & Akbar Aminimehr & Amirhossein Aminimehr, 2022. "A Comprehensive Study of Market Prediction from Efficient Market Hypothesis up to Late Intelligent Market Prediction Approaches," Computational Economics, Springer;Society for Computational Economics, vol. 60(2), pages 781-815, August.
    9. Vitor Azevedo & Christopher Hoegner, 2023. "Enhancing stock market anomalies with machine learning," Review of Quantitative Finance and Accounting, Springer, vol. 60(1), pages 195-230, January.
    10. Basak, Suryoday & Kar, Saibal & Saha, Snehanshu & Khaidem, Luckyson & Dey, Sudeepa Roy, 2019. "Predicting the direction of stock market prices using tree-based classifiers," The North American Journal of Economics and Finance, Elsevier, vol. 47(C), pages 552-567.
    11. Ben Moews, 2023. "On random number generators and practical market efficiency," Papers 2305.17419, arXiv.org, revised Jul 2023.
    12. Zhang, Bing & Li, Xiao-Ming & He, Fei, 2014. "Testing the evolution of crude oil market efficiency: Data have the conn," Energy Policy, Elsevier, vol. 68(C), pages 39-52.
    13. Hakan Pabuccu & Adrian Barbu, 2023. "Feature Selection with Annealing for Forecasting Financial Time Series," Papers 2303.02223, arXiv.org, revised Feb 2024.
    14. Alexander Jakob Dautel & Wolfgang Karl Härdle & Stefan Lessmann & Hsin-Vonn Seow, 2020. "Forex exchange rate forecasting using deep recurrent neural networks," Digital Finance, Springer, vol. 2(1), pages 69-96, September.
    15. Flori, Andrea & Regoli, Daniele, 2021. "Revealing Pairs-trading opportunities with long short-term memory networks," European Journal of Operational Research, Elsevier, vol. 295(2), pages 772-791.
    16. Manahov, Viktor & Hudson, Robert & Gebka, Bartosz, 2014. "Does high frequency trading affect technical analysis and market efficiency? And if so, how?," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 28(C), pages 131-157.
    17. Ehsan Hoseinzade & Saman Haratizadeh & Arash Khoeini, 2019. "U-CNNpred: A Universal CNN-based Predictor for Stock Markets," Papers 1911.12540, arXiv.org.
    18. Iris Chen & Yogeshwar D Kelkar & Yu Gu & Jie Zhou & Xing Qiu & Hulin Wu, 2017. "High-dimensional linear state space models for dynamic microbial interaction networks," PLOS ONE, Public Library of Science, vol. 12(11), pages 1-20, November.
    19. Wai Khuen Cheng & Khean Thye Bea & Steven Mun Hong Leow & Jireh Yi-Le Chan & Zeng-Wei Hong & Yen-Lin Chen, 2022. "A Review of Sentiment, Semantic and Event-Extraction-Based Approaches in Stock Forecasting," Mathematics, MDPI, vol. 10(14), pages 1-20, July.
    20. Rzayev, Khaladdin & Ibikunle, Gbenga, 2019. "A state-space modeling of the information content of trading volume," Journal of Financial Markets, Elsevier, vol. 46(C).

    More about this item

    Keywords

    Stock Forecasting; Recurrent dictionary learning; Kalman filter; expectation-minimization; dynamical modeling; uncertainty quantification;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:hal-03184841. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.