IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2304.04912.html
   My bibliography  Save this paper

Financial Time Series Forecasting using CNN and Transformer

Author

Listed:
  • Zhen Zeng
  • Rachneet Kaur
  • Suchetha Siddagangappa
  • Saba Rahimi
  • Tucker Balch
  • Manuela Veloso

Abstract

Time series forecasting is important across various domains for decision-making. In particular, financial time series such as stock prices can be hard to predict as it is difficult to model short-term and long-term temporal dependencies between data points. Convolutional Neural Networks (CNN) are good at capturing local patterns for modeling short-term dependencies. However, CNNs cannot learn long-term dependencies due to the limited receptive field. Transformers on the other hand are capable of learning global context and long-term dependencies. In this paper, we propose to harness the power of CNNs and Transformers to model both short-term and long-term dependencies within a time series, and forecast if the price would go up, down or remain the same (flat) in the future. In our experiments, we demonstrated the success of the proposed method in comparison to commonly adopted statistical and deep learning methods on forecasting intraday stock price change of S&P 500 constituents.

Suggested Citation

  • Zhen Zeng & Rachneet Kaur & Suchetha Siddagangappa & Saba Rahimi & Tucker Balch & Manuela Veloso, 2023. "Financial Time Series Forecasting using CNN and Transformer," Papers 2304.04912, arXiv.org.
  • Handle: RePEc:arx:papers:2304.04912
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2304.04912
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Peter R. Winters, 1960. "Forecasting Sales by Exponentially Weighted Moving Averages," Management Science, INFORMS, vol. 6(3), pages 324-342, April.
    2. Marcellino, Massimiliano & Stock, James H. & Watson, Mark W., 2006. "A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 499-526.
    3. Wei Bao & Jun Yue & Yulei Rao, 2017. "A deep learning framework for financial time series using stacked autoencoders and long-short term memory," PLOS ONE, Public Library of Science, vol. 12(7), pages 1-24, July.
    4. Salinas, David & Flunkert, Valentin & Gasthaus, Jan & Januschowski, Tim, 2020. "DeepAR: Probabilistic forecasting with autoregressive recurrent networks," International Journal of Forecasting, Elsevier, vol. 36(3), pages 1181-1191.
    5. Holt, Charles C., 2004. "Forecasting seasonals and trends by exponentially weighted moving averages," International Journal of Forecasting, Elsevier, vol. 20(1), pages 5-10.
    6. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2020. "The M4 Competition: 100,000 time series and 61 forecasting methods," International Journal of Forecasting, Elsevier, vol. 36(1), pages 54-74.
    7. Holt, Charles C., 2004. "Author's retrospective on 'Forecasting seasonals and trends by exponentially weighted moving averages'," International Journal of Forecasting, Elsevier, vol. 20(1), pages 11-13.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Qianggang Ding & Haochen Shi & Jiadong Guo & Bang Liu, 2024. "TradExpert: Revolutionizing Trading with Mixture of Expert LLMs," Papers 2411.00782, arXiv.org, revised May 2025.
    2. Maheronnaghsh, Mohammad Javad & Gheidi, Mohammad Mahdi & Fazli, MohammadAmin, 2023. "Machine Learning Methods in Algorithmic Trading: An Experimental Evaluation of Supervised Learning Techniques for Stock Price," OSF Preprints dzp26, Center for Open Science.
    3. Darko B. Vuković & Sonja D. Radenković & Ivana Simeunović & Vyacheslav Zinovev & Milan Radovanović, 2024. "Predictive Patterns and Market Efficiency: A Deep Learning Approach to Financial Time Series Forecasting," Mathematics, MDPI, vol. 12(19), pages 1-26, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tiantian Tu, 2025. "Bridging Short- and Long-Term Dependencies: A CNN-Transformer Hybrid for Financial Time Series Forecasting," Papers 2504.19309, arXiv.org.
    2. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    3. Dalton Garcia Borges de Souza & Erivelton Antonio dos Santos & Francisco Tarcísio Alves Júnior & Mariá Cristina Vasconcelos Nascimento, 2021. "On Comparing Cross-Validated Forecasting Models with a Novel Fuzzy-TOPSIS Metric: A COVID-19 Case Study," Sustainability, MDPI, vol. 13(24), pages 1-25, December.
    4. Filelis - Papadopoulos, Christos K. & Kyziropoulos, Panagiotis E. & Morrison, John P. & O‘Reilly, Philip, 2022. "Modelling and forecasting based on recursive incomplete pseudoinverse matrices," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 197(C), pages 358-376.
    5. Meira, Erick & Cyrino Oliveira, Fernando Luiz & de Menezes, Lilian M., 2022. "Forecasting natural gas consumption using Bagging and modified regularization techniques," Energy Economics, Elsevier, vol. 106(C).
    6. Kosuke Kawakami & Hirokazu Kobayashi & Kazuhide Nakata, 2021. "Seasonal Inventory Management Model for Raw Materials in Steel Industry," Interfaces, INFORMS, vol. 51(4), pages 312-324, July.
    7. Andrea Kolková & Petr Rozehnal, 2022. "Hybrid demand forecasting models: pre-pandemic and pandemic use studies," Equilibrium. Quarterly Journal of Economics and Economic Policy, Institute of Economic Research, vol. 17(3), pages 699-725, September.
    8. Theresa Maria Rausch & Tobias Albrecht & Daniel Baier, 2022. "Beyond the beaten paths of forecasting call center arrivals: on the use of dynamic harmonic regression with predictor variables," Journal of Business Economics, Springer, vol. 92(4), pages 675-706, May.
    9. Ye, Lili & Xie, Naiming & Boylan, John E. & Shang, Zhongju, 2024. "Forecasting seasonal demand for retail: A Fourier time-varying grey model," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1467-1485.
    10. Isra Al-Turaiki & Fahad Almutlaq & Hend Alrasheed & Norah Alballa, 2021. "Empirical Evaluation of Alternative Time-Series Models for COVID-19 Forecasting in Saudi Arabia," IJERPH, MDPI, vol. 18(16), pages 1-19, August.
    11. Maia, André Luis Santiago & de Carvalho, Francisco de A.T., 2011. "Holt’s exponential smoothing and neural network models for forecasting interval-valued time series," International Journal of Forecasting, Elsevier, vol. 27(3), pages 740-759.
    12. Lin, Jiahe & Michailidis, George, 2024. "A multi-task encoder-dual-decoder framework for mixed frequency data prediction," International Journal of Forecasting, Elsevier, vol. 40(3), pages 942-957.
    13. Felipe Santos‐Marquez, 2022. "Spatial beta‐convergence forecasting models: Evidence from municipal homicide rates in Colombia," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(2), pages 294-302, March.
    14. Stefan Mihai Petrea & Cristina Zamfir & Ira Adeline Simionov & Alina Mogodan & Florian Marcel Nuţă & Adrian Turek Rahoveanu & Dumitru Nancu & Dragos Sebastian Cristea & Florin Marian Buhociu, 2021. "A Forecasting and Prediction Methodology for Improving the Blue Economy Resilience to Climate Change in the Romanian Lower Danube Euroregion," Sustainability, MDPI, vol. 13(21), pages 1-36, October.
    15. Vesna Karadzic & Bojan Pejovic, 2021. "Inflation Forecasting in the Western Balkans and EU: A Comparison of Holt-Winters, ARIMA and NNAR Models," The AMFITEATRU ECONOMIC journal, Academy of Economic Studies - Bucharest, Romania, vol. 23(57), pages 517-517.
    16. Benedict J. Drasch & Gilbert Fridgen & Lukas Häfner, 2020. "Demand response through automated air conditioning in commercial buildings—a data-driven approach," Business Research, Springer;German Academic Association for Business Research, vol. 13(3), pages 1491-1525, November.
    17. Konstantinos Oikonomou & Dimitris Damigos, 2025. "Short term forecasting of base metals prices using a LightGBM and a LightGBM - ARIMA ensemble," Mineral Economics, Springer;Raw Materials Group (RMG);Luleå University of Technology, vol. 38(1), pages 37-49, March.
    18. Jan G. De Gooijer & Rob J. Hyndman, 2005. "25 Years of IIF Time Series Forecasting: A Selective Review," Monash Econometrics and Business Statistics Working Papers 12/05, Monash University, Department of Econometrics and Business Statistics.
    19. Zhou, Shenghan & Hu, Chen & Qiao, Xiaoduo & Chang, Wenbing, 2016. "A forecasting method for Chinese civil planes attendance rate based on vague sets," Chaos, Solitons & Fractals, Elsevier, vol. 89(C), pages 518-526.
    20. Zhen Zeng & Tucker Balch & Manuela Veloso, 2021. "Deep Video Prediction for Time Series Forecasting," Papers 2102.12061, arXiv.org, revised Nov 2021.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2304.04912. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.