IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v12y2024i7p945-d1362292.html
   My bibliography  Save this article

A Novel Variant of LSTM Stock Prediction Method Incorporating Attention Mechanism

Author

Listed:
  • Shuai Sang

    (School of Mathematics, Physics and Statistics, Shanghai University of Engineering Science, Shanghai 201620, China)

  • Lu Li

    (School of Mathematics, Physics and Statistics, Shanghai University of Engineering Science, Shanghai 201620, China)

Abstract

Long Short-Term Memory (LSTM) is an effective method for stock price prediction. However, due to the nonlinear and highly random nature of stock price fluctuations over time, LSTM exhibits poor stability and is prone to overfitting, resulting in low prediction accuracy. To address this issue, this paper proposes a novel variant of LSTM that couples the forget gate and input gate in the LSTM structure, and adds a “simple” forget gate to the long-term cell state. In order to enhance the generalization ability and robustness of the variant LSTM, the paper introduces an attention mechanism and combines it with the variant LSTM, presenting the Attention Mechanism Variant LSTM (AMV-LSTM) model along with the corresponding backpropagation algorithm. The parameters in AMV-LSTM are updated using the Adam gradient descent method. Experimental results demonstrate that the variant LSTM alleviates the instability and overfitting issues of LSTM, effectively improving prediction accuracy. AMV-LSTM further enhances accuracy compared to the variant LSTM, and compared to AM-LSTM, it exhibits superior generalization ability, accuracy, and convergence capability.

Suggested Citation

  • Shuai Sang & Lu Li, 2024. "A Novel Variant of LSTM Stock Prediction Method Incorporating Attention Mechanism," Mathematics, MDPI, vol. 12(7), pages 1-20, March.
  • Handle: RePEc:gam:jmathe:v:12:y:2024:i:7:p:945-:d:1362292
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/12/7/945/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2227-7390/12/7/945/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Christian Janiesch & Patrick Zschech & Kai Heinrich, 2021. "Machine learning and deep learning," Electronic Markets, Springer;IIM University of St. Gallen, vol. 31(3), pages 685-695, September.
    2. Fischer, Thomas & Krauss, Christopher, 2018. "Deep learning with long short-term memory networks for financial market predictions," European Journal of Operational Research, Elsevier, vol. 270(2), pages 654-669.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jen-Yu Lee & Tien-Thinh Nguyen & Hong-Giang Nguyen & Jen-Yao Lee, 2022. "Towards Predictive Crude Oil Purchase: A Case Study in the USA and Europe," Energies, MDPI, vol. 15(11), pages 1-15, May.
    2. Wei Dai & Yuan An & Wen Long, 2021. "Price change prediction of ultra high frequency financial data based on temporal convolutional network," Papers 2107.00261, arXiv.org.
    3. Shao, Zhen & Zheng, Qingru & Yang, Shanlin & Gao, Fei & Cheng, Manli & Zhang, Qiang & Liu, Chen, 2020. "Modeling and forecasting the electricity clearing price: A novel BELM based pattern classification framework and a comparative analytic study on multi-layer BELM and LSTM," Energy Economics, Elsevier, vol. 86(C).
    4. Kamaladdin Fataliyev & Aneesh Chivukula & Mukesh Prasad & Wei Liu, 2021. "Stock Market Analysis with Text Data: A Review," Papers 2106.12985, arXiv.org, revised Jul 2021.
    5. Eduard Hartwich & Alexander Rieger & Johannes Sedlmeir & Dominik Jurek & Gilbert Fridgen, 2023. "Machine economies," Electronic Markets, Springer;IIM University of St. Gallen, vol. 33(1), pages 1-13, December.
    6. Rainer Alt, 2021. "Electronic Markets on robotics," Electronic Markets, Springer;IIM University of St. Gallen, vol. 31(3), pages 465-471, September.
    7. Giacomo di Tollo & Joseph Andria & Gianni Filograsso, 2023. "The Predictive Power of Social Media Sentiment: Evidence from Cryptocurrencies and Stock Markets Using NLP and Stochastic ANNs," Mathematics, MDPI, vol. 11(16), pages 1-18, August.
    8. Ghosh, Indranil & Chaudhuri, Tamal Datta & Alfaro-Cortés, Esteban & Gámez, Matías & García, Noelia, 2022. "A hybrid approach to forecasting futures prices with simultaneous consideration of optimality in ensemble feature selection and advanced artificial intelligence," Technological Forecasting and Social Change, Elsevier, vol. 181(C).
    9. Sina Montazeri & Akram Mirzaeinia & Haseebullah Jumakhan & Amir Mirzaeinia, 2024. "CNN-DRL for Scalable Actions in Finance," Papers 2401.06179, arXiv.org.
    10. Alameer, Zakaria & Elaziz, Mohamed Abd & Ewees, Ahmed A. & Ye, Haiwang & Jianhua, Zhang, 2019. "Forecasting gold price fluctuations using improved multilayer perceptron neural network and whale optimization algorithm," Resources Policy, Elsevier, vol. 61(C), pages 250-260.
    11. Abhirup Khanna & Bhawna Yadav Lamba & Sapna Jain & Vadim Bolshev & Dmitry Budnikov & Vladimir Panchenko & Alexandr Smirnov, 2023. "Biodiesel Production from Jatropha: A Computational Approach by Means of Artificial Intelligence and Genetic Algorithm," Sustainability, MDPI, vol. 15(12), pages 1-33, June.
    12. Rui Ma & Jia Wang & Wei Zhao & Hongjie Guo & Dongnan Dai & Yuliang Yun & Li Li & Fengqi Hao & Jinqiang Bai & Dexin Ma, 2022. "Identification of Maize Seed Varieties Using MobileNetV2 with Improved Attention Mechanism CBAM," Agriculture, MDPI, vol. 13(1), pages 1-16, December.
    13. Mst. Shapna Akter & Hossain Shahriar & Reaz Chowdhury & M. R. C. Mahdy, 2022. "Forecasting the Risk Factor of Frontier Markets: A Novel Stacking Ensemble of Neural Network Approach," Future Internet, MDPI, vol. 14(9), pages 1-23, August.
    14. Noura Metawa & Mohamemd I. Alghamdi & Ibrahim M. El-Hasnony & Mohamed Elhoseny, 2021. "Return Rate Prediction in Blockchain Financial Products Using Deep Learning," Sustainability, MDPI, vol. 13(21), pages 1-16, October.
    15. Kentaro Imajo & Kentaro Minami & Katsuya Ito & Kei Nakagawa, 2020. "Deep Portfolio Optimization via Distributional Prediction of Residual Factors," Papers 2012.07245, arXiv.org.
    16. Kailai Ni & Jianzhou Wang & Guangyu Tang & Danxiang Wei, 2019. "Research and Application of a Novel Hybrid Model Based on a Deep Neural Network for Electricity Load Forecasting: A Case Study in Australia," Energies, MDPI, vol. 12(13), pages 1-30, June.
    17. Jie Fang & Jianwu Lin & Shutao Xia & Yong Jiang & Zhikang Xia & Xiang Liu, 2020. "Neural Network-based Automatic Factor Construction," Papers 2008.06225, arXiv.org, revised Oct 2020.
    18. Qing, Xiangyun & Niu, Yugang, 2018. "Hourly day-ahead solar irradiance prediction using weather forecasts by LSTM," Energy, Elsevier, vol. 148(C), pages 461-468.
    19. James Wallbridge, 2020. "Transformers for Limit Order Books," Papers 2003.00130, arXiv.org.
    20. Burka, Dávid & Puppe, Clemens & Szepesváry, László & Tasnádi, Attila, 2022. "Voting: A machine learning approach," European Journal of Operational Research, Elsevier, vol. 299(3), pages 1003-1017.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:12:y:2024:i:7:p:945-:d:1362292. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.