IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v11y2023i6p1503-d1102009.html
   My bibliography  Save this article

Nonstationary Time Series Prediction Based on Deep Echo State Network Tuned by Bayesian Optimization

Author

Listed:
  • Yu-Ting Bai

    (School of Artificial Intelligence, Beijing Technology and Business University, Beijing 100048, China
    Beijing Laboratory for Intelligent Environmental Protection, Beijing Technology and Business University, Beijing 100048, China)

  • Wei Jia

    (School of Artificial Intelligence, Beijing Technology and Business University, Beijing 100048, China)

  • Xue-Bo Jin

    (School of Artificial Intelligence, Beijing Technology and Business University, Beijing 100048, China
    Beijing Laboratory for Intelligent Environmental Protection, Beijing Technology and Business University, Beijing 100048, China)

  • Ting-Li Su

    (School of Artificial Intelligence, Beijing Technology and Business University, Beijing 100048, China)

  • Jian-Lei Kong

    (School of Artificial Intelligence, Beijing Technology and Business University, Beijing 100048, China)

  • Zhi-Gang Shi

    (School of Artificial Intelligence, Beijing Technology and Business University, Beijing 100048, China)

Abstract

The predictions from time series data can help us sense development trends and make scientific decisions in advance. The commonly used forecasting methods with backpropagation consume a lot of computational resources. The deep echo state network (DeepESN) is an advanced prediction method with a deep neural network structure and training algorithm without backpropagation. In this paper, a Bayesian optimization algorithm (BOA) is proposed to optimize DeepESN to address the problem of increasing parameter scale. Firstly, the DeepESN was studied and constructed as the basic prediction model for the time series data. Secondly, the BOA was reconstructed, based on the DeepESN, for optimal parameter searching. The algorithm is proposed within the framework of the DeepESN. Thirdly, an experiment was conducted to verify the DeepESN with a BOA within three datasets: simulation data generated from computer programs, a real humidity dataset collected from Beijing, and a power load dataset obtained from America. Compared with the models of BP (backpropagation), LSTM (long short-term memory), GRU (gated recurrent unit), and ESN (echo state network), DeepESN obtained optimal results, which were 0.0719, 18.6707, and 764.5281 using RMSE evaluation. While getting better accuracy, the BOA optimization time was only 323.4 s, 563.2 s, and 9854 s for the three datasets. It is more efficient than grid search and grey wolf optimizer.

Suggested Citation

  • Yu-Ting Bai & Wei Jia & Xue-Bo Jin & Ting-Li Su & Jian-Lei Kong & Zhi-Gang Shi, 2023. "Nonstationary Time Series Prediction Based on Deep Echo State Network Tuned by Bayesian Optimization," Mathematics, MDPI, vol. 11(6), pages 1-22, March.
    • Handle: RePEc:gam:jmathe:v:11:y:2023:i:6:p:1503-:d:1102009
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/11/6/1503/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/11/6/1503/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Tim Leung & Theodore Zhao, 2021. "Financial Time Series Analysis and Forecasting with HHT Feature Generation and Machine Learning," Papers 2105.10871, arXiv.org.
    2. Wei Bao & Jun Yue & Yulei Rao, 2017. "A deep learning framework for financial time series using stacked autoencoders and long-short term memory," PLOS ONE, Public Library of Science, vol. 12(7), pages 1-24, July.
    3. Xue-Bo Jin & Zhong-Yao Wang & Wen-Tao Gong & Jian-Lei Kong & Yu-Ting Bai & Ting-Li Su & Hui-Jun Ma & Prasun Chakrabarti, 2023. "Variational Bayesian Network with Information Interpretability Filtering for Air Quality Forecasting," Mathematics, MDPI, vol. 11(4), pages 1-18, February.
    4. Umut Ugurlu & Ilkay Oksuz & Oktay Tas, 2018. "Electricity Price Forecasting Using Recurrent Neural Networks," Energies, MDPI, vol. 11(5), pages 1-23, May.
    5. Patrick L. McDermott & Christopher K. Wikle, 2019. "Deep echo state networks with uncertainty quantification for spatio‐temporal forecasting," Environmetrics, John Wiley & Sons, Ltd., vol. 30(3), May.
    6. Xin Xu & Cheng-Cai Yang & Yang Xiao & Jian-Lei Kong, 2023. "A Fine-Grained Recognition Neural Network with High-Order Feature Maps via Graph-Based Embedding for Natural Bird Diversity Conservation," IJERPH, MDPI, vol. 20(6), pages 1-20, March.
    7. Yuebing Xu & Jing Zhang & Zuqiang Long & Yan Chen, 2018. "A Novel Dual-Scale Deep Belief Network Method for Daily Urban Water Demand Forecasting," Energies, MDPI, vol. 11(5), pages 1-15, April.
    8. Jessica Wojtkiewicz & Matin Hosseini & Raju Gottumukkala & Terrence Lynn Chambers, 2019. "Hour-Ahead Solar Irradiance Forecasting Using Multivariate Gated Recurrent Units," Energies, MDPI, vol. 12(21), pages 1-13, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nadja Klein & Michael Stanley Smith & David J. Nott, 2020. "Deep Distributional Time Series Models and the Probabilistic Forecasting of Intraday Electricity Prices," Papers 2010.01844, arXiv.org, revised May 2021.
    2. Shao, Zhen & Zheng, Qingru & Yang, Shanlin & Gao, Fei & Cheng, Manli & Zhang, Qiang & Liu, Chen, 2020. "Modeling and forecasting the electricity clearing price: A novel BELM based pattern classification framework and a comparative analytic study on multi-layer BELM and LSTM," Energy Economics, Elsevier, vol. 86(C).
    3. Md. Nazmul Hasan & Rafia Nishat Toma & Abdullah-Al Nahid & M M Manjurul Islam & Jong-Myon Kim, 2019. "Electricity Theft Detection in Smart Grid Systems: A CNN-LSTM Based Approach," Energies, MDPI, vol. 12(17), pages 1-18, August.
    4. Andrea Bucci, 2020. "Realized Volatility Forecasting with Neural Networks," Journal of Financial Econometrics, Oxford University Press, vol. 18(3), pages 502-531.
    5. Jaydip Sen & Sidra Mehtab & Abhishek Dutta & Saikat Mondal, 2022. "Precise Stock Price Prediction for Optimized Portfolio Design Using an LSTM Model," Papers 2203.01326, arXiv.org.
    6. Jaydip Sen & Sidra Mehtab, 2021. "Design and Analysis of Robust Deep Learning Models for Stock Price Prediction," Papers 2106.09664, arXiv.org.
    7. Umut Ugurlu & Ilkay Oksuz & Oktay Tas, 2018. "Electricity Price Forecasting Using Recurrent Neural Networks," Energies, MDPI, vol. 11(5), pages 1-23, May.
    8. Adebayo Oshingbesan & Eniola Ajiboye & Peruth Kamashazi & Timothy Mbaka, 2022. "Model-Free Reinforcement Learning for Asset Allocation," Papers 2209.10458, arXiv.org.
    9. Sibtain, Muhammad & Li, Xianshan & Saleem, Snoober & Ain, Qurat-ul- & Shi, Qiang & Li, Fei & Saeed, Muhammad & Majeed, Fatima & Shah, Syed Shoaib Ahmed & Saeed, Muhammad Hammad, 2022. "Multifaceted irradiance prediction by exploiting hybrid decomposition-entropy-Spatiotemporal attention based Sequence2Sequence models," Renewable Energy, Elsevier, vol. 196(C), pages 648-682.
    10. Tomoshiro Ochiai & Jose C. Nacher, 2020. "Unveiling the directional network behind the financial statements data using volatility constraint correlation," Papers 2008.07836, arXiv.org, revised Jun 2023.
    11. Junyi Li & Xitong Wang & Yaoyang Lin & Arunesh Sinha & Micheal P. Wellman, 2020. "Generating Realistic Stock Market Order Streams," Papers 2006.04212, arXiv.org.
    12. James Wallbridge, 2020. "Transformers for Limit Order Books," Papers 2003.00130, arXiv.org.
    13. Antoine Proteau & Antoine Tahan & Ryad Zemouri & Marc Thomas, 2023. "Predicting the quality of a machined workpiece with a variational autoencoder approach," Journal of Intelligent Manufacturing, Springer, vol. 34(2), pages 719-737, February.
    14. Yang Qiao & Yiping Xia & Xiang Li & Zheng Li & Yan Ge, 2023. "Higher-order Graph Attention Network for Stock Selection with Joint Analysis," Papers 2306.15526, arXiv.org.
    15. Faheem Jan & Ismail Shah & Sajid Ali, 2022. "Short-Term Electricity Prices Forecasting Using Functional Time Series Analysis," Energies, MDPI, vol. 15(9), pages 1-15, May.
    16. Taewook Kim & Ha Young Kim, 2019. "Forecasting stock prices with a feature fusion LSTM-CNN model using different representations of the same data," PLOS ONE, Public Library of Science, vol. 14(2), pages 1-23, February.
    17. Zihao Zhang & Stefan Zohren & Stephen Roberts, 2018. "DeepLOB: Deep Convolutional Neural Networks for Limit Order Books," Papers 1808.03668, arXiv.org, revised Jan 2020.
    18. Daiki Matsunaga & Toyotaro Suzumura & Toshihiro Takahashi, 2019. "Exploring Graph Neural Networks for Stock Market Predictions with Rolling Window Analysis," Papers 1909.10660, arXiv.org, revised Nov 2019.
    19. Emil Kraft & Dogan Keles & Wolf Fichtner, 2020. "Modeling of frequency containment reserve prices with econometrics and artificial intelligence," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(8), pages 1179-1197, December.
    20. Arumugham, Dinesh Rajan & Rajendran, Parvathy, 2021. "Modelling global solar irradiance for any location on earth through regression analysis using high-resolution data," Renewable Energy, Elsevier, vol. 180(C), pages 1114-1123.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:11:y:2023:i:6:p:1503-:d:1102009. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.