IDEAS home Printed from https://ideas.repec.org/a/spr/snopef/v4y2023i1d10.1007_s43069-023-00196-6.html
   My bibliography  Save this article

Optimizing Structure and Internal Unit Weights of Echo State Network for an Efficient LMS-Based Online Training

Author

Listed:
  • Javad Saadat

    (University of Birjand)

  • Mohsen Farshad

    (University of Birjand)

  • Hussein Eliasi

    (University of Birjand)

Abstract

Echo state network (ESN) is a special type of recurrent neural networks (RNN) wherein a dynamic reservoir is used in the hidden layer, the weight of internal units of ESN is kept fix during training process, and output weights are the only trainable weights. Therefore, network training in an offline mode can be changed into a linear regression equation which is simply solved, although it is required to use online training of ESN in some applied problems. The least mean square (LMS) algorithm can provide an easy and constant method for online training of ESN; however, the huge eigenvalue spreads of the correlation matrix of internal network states reduce the speed of the algorithm convergence. In this study, harmony search algorithm (HSA) is used to optimally produce the structure and weight of internal network units. It is possible to significantly reduce the eigenvalue spreads of the correlation matrix of network states by means of this algorithm. Thereafter, the LMS algorithm is used for the online training of ESN built with the help of HSA. Already-obtained simulation results show that the eigenvalue spreads of the correlation matrix are reduced millions of times, and the LMS algorithm increases the online training speed of the network several times with an acceptable precision of training.

Suggested Citation

  • Javad Saadat & Mohsen Farshad & Hussein Eliasi, 2023. "Optimizing Structure and Internal Unit Weights of Echo State Network for an Efficient LMS-Based Online Training," SN Operations Research Forum, Springer, vol. 4(1), pages 1-14, March.
    • Handle: RePEc:spr:snopef:v:4:y:2023:i:1:d:10.1007_s43069-023-00196-6
    DOI: 10.1007/s43069-023-00196-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s43069-023-00196-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s43069-023-00196-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hu, Huanling & Wang, Lin & Lv, Sheng-Xiang, 2020. "Forecasting energy consumption and wind power generation using deep echo state network," Renewable Energy, Elsevier, vol. 154(C), pages 598-613.
    2. Hu, Huanling & Wang, Lin & Tao, Rui, 2021. "Wind speed forecasting based on variational mode decomposition and improved echo state network," Renewable Energy, Elsevier, vol. 164(C), pages 729-751.
    3. Chitsazan, Mohammad Amin & Sami Fadali, M. & Trzynadlowski, Andrzej M., 2019. "Wind speed and wind direction forecasting using echo state network with nonlinear functions," Renewable Energy, Elsevier, vol. 131(C), pages 879-889.
    4. Hu, Huanling & Wang, Lin & Peng, Lu & Zeng, Yu-Rong, 2020. "Effective energy consumption forecasting using enhanced bagged echo state network," Energy, Elsevier, vol. 193(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hu, Huanling & Wang, Lin & Zhang, Dabin & Ling, Liwen, 2023. "Rolling decomposition method in fusion with echo state network for wind speed forecasting," Renewable Energy, Elsevier, vol. 216(C).
    2. Bai, Yulong & Liu, Ming-De & Ding, Lin & Ma, Yong-Jie, 2021. "Double-layer staged training echo-state networks for wind speed prediction using variational mode decomposition," Applied Energy, Elsevier, vol. 301(C).
    3. Hu, Huanling & Wang, Lin & Lv, Sheng-Xiang, 2020. "Forecasting energy consumption and wind power generation using deep echo state network," Renewable Energy, Elsevier, vol. 154(C), pages 598-613.
    4. Lu, Peng & Ye, Lin & Zhao, Yongning & Dai, Binhua & Pei, Ming & Tang, Yong, 2021. "Review of meta-heuristic algorithms for wind power prediction: Methodologies, applications and challenges," Applied Energy, Elsevier, vol. 301(C).
    5. Peng, Lu & Wang, Lin & Xia, De & Gao, Qinglu, 2022. "Effective energy consumption forecasting using empirical wavelet transform and long short-term memory," Energy, Elsevier, vol. 238(PB).
    6. Xin Ma & Yubin Cai & Hong Yuan & Yanqiao Deng, 2023. "Partially Linear Component Support Vector Machine for Primary Energy Consumption Forecasting of the Electric Power Sector in the United States," Sustainability, MDPI, vol. 15(9), pages 1-26, April.
    7. Xing, Zhikai & He, Yigang, 2023. "Multi-modal multi-step wind power forecasting based on stacking deep learning model," Renewable Energy, Elsevier, vol. 215(C).
    8. Ding, Lin & Bai, Yulong & Liu, Ming-De & Fan, Man-Hong & Yang, Jie, 2022. "Predicting short wind speed with a hybrid model based on a piecewise error correction method and Elman neural network," Energy, Elsevier, vol. 244(PA).
    9. Hu, Huanling & Wang, Lin & Tao, Rui, 2021. "Wind speed forecasting based on variational mode decomposition and improved echo state network," Renewable Energy, Elsevier, vol. 164(C), pages 729-751.
    10. García, Irene & Huo, Stella & Prado, Raquel & Bravo, Lelys, 2020. "Dynamic Bayesian temporal modeling and forecasting of short-term wind measurements," Renewable Energy, Elsevier, vol. 161(C), pages 55-64.
    11. Ren, Weijie & Li, Baisong & Han, Min, 2020. "A novel Granger causality method based on HSIC-Lasso for revealing nonlinear relationship between multivariate time series," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 541(C).
    12. Ivan S. Maksymov, 2023. "Analogue and Physical Reservoir Computing Using Water Waves: Applications in Power Engineering and Beyond," Energies, MDPI, vol. 16(14), pages 1-26, July.
    13. Tian, Zhongda & Chen, Hao, 2021. "Multi-step short-term wind speed prediction based on integrated multi-model fusion," Applied Energy, Elsevier, vol. 298(C).
    14. Acikgoz, Hakan & Budak, Umit & Korkmaz, Deniz & Yildiz, Ceyhun, 2021. "WSFNet: An efficient wind speed forecasting model using channel attention-based densely connected convolutional neural network," Energy, Elsevier, vol. 233(C).
    15. Ijaz Ul Haq & Amin Ullah & Samee Ullah Khan & Noman Khan & Mi Young Lee & Seungmin Rho & Sung Wook Baik, 2021. "Sequential Learning-Based Energy Consumption Prediction Model for Residential and Commercial Sectors," Mathematics, MDPI, vol. 9(6), pages 1-17, March.
    16. Elianne Mora & Jenny Cifuentes & Geovanny Marulanda, 2021. "Short-Term Forecasting of Wind Energy: A Comparison of Deep Learning Frameworks," Energies, MDPI, vol. 14(23), pages 1-26, November.
    17. Shengxiang Lv & Lin Wang & Sirui Wang, 2023. "A Hybrid Neural Network Model for Short-Term Wind Speed Forecasting," Energies, MDPI, vol. 16(4), pages 1-18, February.
    18. Wang, Yun & Zou, Runmin & Liu, Fang & Zhang, Lingjun & Liu, Qianyi, 2021. "A review of wind speed and wind power forecasting with deep neural networks," Applied Energy, Elsevier, vol. 304(C).
    19. Xiang Ying & Keke Zhao & Zhiqiang Liu & Jie Gao & Dongxiao He & Xuewei Li & Wei Xiong, 2022. "Wind Speed Prediction via Collaborative Filtering on Virtual Edge Expanding Graphs," Mathematics, MDPI, vol. 10(11), pages 1-16, June.
    20. Yang, Hufang & Jiang, Ping & Wang, Ying & Li, Hongmin, 2022. "A fuzzy intelligent forecasting system based on combined fuzzification strategy and improved optimization algorithm for renewable energy power generation," Applied Energy, Elsevier, vol. 325(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:snopef:v:4:y:2023:i:1:d:10.1007_s43069-023-00196-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.