IDEAS home Printed from https://ideas.repec.org/a/gam/jforec/v5y2023i3p31-575d1232824.html
   My bibliography  Save this article

Searching for Promisingly Trained Artificial Neural Networks

Author

Listed:
  • Juan M. Lujano-Rojas

    (Department of Electrical Engineering, University of Zaragoza, Calle María de Luna 3, 50018 Zaragoza, Spain)

  • Rodolfo Dufo-López

    (Department of Electrical Engineering, University of Zaragoza, Calle María de Luna 3, 50018 Zaragoza, Spain)

  • Jesús Sergio Artal-Sevil

    (Department of Electrical Engineering, University of Zaragoza, Calle María de Luna 3, 50018 Zaragoza, Spain)

  • Eduardo García-Paricio

    (Department of Electrical Engineering, University of Zaragoza, Calle María de Luna 3, 50018 Zaragoza, Spain)

Abstract

Assessing the training process of artificial neural networks (ANNs) is vital for enhancing their performance and broadening their applicability. This paper employs the Monte Carlo simulation (MCS) technique, integrated with a stopping criterion, to construct the probability distribution of the learning error of an ANN designed for short-term forecasting. The training and validation processes were conducted multiple times, each time considering a unique random starting point, and the subsequent forecasting error was calculated one step ahead. From this, we ascertained the probability of having obtained all the local optima. Our extensive computational analysis involved training a shallow feedforward neural network (FFNN) using wind power and load demand data from the transmission systems of the Netherlands and Germany. Furthermore, the analysis was expanded to include wind speed prediction using a long short-term memory (LSTM) network at a site in Spain. The improvement gained from the FFNN, which has a high probability of being the global optimum, ranges from 0.7% to 8.6%, depending on the forecasting variable. This solution outperforms the persistent model by between 5.5% and 20.3%. For wind speed predictions using an LSTM, the improvement over an average-trained network stands at 9.5%, and is 6% superior to the persistent approach. These outcomes suggest that the advantages of exhaustive search vary based on the problem being analyzed and the type of network in use. The MCS method we implemented, which estimates the probability of identifying all local optima, can act as a foundational step for other techniques like Bayesian model selection, which assumes that the global optimum is encompassed within the available hypotheses.

Suggested Citation

  • Juan M. Lujano-Rojas & Rodolfo Dufo-López & Jesús Sergio Artal-Sevil & Eduardo García-Paricio, 2023. "Searching for Promisingly Trained Artificial Neural Networks," Forecasting, MDPI, vol. 5(3), pages 1-26, September.
    • Handle: RePEc:gam:jforec:v:5:y:2023:i:3:p:31-575:d:1232824
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2571-9394/5/3/31/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2571-9394/5/3/31/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Chang Xuan Mao, 2004. "Predicting the Conditional Probability of Discovering a New Class," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 1108-1118, December.
    2. Wang, Yun & Zou, Runmin & Liu, Fang & Zhang, Lingjun & Liu, Qianyi, 2021. "A review of wind speed and wind power forecasting with deep neural networks," Applied Energy, Elsevier, vol. 304(C).
    3. Lago, Jesus & Marcjasz, Grzegorz & De Schutter, Bart & Weron, Rafał, 2021. "Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark," Applied Energy, Elsevier, vol. 293(C).
    4. Ismail Shah & Hasnain Iftikhar & Sajid Ali, 2020. "Modeling and Forecasting Medium-Term Electricity Consumption Using Component Estimation Technique," Forecasting, MDPI, vol. 2(2), pages 1-17, May.
    5. Ismail Shah & Faheem Jan & Sajid Ali & Tahir Mehmood, 2022. "Functional Data Approach for Short-Term Electricity Demand Forecasting," Mathematical Problems in Engineering, Hindawi, vol. 2022, pages 1-14, June.
    6. Tawn, R. & Browell, J., 2022. "A review of very short-term wind and solar power forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 153(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nazila Pourhaji & Mohammad Asadpour & Ali Ahmadian & Ali Elkamel, 2022. "The Investigation of Monthly/Seasonal Data Clustering Impact on Short-Term Electricity Price Forecasting Accuracy: Ontario Province Case Study," Sustainability, MDPI, vol. 14(5), pages 1-14, March.
    2. Jian Zhu & Zhiyuan Zhao & Xiaoran Zheng & Zhao An & Qingwu Guo & Zhikai Li & Jianling Sun & Yuanjun Guo, 2023. "Time-Series Power Forecasting for Wind and Solar Energy Based on the SL-Transformer," Energies, MDPI, vol. 16(22), pages 1-15, November.
    3. Yang, Mao & Wang, Da & Zhang, Wei, 2023. "A short-term wind power prediction method based on dynamic and static feature fusion mining," Energy, Elsevier, vol. 280(C).
    4. Paweł Piotrowski & Inajara Rutyna & Dariusz Baczyński & Marcin Kopyt, 2022. "Evaluation Metrics for Wind Power Forecasts: A Comprehensive Review and Statistical Analysis of Errors," Energies, MDPI, vol. 15(24), pages 1-38, December.
    5. Hasnain Iftikhar & Josue E. Turpo-Chaparro & Paulo Canas Rodrigues & Javier Linkolk López-Gonzales, 2023. "Day-Ahead Electricity Demand Forecasting Using a Novel Decomposition Combination Method," Energies, MDPI, vol. 16(18), pages 1-22, September.
    6. Li, Min & Yang, Yi & He, Zhaoshuang & Guo, Xinbo & Zhang, Ruisheng & Huang, Bingqing, 2023. "A wind speed forecasting model based on multi-objective algorithm and interpretability learning," Energy, Elsevier, vol. 269(C).
    7. Shengli Liao & Xudong Tian & Benxi Liu & Tian Liu & Huaying Su & Binbin Zhou, 2022. "Short-Term Wind Power Prediction Based on LightGBM and Meteorological Reanalysis," Energies, MDPI, vol. 15(17), pages 1-21, August.
    8. Bartosz Uniejewski, 2024. "Regularization for electricity price forecasting," Papers 2404.03968, arXiv.org.
    9. Cheng-Yu Ho & Ke-Sheng Cheng & Chi-Hang Ang, 2023. "Utilizing the Random Forest Method for Short-Term Wind Speed Forecasting in the Coastal Area of Central Taiwan," Energies, MDPI, vol. 16(3), pages 1-18, January.
    10. Caputo, Antonio C. & Federici, Alessandro & Pelagagge, Pacifico M. & Salini, Paolo, 2023. "Offshore wind power system economic evaluation framework under aleatory and epistemic uncertainty," Applied Energy, Elsevier, vol. 350(C).
    11. Galarneau-Vincent, Rémi & Gauthier, Geneviève & Godin, Frédéric, 2023. "Foreseeing the worst: Forecasting electricity DART spikes," Energy Economics, Elsevier, vol. 119(C).
    12. Wang, Jianzhou & Wang, Shuai & Zeng, Bo & Lu, Haiyan, 2022. "A novel ensemble probabilistic forecasting system for uncertainty in wind speed," Applied Energy, Elsevier, vol. 313(C).
    13. Alessandro Bosisio & Matteo Moncecchi & Andrea Morotti & Marco Merlo, 2021. "Machine Learning and GIS Approach for Electrical Load Assessment to Increase Distribution Networks Resilience," Energies, MDPI, vol. 14(14), pages 1-23, July.
    14. Muhammad Ahmar & Fahad Ali & Yuexiang Jiang & Mamdooh Alwetaishi & Sherif S. M. Ghoneim, 2022. "Households’ Energy Choices in Rural Pakistan," Energies, MDPI, vol. 15(9), pages 1-23, April.
    15. Weronika Nitka & Rafał Weron, 2023. "Combining predictive distributions of electricity prices. Does minimizing the CRPS lead to optimal decisions in day-ahead bidding?," Operations Research and Decisions, Wroclaw University of Science and Technology, Faculty of Management, vol. 33(3), pages 105-118.
    16. Paletta, Quentin & Arbod, Guillaume & Lasenby, Joan, 2023. "Omnivision forecasting: Combining satellite and sky images for improved deterministic and probabilistic intra-hour solar energy predictions," Applied Energy, Elsevier, vol. 336(C).
    17. Nebojsa Bacanin & Catalin Stoean & Miodrag Zivkovic & Miomir Rakic & Roma Strulak-Wójcikiewicz & Ruxandra Stoean, 2023. "On the Benefits of Using Metaheuristics in the Hyperparameter Tuning of Deep Learning Models for Energy Load Forecasting," Energies, MDPI, vol. 16(3), pages 1-21, February.
    18. Rodríguez, Abel, 2013. "On the Jeffreys prior for the multivariate Ewens distribution," Statistics & Probability Letters, Elsevier, vol. 83(6), pages 1539-1546.
    19. Kate Murray & Andrea Rossi & Diego Carraro & Andrea Visentin, 2023. "On Forecasting Cryptocurrency Prices: A Comparison of Machine Learning, Deep Learning, and Ensembles," Forecasting, MDPI, vol. 5(1), pages 1-14, January.
    20. Bhagwan, N. & Evans, M., 2023. "A review of industry 4.0 technologies used in the production of energy in China, Germany, and South Africa," Renewable and Sustainable Energy Reviews, Elsevier, vol. 173(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jforec:v:5:y:2023:i:3:p:31-575:d:1232824. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.