IDEAS home Printed from https://ideas.repec.org/a/bas/econst/y2022i2p15-43.html
   My bibliography  Save this article

Performance of 109 Machine Learning Algorithms across Five Forecasting Tasks: Employee Behavior Modeling, Online Communication, House Pricing, IT Support and Demand Planning

Author

Listed:
  • Anton A. Gerunov

Abstract

This article puts the problem of forecasting in economic and business situations under scrutiny. Starting from the premise that accurate forecasting is now a key capability for analyzing problems of business operations and public policy, we investigate the performance of alternative prediction methods that include both traditional econometric approaches as well as novel algorithms from the field of machine learning. The article tests a total of 109 different regression-type algorithms across five pertinent business domains – employee absenteeism, success of online communication, real estate asset pricing, support ticket processing, and demand forecasting. The results indicate that forecasting algorithms tend to produce a set of widely dispersed outcome, with some methods such as random forecast and neural network implementations being able to consistently generate superior performance. We further argue that forecast accuracy is not necessarily predicated upon computational complexity and thus, an optimization decision between the costs and benefits of using a certain algorithm can feasibly be made.

Suggested Citation

  • Anton A. Gerunov, 2022. "Performance of 109 Machine Learning Algorithms across Five Forecasting Tasks: Employee Behavior Modeling, Online Communication, House Pricing, IT Support and Demand Planning," Economic Studies journal, Bulgarian Academy of Sciences - Economic Research Institute, issue 2, pages 15-43.
    • Handle: RePEc:bas:econst:y:2022:i:2:p:15-43
    as

    Download full text from publisher

    File URL: https://www.iki.bas.bg/Journals/EconomicStudies/2022/2022-2/2_Anton-Gerunov.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Kunze, Frederik & Wegener, Christoph & Bizer, Kilian & Spiwoks, Markus, 2017. "Forecasting European interest rates in times of financial crisis – What insights do we get from international survey forecasts?," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 48(C), pages 192-205.
    2. Kuhn, Max, 2008. "Building Predictive Models in R Using the caret Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 28(i05).
    3. (Jeremy) Chiu, Ching-wai & Harris, Richard D.F. & Stoja, Evarist & Chin, Michael, 2018. "Financial market Volatility, macroeconomic fundamentals and investor Sentiment," Journal of Banking & Finance, Elsevier, vol. 92(C), pages 130-145.
    4. Lago, Jesus & De Ridder, Fjo & De Schutter, Bart, 2018. "Forecasting spot electricity prices: Deep learning approaches and empirical comparison of traditional algorithms," Applied Energy, Elsevier, vol. 221(C), pages 386-405.
    5. Wolter Hassink, 2018. "How to reduce workplace absenteeism," IZA World of Labor, Institute of Labor Economics (IZA), pages 447-447, September.
    6. Mei, Dexiang & Liu, Jing & Ma, Feng & Chen, Wang, 2017. "Forecasting stock market volatility: Do realized skewness and kurtosis help?," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 481(C), pages 153-159.
    7. Debnath, Kumar Biswajit & Mourshed, Monjur, 2018. "Forecasting methods in energy planning models," Renewable and Sustainable Energy Reviews, Elsevier, vol. 88(C), pages 297-325.
    8. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2020. "The M4 Competition: 100,000 time series and 61 forecasting methods," International Journal of Forecasting, Elsevier, vol. 36(1), pages 54-74.
    9. Makridakis, Spyros & Hibon, Michele, 2000. "The M3-Competition: results, conclusions and implications," International Journal of Forecasting, Elsevier, vol. 16(4), pages 451-476.
    10. Hong, Tao & Pinson, Pierre & Fan, Shu & Zareipour, Hamidreza & Troccoli, Alberto & Hyndman, Rob J., 2016. "Probabilistic energy forecasting: Global Energy Forecasting Competition 2014 and beyond," International Journal of Forecasting, Elsevier, vol. 32(3), pages 896-913.
    11. Rob J. Hyndman, 2006. "Another Look at Forecast Accuracy Metrics for Intermittent Demand," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 4, pages 43-46, June.
    12. P. Christensen & K. Gillingham & W. Nordhaus, 2018. "Uncertainty in forecasts of long-run economic growth," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 115(21), pages 5409-5414, May.
    13. Kris Johnson Ferreira & Bin Hong Alex Lee & David Simchi-Levi, 2016. "Analytics for an Online Retailer: Demand Forecasting and Price Optimization," Manufacturing & Service Operations Management, INFORMS, vol. 18(1), pages 69-88, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    2. Olivares, Kin G. & Challu, Cristian & Marcjasz, Grzegorz & Weron, Rafał & Dubrawski, Artur, 2023. "Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with NBEATSx," International Journal of Forecasting, Elsevier, vol. 39(2), pages 884-900.
    3. Katarzyna Maciejowska & Bartosz Uniejewski & Rafa{l} Weron, 2022. "Forecasting Electricity Prices," Papers 2204.11735, arXiv.org.
    4. Makridakis, Spyros & Hyndman, Rob J. & Petropoulos, Fotios, 2020. "Forecasting in social settings: The state of the art," International Journal of Forecasting, Elsevier, vol. 36(1), pages 15-28.
    5. Ulrich Gunter & Irem Önder & Egon Smeral, 2020. "Are Combined Tourism Forecasts Better at Minimizing Forecasting Errors?," Forecasting, MDPI, vol. 2(3), pages 1-19, June.
    6. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2022. "Predicting/hypothesizing the findings of the M5 competition," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1337-1345.
    7. Liyang Tang, 2020. "Application of Nonlinear Autoregressive with Exogenous Input (NARX) neural network in macroeconomic forecasting, national goal setting and global competitiveness assessment," Papers 2005.08735, arXiv.org.
    8. Fotios Petropoulos & Evangelos Spiliotis, 2021. "The Wisdom of the Data: Getting the Most Out of Univariate Time Series Forecasting," Forecasting, MDPI, vol. 3(3), pages 1-20, June.
    9. Marcjasz, Grzegorz & Uniejewski, Bartosz & Weron, Rafał, 2019. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting with NARX neural networks," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1520-1532.
    10. Emilian Dobrescu, 2014. "Attempting to Quantify the Accuracy of Complex Macroeconomic Forecasts," Journal for Economic Forecasting, Institute for Economic Forecasting, vol. 0(4), pages 5-21, December.
    11. Hyndman, Rob J., 2020. "A brief history of forecasting competitions," International Journal of Forecasting, Elsevier, vol. 36(1), pages 7-14.
    12. Andrawis, Robert R. & Atiya, Amir F. & El-Shishiny, Hisham, 2011. "Combination of long term and short term forecasts, with application to tourism demand forecasting," International Journal of Forecasting, Elsevier, vol. 27(3), pages 870-886, July.
    13. Bojer, Casper Solheim & Meldgaard, Jens Peder, 2021. "Kaggle forecasting competitions: An overlooked learning opportunity," International Journal of Forecasting, Elsevier, vol. 37(2), pages 587-603.
    14. Halužan, Marko & Verbič, Miroslav & Zorić, Jelena, 2020. "Performance of alternative electricity price forecasting methods: Findings from the Greek and Hungarian power exchanges," Applied Energy, Elsevier, vol. 277(C).
    15. Ma, Shaohui & Fildes, Robert, 2020. "Forecasting third-party mobile payments with implications for customer flow prediction," International Journal of Forecasting, Elsevier, vol. 36(3), pages 739-760.
    16. Spiliotis, Evangelos & Petropoulos, Fotios, 2024. "On the update frequency of univariate forecasting models," European Journal of Operational Research, Elsevier, vol. 314(1), pages 111-121.
    17. Fantazzini, Dean, 2020. "Short-term forecasting of the COVID-19 pandemic using Google Trends data: Evidence from 158 countries," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 59, pages 33-54.
    18. Claudio Monteiro & L. Alfredo Fernandez-Jimenez & Ignacio J. Ramirez-Rosado, 2020. "Predictive Trading Strategy for Physical Electricity Futures," Energies, MDPI, vol. 13(14), pages 1-24, July.
    19. Van Belle, Jente & Guns, Tias & Verbeke, Wouter, 2021. "Using shared sell-through data to forecast wholesaler demand in multi-echelon supply chains," European Journal of Operational Research, Elsevier, vol. 288(2), pages 466-479.
    20. Nghia Chu & Binh Dao & Nga Pham & Huy Nguyen & Hien Tran, 2022. "Predicting Mutual Funds' Performance using Deep Learning and Ensemble Techniques," Papers 2209.09649, arXiv.org, revised Jul 2023.

    More about this item

    JEL classification:

    • C44 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Operations Research; Statistical Decision Theory
    • C45 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Neural Networks and Related Topics
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • D81 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Criteria for Decision-Making under Risk and Uncertainty

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bas:econst:y:2022:i:2:p:15-43. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Diana Dimitrova (email available below). General contact details of provider: https://edirc.repec.org/data/ikbasbg.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.