IDEAS home Printed from https://ideas.repec.org/a/spr/digfin/v4y2022i1d10.1007_s42521-021-00046-2.html
   My bibliography  Save this article

Machine learning for financial forecasting, planning and analysis: recent developments and pitfalls

Author

Listed:
  • Helmut Wasserbacher

    (Novartis International AG, Novartis Campus)

  • Martin Spindler

    (University of Hamburg)

Abstract

This article is an introduction to machine learning for financial forecasting, planning and analysis (FP&A). Machine learning appears well suited to support FP&A with the highly automated extraction of information from large amounts of data. However, because most traditional machine learning techniques focus on forecasting (prediction), we discuss the particular care that must be taken to avoid the pitfalls of using them for planning and resource allocation (causal inference). While the naive application of machine learning usually fails in this context, the recently developed double machine learning framework can address causal questions of interest. We review the current literature on machine learning in FP&A and illustrate in a simulation study how machine learning can be used for both forecasting and planning. We also investigate how forecasting and planning improve as the number of data points increases.

Suggested Citation

  • Helmut Wasserbacher & Martin Spindler, 2022. "Machine learning for financial forecasting, planning and analysis: recent developments and pitfalls," Digital Finance, Springer, vol. 4(1), pages 63-88, March.
  • Handle: RePEc:spr:digfin:v:4:y:2022:i:1:d:10.1007_s42521-021-00046-2
    DOI: 10.1007/s42521-021-00046-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s42521-021-00046-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s42521-021-00046-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2018. "Double/debiased machine learning for treatment and structural parameters," Econometrics Journal, Royal Economic Society, vol. 21(1), pages 1-68, February.
    2. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "High-Dimensional Methods and Inference on Structural and Treatment Effects," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 29-50, Spring.
    3. Hyndman, Rob J. & Ord, J. Keith, 2006. "Twenty-five years of forecasting," International Journal of Forecasting, Elsevier, vol. 22(3), pages 413-414.
    4. De Gooijer, Jan G. & Hyndman, Rob J., 2006. "25 years of time series forecasting," International Journal of Forecasting, Elsevier, vol. 22(3), pages 443-473.
    5. Daron Acemoglu & Pascual Restrepo, 2018. "Artificial Intelligence, Automation, and Work," NBER Chapters, in: The Economics of Artificial Intelligence: An Agenda, pages 197-236, National Bureau of Economic Research, Inc.
    6. Hal Varian, 2018. "Artificial Intelligence, Economics, and Industrial Organization," NBER Chapters, in: The Economics of Artificial Intelligence: An Agenda, pages 399-419, National Bureau of Economic Research, Inc.
    7. Stephen Hansen, 2011. "A Theoretical Analysis of the Impact of Adopting Rolling Budgets, Activity-Based Budgeting and Beyond Budgeting," European Accounting Review, Taylor & Francis Journals, vol. 20(2), pages 289-319.
    8. Oesterreich, Thuy Duong & Teuteberg, Frank & Bensberg, Frank & Buscher, Gandalf, 2019. "The controlling profession in the digital age: Understanding the impact of digitisation on the controller's job roles, skills and competences," International Journal of Accounting Information Systems, Elsevier, vol. 35(C).
    9. A. Belloni & D. Chen & V. Chernozhukov & C. Hansen, 2012. "Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain," Econometrica, Econometric Society, vol. 80(6), pages 2369-2429, November.
    10. Thomas Blake & Chris Nosko & Steven Tadelis, 2015. "Consumer Heterogeneity and Paid Search Effectiveness: A Large‐Scale Field Experiment," Econometrica, Econometric Society, vol. 83, pages 155-174, January.
    11. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "Inference on Treatment Effects after Selection among High-Dimensional Controlsâ€," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 81(2), pages 608-650.
    12. Victor Chernozhukov & Christian Hansen & Martin Spindler, 2015. "Valid Post-Selection and Post-Regularization Inference: An Elementary, General Approach," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 649-688, August.
    13. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2011. "Inference for high-dimensional sparse econometric models," CeMMAP working papers CWP41/11, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    14. Daron Acemoglu & Pascual Restrepo, 2018. "Artificial Intelligence, Automation and Work," Boston University - Department of Economics - Working Papers Series dp-298, Boston University - Department of Economics.
    15. Donald B. Rubin, 2005. "Causal Inference Using Potential Outcomes: Design, Modeling, Decisions," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 322-331, March.
    16. Patrick Bajari & Victor Chernozhukov & Ali Hortaçsu & Junichi Suzuki, 2019. "The Impact of Big Data on Firm Performance: An Empirical Investigation," AEA Papers and Proceedings, American Economic Association, vol. 109, pages 33-37, May.
    17. Susan Athey, 2018. "The Impact of Machine Learning on Economics," NBER Chapters, in: The Economics of Artificial Intelligence: An Agenda, pages 507-547, National Bureau of Economic Research, Inc.
    18. Ahmet Murat Ozbayoglu & Mehmet Ugur Gudelek & Omer Berat Sezer, 2020. "Deep Learning for Financial Applications : A Survey," Papers 2002.05786, arXiv.org.
    19. Gandomi, Amir & Haider, Murtaza, 2015. "Beyond the hype: Big data concepts, methods, and analytics," International Journal of Information Management, Elsevier, vol. 35(2), pages 137-144.
    20. Sendhil Mullainathan & Jann Spiess, 2017. "Machine Learning: An Applied Econometric Approach," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 87-106, Spring.
    21. Amita Gajewar & Gagan Bansal, 2016. "Revenue Forecasting for Enterprise Products," Papers 1701.06624, arXiv.org.
    22. Victor Chernozhukov & Chris Hansen & Martin Spindler, 2016. "High-Dimensional Metrics in R," Papers 1603.01700, arXiv.org, revised Aug 2016.
    23. Klaus Möller & Utz Schäffer & Frank Verbeeten, 2020. "Digitalization in management accounting and control: an editorial," Journal of Management Control: Zeitschrift für Planung und Unternehmenssteuerung, Springer, vol. 31(1), pages 1-8, April.
    24. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey, 2017. "Double/Debiased/Neyman Machine Learning of Treatment Effects," American Economic Review, American Economic Association, vol. 107(5), pages 261-265, May.
    25. Kusters, Ulrich & McCullough, B.D. & Bell, Michael, 2006. "Forecasting software: Past, present and future," International Journal of Forecasting, Elsevier, vol. 22(3), pages 599-615.
    26. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2016. "Double/Debiased Machine Learning for Treatment and Causal Parameters," Papers 1608.00060, arXiv.org, revised Dec 2017.
    27. Erik Strauß & Christina Zecher, 2013. "Management control systems: a review," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 23(4), pages 233-268, February.
    28. Sebastian D. Becker & Matthias D. Mahlendorf & Utz Schäffer & Mario Thaten, 2016. "Budgeting in Times of Economic Crisis," Contemporary Accounting Research, John Wiley & Sons, vol. 33(4), pages 1489-1517, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Helmut Wasserbacher & Martin Spindler, 2021. "Machine Learning for Financial Forecasting, Planning and Analysis: Recent Developments and Pitfalls," Papers 2107.04851, arXiv.org.
    2. Sander Gerritsen & Mark Kattenberg & Sonny Kuijpers, 2019. "The impact of age at arrival on education and mental health," CPB Discussion Paper 389.rdf, CPB Netherlands Bureau for Economic Policy Analysis.
    3. Sander Gerritsen & Mark Kattenberg & Sonny Kuijpers, 2019. "The impact of age at arrival on education and mental health," CPB Discussion Paper 389, CPB Netherlands Bureau for Economic Policy Analysis.
    4. Su, Liangjun & Ura, Takuya & Zhang, Yichong, 2019. "Non-separable models with high-dimensional data," Journal of Econometrics, Elsevier, vol. 212(2), pages 646-677.
    5. Hoang, Daniel & Wiegratz, Kevin, 2022. "Machine learning methods in finance: Recent applications and prospects," Working Paper Series in Economics 158, Karlsruhe Institute of Technology (KIT), Department of Economics and Management.
    6. Belloni, Alexandre & Hansen, Christian & Newey, Whitney, 2022. "High-dimensional linear models with many endogenous variables," Journal of Econometrics, Elsevier, vol. 228(1), pages 4-26.
    7. Michael C. Knaus, 2021. "A double machine learning approach to estimate the effects of musical practice on student’s skills," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 282-300, January.
    8. Croux, Christophe & Jagtiani, Julapa & Korivi, Tarunsai & Vulanovic, Milos, 2020. "Important factors determining Fintech loan default: Evidence from a lendingclub consumer platform," Journal of Economic Behavior & Organization, Elsevier, vol. 173(C), pages 270-296.
    9. Philipp Bach & Victor Chernozhukov & Malte S. Kurz & Martin Spindler & Sven Klaassen, 2021. "DoubleML -- An Object-Oriented Implementation of Double Machine Learning in R," Papers 2103.09603, arXiv.org, revised Feb 2024.
    10. Alexandre Belloni & Victor Chernozhukov & Denis Chetverikov & Christian Hansen & Kengo Kato, 2018. "High-dimensional econometrics and regularized GMM," CeMMAP working papers CWP35/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    11. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    12. Michael C Knaus & Michael Lechner & Anthony Strittmatter, 2021. "Machine learning estimation of heterogeneous causal effects: Empirical Monte Carlo evidence," The Econometrics Journal, Royal Economic Society, vol. 24(1), pages 134-161.
    13. Domenico Giannone & Michele Lenza & Giorgio E. Primiceri, 2021. "Economic Predictions With Big Data: The Illusion of Sparsity," Econometrica, Econometric Society, vol. 89(5), pages 2409-2437, September.
    14. Ajay Agrawal & Joshua Gans & Avi Goldfarb, 2019. "Economic Policy for Artificial Intelligence," Innovation Policy and the Economy, University of Chicago Press, vol. 19(1), pages 139-159.
    15. Elliott Ash & Daniel L. Chen & Sergio Galletta, 2022. "Measuring Judicial Sentiment: Methods and Application to US Circuit Courts," Economica, London School of Economics and Political Science, vol. 89(354), pages 362-376, April.
    16. Songul Cinaroglu, 2020. "Modelling unbalanced catastrophic health expenditure data by using machine‐learning methods," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 27(4), pages 168-181, October.
    17. Mark Kattenberg & Bas Scheer & Jurre Thiel, 2023. "Causal forests with fixed effects for treatment effect heterogeneity in difference-in-differences," CPB Discussion Paper 452, CPB Netherlands Bureau for Economic Policy Analysis.
    18. Michael Bailey & Drew Johnston & Theresa Kuchler & Johannes Stroebel & Arlene Wong, 2022. "Peer Effects in Product Adoption," American Economic Journal: Applied Economics, American Economic Association, vol. 14(3), pages 488-526, July.
    19. Neng-Chieh Chang, 2020. "The Mode Treatment Effect," Papers 2007.11606, arXiv.org.
    20. Victor Chernozhukov & Whitney Newey & Vira Semenova, 2019. "Inference on weighted average value function in high-dimensional state space," Papers 1908.09173, arXiv.org.

    More about this item

    Keywords

    Financial planning; Machine learning; Forecasting; Causal machine learning; Big data; Double machine learning;
    All these keywords.

    JEL classification:

    • G17 - Financial Economics - - General Financial Markets - - - Financial Forecasting and Simulation
    • G31 - Financial Economics - - Corporate Finance and Governance - - - Capital Budgeting; Fixed Investment and Inventory Studies
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:digfin:v:4:y:2022:i:1:d:10.1007_s42521-021-00046-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.