IDEAS home Printed from https://ideas.repec.org/a/bla/sysdyn/v38y2022i2p135-166.html
   My bibliography  Save this article

Pattern‐oriented analysis of system dynamics models via random forests

Author

Listed:
  • Mert Edali

Abstract

System dynamics (SD) modeling studies aim to reveal the causes of problematic dynamic behaviors and eliminate them through policy design and analysis. The analyst conducts sensitivity/scenario analyses and what‐if experiments to reveal the input–output relationships during modeling. However, during these analyses and investigations, the identification of input‐parameter spaces that cause the generation of different SD model behavior patterns is time consuming and susceptible to human bias. Therefore, we propose a metamodel‐based procedure for SD models that considers the necessity for unbiased and automated analysis and insight generation. The approach uses the random forest algorithm for metamodel generation and extracts interpretable IF–THEN rules from the metamodel, thereby identifying input subspaces that generate different qualitative or numerical SD model outputs. We illustrate the proposed approach using two well‐established SD models. These case studies reveal how the model analyst can utilize the proposed method to capture input–output relationships. © 2022 System Dynamics Society.

Suggested Citation

  • Mert Edali, 2022. "Pattern‐oriented analysis of system dynamics models via random forests," System Dynamics Review, System Dynamics Society, vol. 38(2), pages 135-166, April.
  • Handle: RePEc:bla:sysdyn:v:38:y:2022:i:2:p:135-166
    DOI: 10.1002/sdr.1706
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/sdr.1706
    Download Restriction: no

    File URL: https://libkey.io/10.1002/sdr.1706?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    2. Krauss, Christopher & Do, Xuan Anh & Huck, Nicolas, 2017. "Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the S&P 500," European Journal of Operational Research, Elsevier, vol. 259(2), pages 689-702.
    3. Kleijnen, J.P.C., 1995. "Sensitivity analysis and optimization of system dynamics models : Regression analysis and statistical design of experiments," Other publications TiSEM 87ee6ee0-592c-4204-ac50-6, Tilburg University, School of Economics and Management.
    4. R. S. Garfinkel & G. L. Nemhauser, 1969. "The Set-Partitioning Problem: Set Covering with Equality Constraints," Operations Research, INFORMS, vol. 17(5), pages 848-856, October.
    5. Turner, Benjamin L., 2020. "Model laboratories: A quick-start guide for design of simulation experiments for dynamic systems models," Ecological Modelling, Elsevier, vol. 434(C).
    6. Mert Edali & Gönenç Yücel, 2018. "Automated Analysis of Regularities Between Model Parameters and Output Using Support Vector Regression in Conjunction with Decision Trees," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 21(4), pages 1-1.
    7. Guus ten Broeke & George van Voorn & Arend Ligtenberg & Jaap Molenaar, 2021. "The Use of Surrogate Models to Analyse Agent-Based Models," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 24(2), pages 1-3.
    8. Morteza Mashayekhi & Robin Gras, 2017. "Rule Extraction from Decision Trees Ensembles: New Algorithms Based on Heuristic Search and Sparse Group Lasso Methods," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 16(06), pages 1707-1727, November.
    9. Østergård, Torben & Jensen, Rasmus Lund & Maagaard, Steffen Enersen, 2018. "A comparison of six metamodeling techniques applied to building performance simulations," Applied Energy, Elsevier, vol. 211(C), pages 89-103.
    10. Saleh, Mohamed & Oliva, Rogelio & Kampmann, Christian Erik & Davidsen, Pål I., 2010. "A comprehensive analytical approach for policy analysis of system dynamics models," European Journal of Operational Research, Elsevier, vol. 203(3), pages 673-683, June.
    11. Kleijnen, Jack P.C., 2009. "Kriging metamodeling in simulation: A review," European Journal of Operational Research, Elsevier, vol. 192(3), pages 707-716, February.
    12. G. Dosi & M. C. Pereira & M. E. Virgillito, 2018. "On the robustness of the fat-tailed distribution of firm growth rates: a global sensitivity analysis," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 13(1), pages 173-193, April.
    13. Gönenç Yücel & Yaman Barlas, 2011. "Automated parameter specification in dynamic feedback models based on behavior pattern features," System Dynamics Review, System Dynamics Society, vol. 27(2), pages 195-215, April.
    14. Betancourt, José & Bachoc, François & Klein, Thierry & Idier, Déborah & Pedreros, Rodrigo & Rohmer, Jérémy, 2020. "Gaussian process metamodeling of functional-input code for coastal flood hazard assessment," Reliability Engineering and System Safety, Elsevier, vol. 198(C).
    15. repec:hal:spmain:info:hdl:2441/13thfd12aa8rmplfudlgvgahff is not listed on IDEAS
    16. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    17. Christopher Krauss & Anh Do & Nicolas Huck, 2017. "Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the S&P 500," Post-Print hal-01768895, HAL.
    18. Mert Edali & Gönenç Yücel, 2020. "Analysis of an individual‐based influenza epidemic model using random forest metamodels and adaptive sequential sampling," Systems Research and Behavioral Science, Wiley Blackwell, vol. 37(6), pages 936-958, November.
    19. Mustafa Hekimoğlu & Yaman Barlas & Luis Luna-Reyes, 2016. "Sensitivity analysis for models with multiple behavior modes: a method based on behavior pattern measures," System Dynamics Review, System Dynamics Society, vol. 32(3-4), pages 332-362, July.
    20. Bob Walrave, 2016. "Determining intervention thresholds that change output behavior patterns," System Dynamics Review, System Dynamics Society, vol. 32(3-4), pages 261-278, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mert Edali & Gönenç Yücel, 2020. "Analysis of an individual‐based influenza epidemic model using random forest metamodels and adaptive sequential sampling," Systems Research and Behavioral Science, Wiley Blackwell, vol. 37(6), pages 936-958, November.
    2. Karl Naumann-Woleske & Max Sina Knicker & Michael Benzaquen & Jean-Philippe Bouchaud, 2022. "Exploration of the Parameter Space in Macroeconomic Models," Post-Print hal-03797418, HAL.
    3. Turner, Benjamin L., 2020. "Model laboratories: A quick-start guide for design of simulation experiments for dynamic systems models," Ecological Modelling, Elsevier, vol. 434(C).
    4. Weiguang Han & Boyi Zhang & Qianqian Xie & Min Peng & Yanzhao Lai & Jimin Huang, 2023. "Select and Trade: Towards Unified Pair Trading with Hierarchical Reinforcement Learning," Papers 2301.10724, arXiv.org, revised Feb 2023.
    5. Baoqiang Zhan & Shu Zhang & Helen S. Du & Xiaoguang Yang, 2022. "Exploring Statistical Arbitrage Opportunities Using Machine Learning Strategy," Computational Economics, Springer;Society for Computational Economics, vol. 60(3), pages 861-882, October.
    6. Kamaladdin Fataliyev & Aneesh Chivukula & Mukesh Prasad & Wei Liu, 2021. "Stock Market Analysis with Text Data: A Review," Papers 2106.12985, arXiv.org, revised Jul 2021.
    7. Kentaro Imajo & Kentaro Minami & Katsuya Ito & Kei Nakagawa, 2020. "Deep Portfolio Optimization via Distributional Prediction of Residual Factors," Papers 2012.07245, arXiv.org.
    8. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    9. Fischer, Thomas G., 2018. "Reinforcement learning in financial markets - a survey," FAU Discussion Papers in Economics 12/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    10. Knoll, Julian & Stübinger, Johannes & Grottke, Michael, 2017. "Exploiting social media with higher-order Factorization Machines: Statistical arbitrage on high-frequency data of the S&P 500," FAU Discussion Papers in Economics 13/2017, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    11. Moews, Ben & Ibikunle, Gbenga, 2020. "Predictive intraday correlations in stable and volatile market environments: Evidence from deep learning," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 547(C).
    12. Guillaume Coqueret & Tony Guida, 2020. "Training trees on tails with applications to portfolio choice," Post-Print hal-04144665, HAL.
    13. Zhou, Hao & Kalev, Petko S., 2019. "Algorithmic and high frequency trading in Asia-Pacific, now and the future," Pacific-Basin Finance Journal, Elsevier, vol. 53(C), pages 186-207.
    14. Fischer, Thomas & Krauss, Christopher, 2017. "Deep learning with long short-term memory networks for financial market predictions," FAU Discussion Papers in Economics 11/2017, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    15. Mercadier, Mathieu & Lardy, Jean-Pierre, 2019. "Credit spread approximation and improvement using random forest regression," European Journal of Operational Research, Elsevier, vol. 277(1), pages 351-365.
    16. Nazemi, Abdolreza & Rezazadeh, Hani & Fabozzi, Frank J. & Höchstötter, Markus, 2022. "Deep learning for modeling the collection rate for third-party buyers," International Journal of Forecasting, Elsevier, vol. 38(1), pages 240-252.
    17. Alexander Jakob Dautel & Wolfgang Karl Härdle & Stefan Lessmann & Hsin-Vonn Seow, 2020. "Forex exchange rate forecasting using deep recurrent neural networks," Digital Finance, Springer, vol. 2(1), pages 69-96, September.
    18. Flori, Andrea & Regoli, Daniele, 2021. "Revealing Pairs-trading opportunities with long short-term memory networks," European Journal of Operational Research, Elsevier, vol. 295(2), pages 772-791.
    19. Kriebel, Johannes & Stitz, Lennart, 2022. "Credit default prediction from user-generated text in peer-to-peer lending using deep learning," European Journal of Operational Research, Elsevier, vol. 302(1), pages 309-323.
    20. Jian Ni & Yue Xu, 2023. "Forecasting the Dynamic Correlation of Stock Indices Based on Deep Learning Method," Computational Economics, Springer;Society for Computational Economics, vol. 61(1), pages 35-55, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:sysdyn:v:38:y:2022:i:2:p:135-166. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://onlinelibrary.wiley.com/journal/10.1111/0883-7066 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.