IDEAS home Printed from https://ideas.repec.org/a/spr/jeicoo/v13y2018i1d10.1007_s11403-017-0206-3.html
   My bibliography  Save this article

Empirical validation of simulated models through the GSL-div: an illustrative application

Author

Listed:
  • Francesco Lamperti

    (Scuola Superiore Sant’Anna)

Abstract

A major concern about the use of simulation models regards their relationship with the empirical data. The identification of a suitable indicator quantifying the distance between the model and the data would help and guide model selection and output validation. This paper proposes the use of a new criterion, called GSL-div and developed in Lamperti (Econ Stat, 2017. https://doi.org/10.1016/j.ecosta.2017.01.006 ), to assess the degree of similarity between the dynamics observed in the data and those generated by the numerical simulation of models. As an illustrative application, this approach is used to distinguish between different versions of the well known asset pricing model with heterogeneous beliefs proposed in Brock and Hommes (J Econ Dyn Control 22(8–9):1235–1274, 1998. https://doi.org/10.1016/S0165-1889(98)00011-6 ). Once the discrimination ability of the GSL-div is proved, model’s dynamics are directly compared with actual data coming from two major stock market indexes (EuroSTOXX 50 for Europe and CSI 300 for China). Results show that the model, once calibrated, is fairly able to track the evolution of both the two indexes, even though a better fit is reported for the Chinese stock market. However, I also find that many different combinations of traders’ behavioural rules are compatible with the same observed dynamics. Within this heterogeneity, an emerging common trait is found: to be empirically valid, the model has to account for a strong trend following component, which might either come from a unique trend type that heavily extrapolates information from past observations or the combinations of different types with milder, or even opposite, attitudes towards the trend.

Suggested Citation

  • Francesco Lamperti, 2018. "Empirical validation of simulated models through the GSL-div: an illustrative application," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 13(1), pages 143-171, April.
  • Handle: RePEc:spr:jeicoo:v:13:y:2018:i:1:d:10.1007_s11403-017-0206-3
    DOI: 10.1007/s11403-017-0206-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11403-017-0206-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11403-017-0206-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Barde, Sylvain, 2016. "Direct comparison of agent-based models of herding in financial markets," Journal of Economic Dynamics and Control, Elsevier, vol. 73(C), pages 329-353.
    2. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    3. Chiarella, Carl & Iori, Giulia, 2009. "The impact of heterogeneous trading rules on the limit order book and order flows," Journal of Economic Dynamics and Control, Elsevier, vol. 33(3), pages 525-537.
    4. Sandrine Jacob Leal & Mauro Napoletano & Andrea Roventini & Giorgio Fagiolo, 2016. "Rock around the clock: An agent-based model of low- and high-frequency trading," Journal of Evolutionary Economics, Springer, vol. 26(1), pages 49-76, March.
    5. Recchioni, Maria Cristina & Tedeschi, Gabriele & Gallegati, Mauro, 2015. "A calibration procedure for analyzing stock price dynamics in an agent-based framework," Journal of Economic Dynamics and Control, Elsevier, vol. 60(C), pages 1-25.
    6. repec:hal:spmain:info:hdl:2441/3qv4spsglp8tmorvev1h0duo4p is not listed on IDEAS
    7. Paul Windrum & Giorgio Fagiolo & Alessio Moneta, 2007. "Empirical Validation of Agent-Based Models: Alternatives and Prospects," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 10(2), pages 1-8.
    8. repec:hal:spmain:info:hdl:2441/7kr9gv74ut9ngo58gia97t83i7 is not listed on IDEAS
    9. Canova, Fabio & Sala, Luca, 2009. "Back to square one: Identification issues in DSGE models," Journal of Monetary Economics, Elsevier, vol. 56(4), pages 431-449, May.
    10. Matteo Richiardi & Roberto Leombruni & Nicole J. Saam & Michele Sonnessa, 2006. "A Common Protocol for Agent-Based Social Simulation," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 9(1), pages 1-15.
    11. Thomas Lux & Michele Marchesi, 2000. "Volatility Clustering In Financial Markets: A Microsimulation Of Interacting Agents," International Journal of Theoretical and Applied Finance (IJTAF), World Scientific Publishing Co. Pte. Ltd., vol. 3(04), pages 675-702.
    12. Paolo Pellizzari & Arianna Forno, 2007. "A comparison of different trading protocols in an agent-based market," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 2(1), pages 27-43, June.
    13. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    14. Dosi, Giovanni & Fagiolo, Giorgio & Napoletano, Mauro & Roventini, Andrea, 2013. "Income distribution, credit and fiscal policies in an agent-based Keynesian model," Journal of Economic Dynamics and Control, Elsevier, vol. 37(8), pages 1598-1625.
    15. Grazzini, Jakob & Richiardi, Matteo G. & Tsionas, Mike, 2017. "Bayesian estimation of agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 77(C), pages 26-47.
    16. Boswijk, H. Peter & Hommes, Cars H. & Manzan, Sebastiano, 2007. "Behavioral heterogeneity in stock prices," Journal of Economic Dynamics and Control, Elsevier, vol. 31(6), pages 1938-1970, June.
    17. Dosi, Giovanni & Fagiolo, Giorgio & Roventini, Andrea, 2010. "Schumpeter meeting Keynes: A policy-friendly model of endogenous growth and business cycles," Journal of Economic Dynamics and Control, Elsevier, vol. 34(9), pages 1748-1767, September.
    18. Giannone, Domenico & Reichlin, Lucrezia & Sala, Luca, 2006. "VARs, common factors and the empirical validation of equilibrium business cycle models," Journal of Econometrics, Elsevier, vol. 132(1), pages 257-279, May.
    19. Dosi, Giovanni & Fagiolo, Giorgio & Napoletano, Mauro & Roventini, Andrea & Treibich, Tania, 2015. "Fiscal and monetary policies in complex evolving economies," Journal of Economic Dynamics and Control, Elsevier, vol. 52(C), pages 166-189.
    20. Peter Winker & Manfred Gilli & Vahidin Jeleskovic, 2007. "An objective function for simulation based inference on exchange rate data," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 2(2), pages 125-145, December.
    21. LeBaron Blake & Winker Peter, 2008. "Introduction to the Special Issue on Agent-Based Models for Economic Policy Advice," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 228(2-3), pages 141-148, April.
    22. repec:hal:spmain:info:hdl:2441/20hflp7eqn97boh50no50tv67n is not listed on IDEAS
    23. Anna, Petrenko, 2016. "Мaркування готової продукції як складова частина інформаційного забезпечення маркетингової діяльності підприємств овочепродуктового підкомплексу," Agricultural and Resource Economics: International Scientific E-Journal, Agricultural and Resource Economics: International Scientific E-Journal, vol. 2(1), March.
    24. Horst Hanusch & Andreas Pyka (ed.), 2007. "Elgar Companion to Neo-Schumpeterian Economics," Books, Edward Elgar Publishing, number 2973.
    25. Amilon, Henrik, 2008. "Estimation of an adaptive stock market model with heterogeneous agents," Journal of Empirical Finance, Elsevier, vol. 15(2), pages 342-362, March.
    26. repec:hal:spmain:info:hdl:2441/5fafm6me7k8omq5jbo61urqq27 is not listed on IDEAS
    27. William A. Brock & Cars H. Hommes, 1997. "A Rational Route to Randomness," Econometrica, Econometric Society, vol. 65(5), pages 1059-1096, September.
    28. Guerini, Mattia & Moneta, Alessio, 2017. "A method for agent-based models validation," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 125-141.
    29. Abhijit V. Banerjee, 1992. "A Simple Model of Herd Behavior," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 107(3), pages 797-817.
    30. Simone Alfarano & Thomas Lux & Friedrich Wagner, 2005. "Estimation of Agent-Based Models: The Case of an Asymmetric Herding Model," Computational Economics, Springer;Society for Computational Economics, vol. 26(1), pages 19-49, August.
    31. Brock, William A. & Hommes, Cars H., 1998. "Heterogeneous beliefs and routes to chaos in a simple asset pricing model," Journal of Economic Dynamics and Control, Elsevier, vol. 22(8-9), pages 1235-1274, August.
    32. Gilli, M. & Winker, P., 2003. "A global optimization heuristic for estimating agent based models," Computational Statistics & Data Analysis, Elsevier, vol. 42(3), pages 299-312, March.
    33. William A. Brock & Cars H. Hommes, 2001. "A Rational Route to Randomness," Chapters, in: W. D. Dechert (ed.), Growth Theory, Nonlinear Dynamics and Economic Modelling, chapter 16, pages 402-438, Edward Elgar Publishing.
    34. Gualdi, Stanislao & Tarzia, Marco & Zamponi, Francesco & Bouchaud, Jean-Philippe, 2015. "Tipping points in macroeconomic agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 50(C), pages 29-61.
    35. Francesco Lamperti, 2015. "An Information Theoretic Criterion for Empirical Validation of Time Series Models," LEM Papers Series 2015/02, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    36. Alfarano, Simone & Lux, Thomas & Wagner, Friedrich, 2006. "Estimation of a simple agent-based model of financial markets: An application to Australian stock and foreign exchange data," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 370(1), pages 38-42.
    37. Bianchi, Carlo & Cirillo, Pasquale & Gallegati, Mauro & Vagliasindi, Pietro A., 2008. "Validation in agent-based models: An investigation on the CATS model," Journal of Economic Behavior & Organization, Elsevier, vol. 67(3-4), pages 947-964, September.
    38. Carlo Bianchi & Pasquale Cirillo & Mauro Gallegati & Pietro Vagliasindi, 2007. "Validating and Calibrating Agent-Based Models: A Case Study," Computational Economics, Springer;Society for Computational Economics, vol. 30(3), pages 245-264, October.
    39. Grazzini, Jakob & Richiardi, Matteo, 2015. "Estimation of ergodic agent-based models by simulated minimum distance," Journal of Economic Dynamics and Control, Elsevier, vol. 51(C), pages 148-165.
    40. Robert Marks, 2007. "Validating Simulation Models: A General Framework and Four Applied Examples," Computational Economics, Springer;Society for Computational Economics, vol. 30(3), pages 265-290, October.
    41. Franke, Reiner & Westerhoff, Frank, 2012. "Structural stochastic volatility in asset pricing dynamics: Estimation and model contest," Journal of Economic Dynamics and Control, Elsevier, vol. 36(8), pages 1193-1211.
    42. Jakob Grazzini, 2012. "Analysis of the Emergent Properties: Stationarity and Ergodicity," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 15(2), pages 1-7.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dosi, Giovanni & Lamperti, Francesco & Mazzucato, Mariana & Napoletano, Mauro & Roventini, Andrea, 2023. "Mission-oriented policies and the “Entrepreneurial State” at work: An agent-based exploration," Journal of Economic Dynamics and Control, Elsevier, vol. 151(C).
    2. Platt, Donovan, 2020. "A comparison of economic agent-based model calibration methods," Journal of Economic Dynamics and Control, Elsevier, vol. 113(C).
    3. Barde, Sylvain, 2020. "Macroeconomic simulation comparison with a multivariate extension of the Markov information criterion," Journal of Economic Dynamics and Control, Elsevier, vol. 111(C).
    4. Lamperti, Francesco & Bosetti, Valentina & Roventini, Andrea & Tavoni, Massimo & Treibich, Tania, 2021. "Three green financial policies to address climate risks," Journal of Financial Stability, Elsevier, vol. 54(C).
    5. Mérő, Bence & Borsos, András & Hosszú, Zsuzsanna & Oláh, Zsolt & Vágó, Nikolett, 2023. "A high-resolution, data-driven agent-based model of the housing market," Journal of Economic Dynamics and Control, Elsevier, vol. 155(C).
    6. Seri, Raffaello & Martinoli, Mario & Secchi, Davide & Centorrino, Samuele, 2021. "Model calibration and validation via confidence sets," Econometrics and Statistics, Elsevier, vol. 20(C), pages 62-86.
    7. Andrea Vandin & Daniele Giachini & Francesco Lamperti & Francesca Chiaromonte, 2021. "Automated and Distributed Statistical Analysis of Economic Agent-Based Models," Papers 2102.05405, arXiv.org, revised Nov 2023.
    8. Kukacka, Jiri & Sacht, Stephen, 2023. "Estimation of heuristic switching in behavioral macroeconomic models," Journal of Economic Dynamics and Control, Elsevier, vol. 146(C).
    9. repec:hal:spmain:info:hdl:2441/401t6job098n79ch91o9giov9d is not listed on IDEAS
    10. Giovanni Dosi & Andrea Roventini, 2019. "More is different ... and complex! the case for agent-based macroeconomics," Journal of Evolutionary Economics, Springer, vol. 29(1), pages 1-37, March.
    11. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    12. Campiglio, Emanuele & Lamperti, Francesco & Terranova, Roberta, 2023. "Believe me when I say green! Heterogeneous expectations and climate policy uncertainty," LSE Research Online Documents on Economics 119258, London School of Economics and Political Science, LSE Library.
    13. Siyan Chen & Saul Desiderio, 2022. "Calibration of Agent-Based Models by Means of Meta-Modeling and Nonparametric Regression," Computational Economics, Springer;Society for Computational Economics, vol. 60(4), pages 1457-1478, December.
    14. Kukacka, Jiri & Kristoufek, Ladislav, 2021. "Does parameterization affect the complexity of agent-based models?," Journal of Economic Behavior & Organization, Elsevier, vol. 192(C), pages 324-356.
    15. Francesco Lamperti & Andrea Roventini, 2022. "Beyond climate economics orthodoxy: impacts and policies in the agent-based integrated-assessment DSK model," European Journal of Economics and Economic Policies: Intervention, Edward Elgar Publishing, vol. 19(3), pages 357-380, December.
    16. Vandin, Andrea & Giachini, Daniele & Lamperti, Francesco & Chiaromonte, Francesca, 2022. "Automated and distributed statistical analysis of economic agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 143(C).
    17. Emanuele Campiglio & Francesco Lamperti & Roberta Terranova, 2023. "Believe me when I say green! Heterogeneous expectations and climate policy uncertainty," LEM Papers Series 2023/12, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    18. Campiglio, Emanuele & Lamperti, Francesco & Terranova, Roberta, 2023. "Believe me when I say green! Heterogeneous expectations and climate policy uncertainty," LSE Research Online Documents on Economics 119257, London School of Economics and Political Science, LSE Library.
    19. Andrea Vandin & Daniele Giachini & Francesco Lamperti & Francesca Chiaromonte, 2020. "Automated and Distributed Statistical Analysis of Economic Agent-Based Models," LEM Papers Series 2020/31, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    20. Siyan Chen & Saul Desiderio, 2022. "A Regression-Based Calibration Method for Agent-Based Models," Computational Economics, Springer;Society for Computational Economics, vol. 59(2), pages 687-700, February.
    21. Donovan Platt, 2022. "Bayesian Estimation of Economic Simulation Models Using Neural Networks," Computational Economics, Springer;Society for Computational Economics, vol. 59(2), pages 599-650, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Francesco Lamperti, 2016. "Empirical Validation of Simulated Models through the GSL-div: an Illustrative Application," LEM Papers Series 2016/18, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    2. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    3. repec:hal:spmain:info:hdl:2441/20hflp7eqn97boh50no50tv67n is not listed on IDEAS
    4. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    5. repec:hal:spmain:info:hdl:2441/13thfd12aa8rmplfudlgvgahff is not listed on IDEAS
    6. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    7. Lamperti, Francesco, 2018. "An information theoretic criterion for empirical validation of simulation models," Econometrics and Statistics, Elsevier, vol. 5(C), pages 83-106.
    8. Giorgio Fagiolo & Andrea Roventini, 2017. "Macroeconomic Policy in DSGE and Agent-Based Models Redux: New Developments and Challenges Ahead," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 20(1), pages 1-1.
    9. Francesco Lamperti, 2015. "An Information Theoretic Criterion for Empirical Validation of Time Series Models," LEM Papers Series 2015/02, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    10. repec:hal:spmain:info:hdl:2441/dcditnq6282sbu1u151qe5p7f is not listed on IDEAS
    11. Donovan Platt, 2019. "A Comparison of Economic Agent-Based Model Calibration Methods," Papers 1902.05938, arXiv.org.
    12. Donovan Platt & Tim Gebbie, 2016. "The Problem of Calibrating an Agent-Based Model of High-Frequency Trading," Papers 1606.01495, arXiv.org, revised Mar 2017.
    13. Guerini, Mattia & Moneta, Alessio, 2017. "A method for agent-based models validation," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 125-141.
    14. Platt, Donovan, 2020. "A comparison of economic agent-based model calibration methods," Journal of Economic Dynamics and Control, Elsevier, vol. 113(C).
    15. Tubbenhauer, Tobias & Fieberg, Christian & Poddig, Thorsten, 2021. "Multi-agent-based VaR forecasting," Journal of Economic Dynamics and Control, Elsevier, vol. 131(C).
    16. Mario Martinoli & Alessio Moneta & Gianluca Pallante, 2022. "Calibration and Validation of Macroeconomic Simulation Models by Statistical Causal Search," LEM Papers Series 2022/33, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    17. Kukacka, Jiri & Barunik, Jozef, 2017. "Estimation of financial agent-based models with simulated maximum likelihood," Journal of Economic Dynamics and Control, Elsevier, vol. 85(C), pages 21-45.
    18. Vandin, Andrea & Giachini, Daniele & Lamperti, Francesco & Chiaromonte, Francesca, 2022. "Automated and distributed statistical analysis of economic agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 143(C).
    19. Grazzini, Jakob & Richiardi, Matteo G. & Tsionas, Mike, 2017. "Bayesian estimation of agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 77(C), pages 26-47.
    20. Barde, Sylvain, 2020. "Macroeconomic simulation comparison with a multivariate extension of the Markov information criterion," Journal of Economic Dynamics and Control, Elsevier, vol. 111(C).
    21. Donovan Platt, 2022. "Bayesian Estimation of Economic Simulation Models Using Neural Networks," Computational Economics, Springer;Society for Computational Economics, vol. 59(2), pages 599-650, February.
    22. Blaurock, Ivonne & Schmitt, Noemi & Westerhoff, Frank, 2018. "Market entry waves and volatility outbursts in stock markets," Journal of Economic Behavior & Organization, Elsevier, vol. 153(C), pages 19-37.
    23. Grazzini, Jakob & Richiardi, Matteo, 2015. "Estimation of ergodic agent-based models by simulated minimum distance," Journal of Economic Dynamics and Control, Elsevier, vol. 51(C), pages 148-165.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jeicoo:v:13:y:2018:i:1:d:10.1007_s11403-017-0206-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.