Macroeconomic simulation comparison with a multivariate extension of the Markov Information Criterion

Comparison of macroeconomic simulation models, particularly agent-based models (ABMs), with more traditional approaches such as VAR and DSGE models has long been identified as an important yet problematic issue in the literature. This is due to the fact that many such simulations have been developed following the great recession with a clear aim to inform policy, yet the methodological tools required for validating these models on empirical data are still in their infancy. The paper aims to address this issue by developing and testing a comparison framework for macroeconomic simulation models based on a multivariate extension of the Markov Information Criterion (MIC) originally developed in Barde (2017). The MIC is designed to measure the informational distance between a set of models and some empirical data by mapping the simulated data to the markov transition matrix of the underlying data generating process, and is proven to perform optimally (i.e. the measurement is unbiased in expectation) for all models reducible to a markov process. As a result, not only can the MIC provide an accurate measure of distance solely on the basis of simulated data, but it can do it for a very wide class of data generating processes. The paper first presents the strategies adopted to address the computational challenges that arise from extending the methodology to multivariate settings and validates the extension on VAR and DGSE models. The paper then carries out a comparison of the benchmark ABM of Caiani et al. (2016) and the DGSE framework of Smets and Wouters (2007), which to our knowledge, is the first direct comparison between a macroeconomic ABM and a DGSE model.