IDEAS home Printed from https://ideas.repec.org/p/ukc/ukcedp/1504.html
   My bibliography  Save this paper

A Practical, Universal, Information Criterion over Nth Order Markov Processes

Author

Listed:
  • Sylvain Barde

    ()

Abstract

The recent increase in the breath of computational methodologies has been matched with a corresponding increase in the difficulty of comparing the relative explanatory power of models from different methodological lineages. In order to help address this problem a universal information criterion (UIC) is developed that is analogous to the Akaike information criterion (AIC) in its theoretical derivation and yet can be applied to any model able to generate simulated or predicted data, regardless of its methodology. Both the AIC and proposed UIC rely on the Kullback-Leibler (KL) distance between model predictions and real data as a measure of prediction accuracy. Instead of using the maximum likelihood approach like the AIC, the proposed UIC relies instead on the literal interpretation of the KL distance as the inefficiency of compressing real data using modelled probabilities, and therefore uses the output of a universal compression algorithm to obtain an estimate of the KL distance. Several Monte Carlo tests are carried out in order to (a) confirm the performance of the algorithm and (b) evaluate the ability of the UIC to identify the true data-generating process from a set of alternative models.

Suggested Citation

  • Sylvain Barde, 2015. "A Practical, Universal, Information Criterion over Nth Order Markov Processes," Studies in Economics 1504, School of Economics, University of Kent.
  • Handle: RePEc:ukc:ukcedp:1504
    as

    Download full text from publisher

    File URL: https://www.kent.ac.uk/economics/repec/1504.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Dimitris Politis & Halbert White, 2004. "Automatic Block-Length Selection for the Dependent Bootstrap," Econometric Reviews, Taylor & Francis Journals, vol. 23(1), pages 53-70.
    2. Gourieroux, Christian & Monfort, Alain, 1993. "Simulation-based inference : A survey with special reference to panel data models," Journal of Econometrics, Elsevier, vol. 59(1-2), pages 5-33, September.
    3. Francesco Lamperti, 2015. "An Information Theoretic Criterion for Empirical Validation of Time Series Models," LEM Papers Series 2015/02, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lamperti, F. & Dosi, G. & Napoletano, M. & Roventini, A. & Sapio, A., 2018. "Faraway, So Close: Coupled Climate and Economic Dynamics in an Agent-based Integrated Assessment Model," Ecological Economics, Elsevier, vol. 150(C), pages 315-339.
    2. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    3. Francesco Lamperti, 2016. "Empirical Validation of Simulated Models through the GSL-div: an Illustrative Application," LEM Papers Series 2016/18, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    4. Sylvain Barde & Sander van der Hoog, 2017. "An empirical validation protocol for large-scale agent-based models," Studies in Economics 1712, School of Economics, University of Kent.
    5. Alexandru Mandes & Peter Winker, 2017. "Complexity and model comparison in agent based modeling of financial markets," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 12(3), pages 469-506, October.
    6. Barde, Sylvain, 2016. "Direct comparison of agent-based models of herding in financial markets," Journal of Economic Dynamics and Control, Elsevier, vol. 73(C), pages 329-353.
    7. Francesco Lamperti, 2015. "An Information Theoretic Criterion for Empirical Validation of Time Series Models," LEM Papers Series 2015/02, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    8. Sylvain Barde, 2015. "Direct calibration and comparison of agent-based herding models of financial markets," Studies in Economics 1507, School of Economics, University of Kent.
    9. Sander van der Hoog, 2017. "Deep Learning in (and of) Agent-Based Models: A Prospectus," Papers 1706.06302, arXiv.org.

    More about this item

    Keywords

    AIC; Minimum description length; Model selection;

    JEL classification:

    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ukc:ukcedp:1504. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Tracey Girling). General contact details of provider: https://www.kent.ac.uk/economics/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.