IDEAS home Printed from https://ideas.repec.org/a/bla/metroe/v55y2004i2-3p195-218.html
   My bibliography  Save this article

Induction: From Kolmogorov and Solomonoff to De Finetti and Back to Kolmogorov

Author

Listed:
  • John J. McCall

Abstract

This paper compares the solutions to “the induction problem” by Kolmogorov, de Finetti, and Solomonoff. Brief sketches of the intellectual history of de Finetti and Kolmogorov are also composed. Kolmogorov's contributions to information theory culminated in his notion of algorithmic complexity. The development of algorithmic complexity was inspired by information theory and randomness. Kolmogorov's best‐known contribution was the axiomatization of probability in 1933. Its influence on probability and statistics was swift, dramatic, and fundamental. However, Kolmogorov was not satisfied by his treatment of the frequency aspect of his creation. This in time gave rise to Kolmogorov complexity. De Finetti, on the other hand, had a profound vision early in his life which was encapsulated in his exchangeability theorem. This insight simultaneously resolved a fundamental philosophical conundrum—Hume's problem, and provided the bricks and mortar for de Finetti's constructive probabilistic theory. Most of his subsequent research involved extensions of his representation theorem. De Finetti was against determinism and celebrated quantum theory, while Kolmogorov was convinced that in every seemingly indeterministic manifestation there lurked a hidden deterministic mechanism. Solomonoff introduced algorithmic complexity independently of Kolmogorov and Chaitin. Solomonoff's motivation was firmly focused on induction. His interest in induction was to a marked extent sparked by Keynes’ 1921 seminal book. This interest in induction has never faltered, remaining prominent in his most recent research. The decisive connection between de Finetti and Kolmogorov was their lifelong interest in the frequency aspect of induction. Kolmogorov's solution to the problem was algorithmic complexity. De Finetti's solution to his frequency problem occurred early in his career with the discovery of the representation theorem. In this paper, we try to explain these solutions and mention related topics which captured the interest of these giants.

Suggested Citation

  • John J. McCall, 2004. "Induction: From Kolmogorov and Solomonoff to De Finetti and Back to Kolmogorov," Metroeconomica, Wiley Blackwell, vol. 55(2‐3), pages 195-218, May.
    • Handle: RePEc:bla:metroe:v:55:y:2004:i:2-3:p:195-218
    DOI: 10.1111/j.0026-1386.2004.00190.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.0026-1386.2004.00190.x
    Download Restriction: no
    File URL: https://libkey.io/10.1111/j.0026-1386.2004.00190.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Geweke, John, 1989. "Bayesian Inference in Econometric Models Using Monte Carlo Integration," Econometrica, Econometric Society, vol. 57(6), pages 1317-1339, November.
    2. Muliere, P. & Secchi, P. & Walker, S. G., 2000. "Urn schemes and reinforced random walks," Stochastic Processes and their Applications, Elsevier, vol. 88(1), pages 59-78, July.
    3. Poirier, Dale J, 1988. "Frequentist and Subjectivist Perspectives on the Problems of Model Building in Economics," Journal of Economic Perspectives, American Economic Association, vol. 2(1), pages 121-144, Winter.
    4. John Geweke, 1999. "Using Simulation Methods for Bayesian Econometric Models," Computing in Economics and Finance 1999 832, Society for Computational Economics.
    5. John Geweke, 1999. "Using simulation methods for bayesian econometric models: inference, development,and communication," Econometric Reviews, Taylor & Francis Journals, vol. 18(1), pages 1-73.
    6. Stephen G. Walker & Paul Damien & PuruShottam W. Laud & Adrian F. M. Smith, 1999. "Bayesian Nonparametric Inference for Random Distributions and Related Functions," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 61(3), pages 485-527.
    7. Zellner, A., 1988. "Optimal Information-Processing And Bayes' Theorem," Papers m8803, Southern California - Department of Economics.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hajargasht, Gholamreza & Rao, D.S. Prasada, 2019. "Multilateral index number systems for international price comparisons: Properties, existence and uniqueness," Journal of Mathematical Economics, Elsevier, vol. 83(C), pages 36-47.
    2. John Geweke & Joel Horowitz & M. Hashem Pesaran, 2006. "Econometrics: A Bird’s Eye View," CESifo Working Paper Series 1870, CESifo.
    3. Gael M. Martin & David T. Frazier & Christian P. Robert, 2020. "Computing Bayes: Bayesian Computation from 1763 to the 21st Century," Monash Econometrics and Business Statistics Working Papers 14/20, Monash University, Department of Econometrics and Business Statistics.
    4. Lennart F. Hoogerheide & Johan F. Kaashoek, 2004. "Functional Approximations to Likelihoods/Posterior Densities: A Neural Network Approach to Efficient Sampling," Computing in Economics and Finance 2004 74, Society for Computational Economics.
    5. Isaiah Andrews & Jesse M. Shapiro, 2021. "A Model of Scientific Communication," Econometrica, Econometric Society, vol. 89(5), pages 2117-2142, September.
    6. Liesenfeld, Roman & Richard, Jean-François, 2008. "Improving MCMC, using efficient importance sampling," Computational Statistics & Data Analysis, Elsevier, vol. 53(2), pages 272-288, December.
    7. Karlsson, Sune, 2013. "Forecasting with Bayesian Vector Autoregression," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 791-897, Elsevier.
    8. Lahiri, Kajal & Gao, Jian, 2002. "Bayesian analysis of nested logit model by Markov chain Monte Carlo," Journal of Econometrics, Elsevier, vol. 111(1), pages 103-133, November.
    9. HOOGERHEIDE, Lennart F. & VAN DIJK, Herman K. & VAN OEST, Rutger D., 2007. "Simulation based Bayesian econometric inference: principles and some recent computational advances," LIDAM Discussion Papers CORE 2007015, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    10. Matthieu Droumaguet & Tomasz Wozniak, 2012. "Bayesian Testing of Granger Causality in Markov-Switching VARs," Economics Working Papers ECO2012/06, European University Institute.
    11. Villani, Mattias, 2005. "Bayesian Inference of General Linear Restrictions on the Cointegration Space," Working Paper Series 189, Sveriges Riksbank (Central Bank of Sweden).
    12. Villani, Mattias & Larsson, Rolf, 2004. "The Multivariate Split Normal Distribution and Asymmetric Principal Components Analysis," Working Paper Series 175, Sveriges Riksbank (Central Bank of Sweden).
    13. Fuentes-Albero, Cristina & Melosi, Leonardo, 2013. "Methods for computing marginal data densities from the Gibbs output," Journal of Econometrics, Elsevier, vol. 175(2), pages 132-141.
    14. Woźniak, Tomasz, 2015. "Testing causality between two vectors in multivariate GARCH models," International Journal of Forecasting, Elsevier, vol. 31(3), pages 876-894.
    15. Brownstone, David, 2001. "Discrete Choice Modeling for Transportation," University of California Transportation Center, Working Papers qt29v7d1pk, University of California Transportation Center.
    16. Hoogerheide, Lennart F. & Kaashoek, Johan F. & van Dijk, Herman K., 2007. "On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: An application of flexible sampling methods using neural networks," Journal of Econometrics, Elsevier, vol. 139(1), pages 154-180, July.
    17. van Dijk, H.K., 2002. "On Bayesian structural inference in a simultaneous equation model," Econometric Institute Research Papers EI 2002-10, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    18. Ardia, David & Baştürk, Nalan & Hoogerheide, Lennart & van Dijk, Herman K., 2012. "A comparative study of Monte Carlo methods for efficient evaluation of marginal likelihood," Computational Statistics & Data Analysis, Elsevier, vol. 56(11), pages 3398-3414.
    19. Ross Doppelt & Keith O'Hara, 2018. "Bayesian Estimation of Fractionally Integrated Vector Autoregressions and an Application to Identified Technology Shocks," 2018 Meeting Papers 1212, Society for Economic Dynamics.
    20. David Ardia & Lennart Hoogerheide & Herman K. van Dijk, 2009. "To Bridge, to Warp or to Wrap? A Comparative Study of Monte Carlo Methods for Efficient Evaluation of Marginal Likelihoods," Tinbergen Institute Discussion Papers 09-017/4, Tinbergen Institute.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:metroe:v:55:y:2004:i:2-3:p:195-218. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0026-1386 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.