IDEAS home Printed from
   My bibliography  Save this article

An efficient algorithm to compute maximum entropy densities


  • D. Ormoneit
  • H. White


We describe an algorithm to efficiently compute maximum entropy densities, i.e. densities maximizing the Shannon entropy - [image omitted] under a set of constraints [image omitted] . Our method is based on an algorithm by Zellner and Highfield, which has been found not to converge under a variety of circumstances. To demonstrate that our method overcomes these difficulties, we conduct numerous experiments for the special case gi(x) = xi, n = 4. An extensive table of results for this case and computer code are available on the World Wide Web

Suggested Citation

  • D. Ormoneit & H. White, 1999. "An efficient algorithm to compute maximum entropy densities," Econometric Reviews, Taylor & Francis Journals, vol. 18(2), pages 127-140.
  • Handle: RePEc:taf:emetrv:v:18:y:1999:i:2:p:127-140 DOI: 10.1080/07474939908800436

    Download full text from publisher

    File URL:
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    1. Geweke, John, 1989. "Bayesian Inference in Econometric Models Using Monte Carlo Integration," Econometrica, Econometric Society, vol. 57(6), pages 1317-1339, November.
    2. Chib, Siddhartha & Greenberg, Edward, 1996. "Markov Chain Monte Carlo Simulation Methods in Econometrics," Econometric Theory, Cambridge University Press, vol. 12(03), pages 409-431, August.
    3. Engle, Robert F & Hendry, David F & Richard, Jean-Francois, 1983. "Exogeneity," Econometrica, Econometric Society, vol. 51(2), pages 277-304, March.
    4. Zellner, Arnold, 1985. "Bayesian Econometrics," Econometrica, Econometric Society, vol. 53(2), pages 253-269, March.
    5. Steel, Mark F. J. & Richard, Jean-Francois, 1991. "Bayesian multivariate exogeneity analysis : An application to a UK money demand equation," Journal of Econometrics, Elsevier, vol. 49(1-2), pages 239-274.
    6. Anglin, Paul M & Gencay, Ramazan, 1996. "Semiparametric Estimation of a Hedonic Price Function," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 11(6), pages 633-648, Nov.-Dec..
    7. John F. Geweke & Michael P. Keane, 1997. "Mixture of normals probit models," Staff Report 237, Federal Reserve Bank of Minneapolis.
    8. Geweke, John, 1989. "Exact predictive densities for linear models with arch disturbances," Journal of Econometrics, Elsevier, vol. 40(1), pages 63-86, January.
    9. Poirier, Dale J., 1997. "Comparing and choosing between two models with a third model in the background," Journal of Econometrics, Elsevier, vol. 78(2), pages 139-151, June.
    10. Geweke, John, 1996. "Monte carlo simulation and numerical integration," Handbook of Computational Economics,in: H. M. Amman & D. A. Kendrick & J. Rust (ed.), Handbook of Computational Economics, edition 1, volume 1, chapter 15, pages 731-800 Elsevier.
    11. John F. Geweke, 1991. "Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments," Staff Report 148, Federal Reserve Bank of Minneapolis.
    12. Geweke, John, 1988. "Antithetic acceleration of Monte Carlo integration in Bayesian inference," Journal of Econometrics, Elsevier, vol. 38(1-2), pages 73-89.
    13. Kloek, Tuen & van Dijk, Herman K, 1978. "Bayesian Estimates of Equation System Parameters: An Application of Integration by Monte Carlo," Econometrica, Econometric Society, vol. 46(1), pages 1-19, January.
    14. Kiefer, Nicholas M. & Salmon, Mark, 1983. "Testing normality in econometric models," Economics Letters, Elsevier, vol. 11(1-2), pages 123-127.
    15. Poirier, Dale J, 1988. "Frequentist and Subjectivist Perspectives on the Problems of Model Building in Economics," Journal of Economic Perspectives, American Economic Association, vol. 2(1), pages 121-144, Winter.
    16. repec:cup:etheor:v:12:y:1996:i:3:p:409-31 is not listed on IDEAS
    17. Roberts, G. O. & Smith, A. F. M., 1994. "Simple conditions for the convergence of the Gibbs sampler and Metropolis-Hastings algorithms," Stochastic Processes and their Applications, Elsevier, vol. 49(2), pages 207-216, February.
    18. Hans M. Amman & David A. Kendrick, . "Computational Economics," Online economics textbooks, SUNY-Oswego, Department of Economics, number comp1, March.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Thanasis Stengos & Ximing Wu, 2010. "Information-Theoretic Distribution Test with Application to Normality," Econometric Reviews, Taylor & Francis Journals, vol. 29(3), pages 307-329.
    2. Johannes Vilsmeier, 2011. "Updating the Option Implied Probability of Default Methodology," Working Papers 107, Bavarian Graduate Program in Economics (BGPE).
    3. Rockinger, Michael & Jondeau, Eric, 2002. "Entropy densities with an application to autoregressive conditional skewness and kurtosis," Journal of Econometrics, Elsevier, vol. 106(1), pages 119-142, January.
    4. Holly, Alberto & Monfort, Alain & Rockinger, Michael, 2011. "Fourth order pseudo maximum likelihood methods," Journal of Econometrics, Elsevier, vol. 162(2), pages 278-293, June.
    5. Jesse Tack & Ardian Harri & Keith Coble, 2012. "More than Mean Effects: Modeling the Effect of Climate on the Higher Order Moments of Crop Yields," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 94(5), pages 1037-1054.
    6. Miller, Douglas J. & Liu, Wei-han, 2002. "On the recovery of joint distributions from limited information," Journal of Econometrics, Elsevier, vol. 107(1-2), pages 259-274, March.
    7. Usta, Ilhan & Kantar, Yeliz Mert, 2011. "On the performance of the flexible maximum entropy distributions within partially adaptive estimation," Computational Statistics & Data Analysis, Elsevier, vol. 55(6), pages 2172-2182, June.
    8. Wu, Ximing, 2003. "Calculation of maximum entropy densities with application to income distribution," Journal of Econometrics, Elsevier, vol. 115(2), pages 347-354, August.
    9. Wu, Ximing & Perloff, Jeffrey M., 2004. "China's Income Distribution Over Time: Reasons for Rising Inequality," Institute for Research on Labor and Employment, Working Paper Series qt9jw2v939, Institute of Industrial Relations, UC Berkeley.
    10. repec:hal:journl:peer-00815562 is not listed on IDEAS
    11. Vilsmeier, Johannes, 2014. "Updating the option implied probability of default methodology," Discussion Papers 43/2014, Deutsche Bundesbank.
    12. Herrmann Klaus & Fischer Matthias, 2010. "An Alternative Maximum Entropy Model for Time-Varying Moments with Application to Financial Returns," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 14(3), pages 1-23, May.

    More about this item


    Density Estimation; Maximum Entropy Principle; Shannon Entropy; JEL Classification:C61; C63; C87;

    JEL classification:

    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • C87 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Econometric Software


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:emetrv:v:18:y:1999:i:2:p:127-140. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.