IDEAS home Printed from https://ideas.repec.org/p/ems/eureir/1281.html
   My bibliography  Save this paper

Neural network based approximations to posterior densities: a class of flexible sampling methods with applications to reduced rank models

Author

Listed:
  • Hoogerheide, L.F.
  • Kaashoek, J.F.
  • van Dijk, H.K.

Abstract

Likelihoods and posteriors of econometric models with strong endogeneity and weak instruments may exhibit rather non-elliptical contours in the parameter space. This feature also holds for cointegration models when near non-stationarity occurs and determining the number of cointegrating relations is a nontrivial issue, and in mixture processes where the modes are relatively far apart. The performance of Monte Carlo integration methods like importance sampling or Markov Chain Monte Carlo procedures greatly depends in all these cases on the choice of the importance or candidate density. Such a density has to be `close' to the target density in order to yield numerically accurate results with efficient sampling. Neural networks seem to be natural importance or candidate densities, as they have a universal approximation property and are easy to sample from. That is, conditionally upon the specification of the neural network, sampling can be done either directly or using a Gibbs sampling technique, possibly using auxiliary variables. A key step in the proposed class of methods is the construction of a neural network that approximates the target density accurately. The methods are tested on a set of illustrative models which include a mixture of normal distributions, a Bayesian instrumental variable regression problem with weak instruments and near non-identification, a cointegration model with near non-stationarity and a two-regime growth model for US recessions and expansions. These examples involve experiments with non-standard, non-elliptical posterior distributions. The results indicate the feasibility of the neural network approach.

Suggested Citation

  • Hoogerheide, L.F. & Kaashoek, J.F. & van Dijk, H.K., 2004. "Neural network based approximations to posterior densities: a class of flexible sampling methods with applications to reduced rank models," Econometric Institute Research Papers EI 2004-19, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
  • Handle: RePEc:ems:eureir:1281
    as

    Download full text from publisher

    File URL: https://repub.eur.nl/pub/1281/ei200419.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Paap, Richard & van Dijk, Herman K, 2003. "Bayes Estimates of Markov Trends in Possibly Cointegrated Series: An Application to U.S. Consumption and Income," Journal of Business & Economic Statistics, American Statistical Association, vol. 21(4), pages 547-563, October.
    2. John Y. Campbell & N. Gregory Mankiw, 1989. "Consumption, Income, and Interest Rates: Reinterpreting the Time Series Evidence," NBER Chapters, in: NBER Macroeconomics Annual 1989, Volume 4, pages 185-246, National Bureau of Economic Research, Inc.
    3. P. Damlen & J. Wakefield & S. Walker, 1999. "Gibbs sampling for Bayesian non‐conjugate and hierarchical models by using auxiliary variables," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 61(2), pages 331-344, April.
    4. John Geweke, 1999. "Using Simulation Methods for Bayesian Econometric Models," Computing in Economics and Finance 1999 832, Society for Computational Economics.
    5. Campbell, John Y & Shiller, Robert J, 1987. "Cointegration and Tests of Present Value Models," Journal of Political Economy, University of Chicago Press, vol. 95(5), pages 1062-1088, October.
    6. Bauwens, Luc & Bos, Charles S. & van Dijk, Herman K. & van Oest, Rutger D., 2004. "Adaptive radial-based direction sampling: some flexible and robust Monte Carlo integration methods," Journal of Econometrics, Elsevier, vol. 123(2), pages 201-225, December.
    7. Chib, Siddhartha & Greenberg, Edward, 1996. "Markov Chain Monte Carlo Simulation Methods in Econometrics," Econometric Theory, Cambridge University Press, vol. 12(3), pages 409-431, August.
    8. Douglas Staiger & James H. Stock, 1997. "Instrumental Variables Regression with Weak Instruments," Econometrica, Econometric Society, vol. 65(3), pages 557-586, May.
    9. Campbell, John Y, 1987. "Does Saving Anticipate Declining Labor Income? An Alternative Test of the Permanent Income Hypothesis," Econometrica, Econometric Society, vol. 55(6), pages 1249-1273, November.
    10. Schotman, Peter & van Dijk, Herman K., 1991. "A Bayesian analysis of the unit root in real exchange rates," Journal of Econometrics, Elsevier, vol. 49(1-2), pages 195-238.
    11. van Dijk, H. K. & Kloek, T., 1980. "Further experience in Bayesian analysis using Monte Carlo integration," Journal of Econometrics, Elsevier, vol. 14(3), pages 307-328, December.
    12. Geweke, John, 1989. "Bayesian Inference in Econometric Models Using Monte Carlo Integration," Econometrica, Econometric Society, vol. 57(6), pages 1317-1339, November.
    13. Hutchinson, James M & Lo, Andrew W & Poggio, Tomaso, 1994. "A Nonparametric Approach to Pricing and Hedging Derivative Securities via Learning Networks," Journal of Finance, American Finance Association, vol. 49(3), pages 851-889, July.
    14. repec:fth:harver:1435 is not listed on IDEAS
    15. Hamilton, James D, 1989. "A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle," Econometrica, Econometric Society, vol. 57(2), pages 357-384, March.
    16. Hoogerheide, L.F. & Kaashoek, J.F. & van Dijk, H.K., 2002. "Functional approximations to posterior densities: a neural network approach to efficient sampling," Econometric Institute Research Papers EI 2002-48, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    17. Kloek, Tuen & van Dijk, Herman K, 1978. "Bayesian Estimates of Equation System Parameters: An Application of Integration by Monte Carlo," Econometrica, Econometric Society, vol. 46(1), pages 1-19, January.
    18. Hoogerheide, L.F. & Kaashoek, J.F. & van Dijk, H.K., 2003. "Neural network approximations to posterior densities: an analytical approach," Econometric Institute Research Papers EI 2003-38, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    19. Hall, Anthony D & Anderson, Heather M & Granger, Clive W J, 1992. "A Cointegration Analysis of Treasury Bill Yields," The Review of Economics and Statistics, MIT Press, vol. 74(1), pages 116-126, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hoogerheide, Lennart F. & Kaashoek, Johan F. & van Dijk, Herman K., 2007. "On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: An application of flexible sampling methods using neural networks," Journal of Econometrics, Elsevier, vol. 139(1), pages 154-180, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hoogerheide, Lennart F. & Kaashoek, Johan F. & van Dijk, Herman K., 2007. "On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: An application of flexible sampling methods using neural networks," Journal of Econometrics, Elsevier, vol. 139(1), pages 154-180, July.
    2. Lennart F. Hoogerheide & Johan F. Kaashoek, 2004. "Functional Approximations to Likelihoods/Posterior Densities: A Neural Network Approach to Efficient Sampling," Computing in Economics and Finance 2004 74, Society for Computational Economics.
    3. Hoogerheide, L.F. & Kaashoek, J.F. & van Dijk, H.K., 2002. "Functional approximations to posterior densities: a neural network approach to efficient sampling," Econometric Institute Research Papers EI 2002-48, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    4. Nalan Basturk & Cem Cakmakli & S. Pinar Ceyhan & Herman K. van Dijk, 2014. "On the Rise of Bayesian Econometrics after Cowles Foundation Monographs 10, 14," Tinbergen Institute Discussion Papers 14-085/III, Tinbergen Institute, revised 04 Sep 2014.
    5. van Dijk, H.K., 2002. "On Bayesian structural inference in a simultaneous equation model," Econometric Institute Research Papers EI 2002-10, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    6. Hoogerheide, L.F. & van Dijk, H.K. & van Oest, R.D., 2007. "Simulation based bayesian econometric inference: principles and some recent computational advances," Econometric Institute Research Papers EI 2007-03, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    7. Hoogerheide, L.F. & Kaashoek, J.F. & van Dijk, H.K., 2003. "Neural network approximations to posterior densities: an analytical approach," Econometric Institute Research Papers EI 2003-38, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    8. David Ardia & Lennart Hoogerheide & Herman K. van Dijk, 2009. "To Bridge, to Warp or to Wrap? A Comparative Study of Monte Carlo Methods for Efficient Evaluation of Marginal Likelihoods," Tinbergen Institute Discussion Papers 09-017/4, Tinbergen Institute.
    9. Pagan, Adrian, 1996. "The econometrics of financial markets," Journal of Empirical Finance, Elsevier, vol. 3(1), pages 15-102, May.
    10. Ardia, David & Baştürk, Nalan & Hoogerheide, Lennart & van Dijk, Herman K., 2012. "A comparative study of Monte Carlo methods for efficient evaluation of marginal likelihood," Computational Statistics & Data Analysis, Elsevier, vol. 56(11), pages 3398-3414.
    11. Villani, Mattias, 2006. "Bayesian point estimation of the cointegration space," Journal of Econometrics, Elsevier, vol. 134(2), pages 645-664, October.
    12. Richard Kleijn & Herman K. van Dijk, 2006. "Bayes model averaging of cyclical decompositions in economic time series," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(2), pages 191-212.
    13. Luc Bauwens & Charles S. Bos & Herman K. van Dijk, 1999. "Adaptive Polar Sampling with an Application to a Bayes Measure of Value-at-Risk," Tinbergen Institute Discussion Papers 99-082/4, Tinbergen Institute.
    14. Villani, Mattias, 2005. "Bayesian Inference of General Linear Restrictions on the Cointegration Space," Working Paper Series 189, Sveriges Riksbank (Central Bank of Sweden).
    15. Fuentes-Albero, Cristina & Melosi, Leonardo, 2013. "Methods for computing marginal data densities from the Gibbs output," Journal of Econometrics, Elsevier, vol. 175(2), pages 132-141.
    16. Dellaportas, Petros & Tsionas, Mike G., 2019. "Importance sampling from posterior distributions using copula-like approximations," Journal of Econometrics, Elsevier, vol. 210(1), pages 45-57.
    17. Gael M. Martin & David T. Frazier & Christian P. Robert, 2020. "Computing Bayes: Bayesian Computation from 1763 to the 21st Century," Monash Econometrics and Business Statistics Working Papers 14/20, Monash University, Department of Econometrics and Business Statistics.
    18. repec:bla:jecsur:v:16:y:2002:i:3:p:301-55 is not listed on IDEAS
    19. Nalan Basturk & Cem Cakmakli & S. Pinar Ceyhan & Herman K. van Dijk, 2013. "Historical Developments in Bayesian Econometrics after Cowles Foundation Monographs 10, 14," Tinbergen Institute Discussion Papers 13-191/III, Tinbergen Institute.
    20. Gael M. Martin & David T. Frazier & Christian P. Robert, 2022. "Computing Bayes: From Then `Til Now," Monash Econometrics and Business Statistics Working Papers 14/22, Monash University, Department of Econometrics and Business Statistics.
    21. Bauwens, Luc & Bos, Charles S. & van Dijk, Herman K. & van Oest, Rutger D., 2004. "Adaptive radial-based direction sampling: some flexible and robust Monte Carlo integration methods," Journal of Econometrics, Elsevier, vol. 123(2), pages 201-225, December.

    More about this item

    Keywords

    Bayesian inference; Markov chain Monte Carlo; importance sample; neural networks;
    All these keywords.

    JEL classification:

    • C11 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Bayesian Analysis: General
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C45 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Neural Networks and Related Topics

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ems:eureir:1281. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: RePub (email available below). General contact details of provider: https://edirc.repec.org/data/feeurnl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.