IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v153y2009i2p155-173.html
   My bibliography  Save this article

Regression density estimation using smooth adaptive Gaussian mixtures

Author

Listed:
  • Villani, Mattias
  • Kohn, Robert
  • Giordani, Paolo

Abstract

We model a regression density flexibly so that at each value of the covariates the density is a mixture of normals with the means, variances and mixture probabilities of the components changing smoothly as a function of the covariates. The model extends the existing models in two important ways. First, the components are allowed to be heteroscedastic regressions as the standard model with homoscedastic regressions can give a poor fit to heteroscedastic data, especially when the number of covariates is large. Furthermore, we typically need fewer components, which makes it easier to interpret the model and speeds up the computation. The second main extension is to introduce a novel variable selection prior into all the components of the model. The variable selection prior acts as a self-adjusting mechanism that prevents overfitting and makes it feasible to fit flexible high-dimensional surfaces. We use Bayesian inference and Markov Chain Monte Carlo methods to estimate the model. Simulated and real examples are used to show that the full generality of our model is required to fit a large class of densities, but also that special cases of the general model are interesting models for economic data.

Suggested Citation

  • Villani, Mattias & Kohn, Robert & Giordani, Paolo, 2009. "Regression density estimation using smooth adaptive Gaussian mixtures," Journal of Econometrics, Elsevier, vol. 153(2), pages 155-173, December.
  • Handle: RePEc:eee:econom:v:153:y:2009:i:2:p:155-173
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304-4076(09)00141-9
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. De Iorio, Maria & Muller, Peter & Rosner, Gary L. & MacEachern, Steven N., 2004. "An ANOVA Model for Dependent Random Measures," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 205-215, January.
    2. George Kapetanios, 2007. "Measuring Conditional Persistence in Nonlinear Time Series," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 69(3), pages 363-386, June.
    3. Villani, Mattias & Kohn, Robert & Giordani, Paolo, 2007. "Nonparametric Regression Density Estimation Using Smoothly Varying Normal Mixtures," Working Paper Series 211, Sveriges Riksbank (Central Bank of Sweden).
    4. David J. Nott & Robert Kohn, 2005. "Adaptive sampling for Bayesian variable selection," Biometrika, Biometrika Trust, vol. 92(4), pages 747-763, December.
    5. D. G. T. Denison & B. K. Mallick & A. F. M. Smith, 1998. "Automatic Bayesian curve fitting," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 60(2), pages 333-350.
    6. Sylvia. Richardson & Peter J. Green, 1997. "On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion)," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 59(4), pages 731-792.
    7. Geweke, John & Keane, Michael, 2007. "Smoothly mixing regressions," Journal of Econometrics, Elsevier, vol. 138(1), pages 252-290, May.
    8. David B. Dunson & Natesh Pillai & Ju‐Hyun Park, 2007. "Bayesian density regression," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(2), pages 163-183, April.
    9. Smith, Michael & Kohn, Robert, 1996. "Nonparametric regression using Bayesian variable selection," Journal of Econometrics, Elsevier, vol. 75(2), pages 317-343, December.
    10. Sally A. Wood, 2002. "Bayesian mixture of splines for spatially adaptive nonparametric regression," Biometrika, Biometrika Trust, vol. 89(3), pages 513-528, August.
    11. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521785167, September.
    12. Geweke, John, 2007. "Interpretation and inference in mixture models: Simple MCMC works," Computational Statistics & Data Analysis, Elsevier, vol. 51(7), pages 3529-3550, April.
    13. Lawrence J. Christiano & Terry J. Fitzgerald, 2003. "Inflation and monetary policy in the twentieth century," Economic Perspectives, Federal Reserve Bank of Chicago, vol. 27(Q I), pages 22-45.
    14. Sassan Alizadeh & Michael W. Brandt & Francis X. Diebold, 2002. "Range‐Based Estimation of Stochastic Volatility Models," Journal of Finance, American Finance Association, vol. 57(3), pages 1047-1091, June.
    15. Racine, Jeffrey S., 2008. "Nonparametric Econometrics: A Primer," Foundations and Trends(R) in Econometrics, now publishers, vol. 3(1), pages 1-88, March.
    16. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521780506, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Roberto Casarin & Stefano Grassi & Francesco Ravazzolo & Herman K. van Dijk, 2015. "Dynamic predictive density combinations for large data sets in economics and finance," Working Paper 2015/12, Norges Bank.
    2. Tsionas, Mike & Parmeter, Christopher F. & Zelenyuk, Valentin, 2023. "Bayesian Artificial Neural Networks for frontier efficiency analysis," Journal of Econometrics, Elsevier, vol. 236(2).
    3. Mike Tsionas & Marwan Izzeldin & Lorenzo Trapani, 2019. "Bayesian estimation of large dimensional time varying VARs using copulas," Papers 1912.12527, arXiv.org.
    4. Salimans, Tim, 2012. "Variable selection and functional form uncertainty in cross-country growth regressions," Journal of Econometrics, Elsevier, vol. 171(2), pages 267-280.
    5. Tsionas, Mike G. & Izzeldin, Marwan & Trapani, Lorenzo, 2022. "Estimation of large dimensional time varying VARs using copulas," European Economic Review, Elsevier, vol. 141(C).
    6. Kalli, Maria & Griffin, Jim E., 2018. "Bayesian nonparametric vector autoregressive models," Journal of Econometrics, Elsevier, vol. 203(2), pages 267-282.
    7. Kalliovirta, Leena & Meitz, Mika & Saikkonen, Pentti, 2016. "Gaussian mixture vector autoregression," Journal of Econometrics, Elsevier, vol. 192(2), pages 485-498.
    8. Mike G. Tsionas, 2017. "“When, Where, and How” of Efficiency Estimation: Improved Procedures for Stochastic Frontier Modeling," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 948-965, July.
    9. Talagala, Thiyanga S. & Li, Feng & Kang, Yanfei, 2022. "FFORMPP: Feature-based forecast model performance prediction," International Journal of Forecasting, Elsevier, vol. 38(3), pages 920-943.
    10. Kastner, Gregor, 2019. "Sparse Bayesian time-varying covariance estimation in many dimensions," Journal of Econometrics, Elsevier, vol. 210(1), pages 98-115.
    11. Li, Feng & Kang, Yanfei, 2018. "Improving forecasting performance using covariate-dependent copula models," International Journal of Forecasting, Elsevier, vol. 34(3), pages 456-476.
    12. Norets, Andriy, 2015. "Bayesian regression with nonparametric heteroskedasticity," Journal of Econometrics, Elsevier, vol. 185(2), pages 409-419.
    13. Paolo Giordani & Xiuyan Mun & Robert Kohn, 2012. "Efficient Estimation of Covariance Matrices using Posterior Mode Multiple Shrinkage," Journal of Financial Econometrics, Oxford University Press, vol. 11(1), pages 154-192, December.
    14. Norets, Andriy & Pelenis, Justinas, 2012. "Bayesian modeling of joint and conditional distributions," Journal of Econometrics, Elsevier, vol. 168(2), pages 332-346.
    15. Cozzini, Alberto & Jasra, Ajay & Montana, Giovanni & Persing, Adam, 2014. "A Bayesian mixture of lasso regressions with t-errors," Computational Statistics & Data Analysis, Elsevier, vol. 77(C), pages 84-97.
    16. Cheng Peng & Stanislav Uryasev, 2023. "Factor Model of Mixtures," Papers 2301.13843, arXiv.org, revised Mar 2023.
    17. Keane, Michael & Ketcham, Jonathan & Kuminoff, Nicolai & Neal, Timothy, 2021. "Evaluating consumers’ choices of Medicare Part D plans: A study in behavioral welfare economics," Journal of Econometrics, Elsevier, vol. 222(1), pages 107-140.
    18. Marco Berrettini & Giuliano Galimberti & Saverio Ranciati, 2023. "Semiparametric finite mixture of regression models with Bayesian P-splines," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 17(3), pages 745-775, September.
    19. Norets, Andriy & Pelenis, Justinas, 2022. "Adaptive Bayesian estimation of conditional discrete-continuous distributions with an application to stock market trading activity," Journal of Econometrics, Elsevier, vol. 230(1), pages 62-82.
    20. Villani, Mattias & Kohn, Robert & Nott, David J., 2012. "Generalized smooth finite mixtures," Journal of Econometrics, Elsevier, vol. 171(2), pages 121-133.
    21. Roberto Casarin & Stefano Grassi & Francesco Ravazzolo & Herman K. van Dijk, 2020. "A Bayesian Dynamic Compositional Model for Large Density Combinations in Finance," Working Paper series 20-27, Rimini Centre for Economic Analysis.
    22. Yanfei Kang & Rob J Hyndman & Feng Li, 2018. "Efficient generation of time series with diverse and controllable characteristics," Monash Econometrics and Business Statistics Working Papers 15/18, Monash University, Department of Econometrics and Business Statistics.
    23. Meitz, Mika & Saikkonen, Pentti, 2021. "Testing for observation-dependent regime switching in mixture autoregressive models," Journal of Econometrics, Elsevier, vol. 222(1), pages 601-624.
    24. Quiroz, Matias & Villani, Mattias, 2013. "Dynamic mixture-of-experts models for longitudinal and discrete-time survival data," Working Paper Series 268, Sveriges Riksbank (Central Bank of Sweden).
    25. Mike Tsionas & Christopher F. Parmeter & Valentin Zelenyuk, 2021. "Bridging the Divide? Bayesian Artificial Neural Networks for Frontier Efficiency Analysis," CEPA Working Papers Series WP082021, School of Economics, University of Queensland, Australia.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Villani, Mattias & Kohn, Robert & Giordani, Paolo, 2007. "Nonparametric Regression Density Estimation Using Smoothly Varying Normal Mixtures," Working Paper Series 211, Sveriges Riksbank (Central Bank of Sweden).
    2. Feng Li & Mattias Villani, 2013. "Efficient Bayesian Multivariate Surface Regression," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 40(4), pages 706-723, December.
    3. Yu Yue & Paul Speckman & Dongchu Sun, 2012. "Priors for Bayesian adaptive spline smoothing," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 64(3), pages 577-613, June.
    4. Villani, Mattias & Kohn, Robert & Nott, David J., 2012. "Generalized smooth finite mixtures," Journal of Econometrics, Elsevier, vol. 171(2), pages 121-133.
    5. Leitenstorfer, Florian & Tutz, Gerhard, 2007. "Knot selection by boosting techniques," Computational Statistics & Data Analysis, Elsevier, vol. 51(9), pages 4605-4621, May.
    6. Norets, Andriy & Pelenis, Justinas, 2012. "Bayesian modeling of joint and conditional distributions," Journal of Econometrics, Elsevier, vol. 168(2), pages 332-346.
    7. Panagiotelis, Anastasios & Smith, Michael, 2008. "Bayesian identification, selection and estimation of semiparametric functions in high-dimensional additive models," Journal of Econometrics, Elsevier, vol. 143(2), pages 291-316, April.
    8. Brezger, Andreas & Lang, Stefan, 2006. "Generalized structured additive regression based on Bayesian P-splines," Computational Statistics & Data Analysis, Elsevier, vol. 50(4), pages 967-991, February.
    9. Li, Feng & Kang, Yanfei, 2018. "Improving forecasting performance using covariate-dependent copula models," International Journal of Forecasting, Elsevier, vol. 34(3), pages 456-476.
    10. Nott, David J., 2008. "Predictive performance of Dirichlet process shrinkage methods in linear regression," Computational Statistics & Data Analysis, Elsevier, vol. 52(7), pages 3658-3669, March.
    11. Huaihou Chen & Yuanjia Wang, 2011. "A Penalized Spline Approach to Functional Mixed Effects Model Analysis," Biometrics, The International Biometric Society, vol. 67(3), pages 861-870, September.
    12. Gholamreza Hajargasht, 2009. "Nonparametric Panel Data Models, A Penalized Spline Approach," CEPA Working Papers Series WP052009, School of Economics, University of Queensland, Australia.
    13. Griffin, J.E. & Steel, M.F.J., 2011. "Stick-breaking autoregressive processes," Journal of Econometrics, Elsevier, vol. 162(2), pages 383-396, June.
    14. Hübler, Olaf, 2017. "Health and Body Mass Index: No Simple Relationship," IZA Discussion Papers 10620, Institute of Labor Economics (IZA).
    15. Congdon, Peter, 2006. "A model for non-parametric spatially varying regression effects," Computational Statistics & Data Analysis, Elsevier, vol. 50(2), pages 422-445, January.
    16. Stefan Lang & Nikolaus Umlauf & Peter Wechselberger & Kenneth Harttgen & Thomas Kneib, 2012. "Multilevel structured additive regression," Working Papers 2012-07, Faculty of Economics and Statistics, Universität Innsbruck.
    17. Norets, Andriy, 2015. "Bayesian regression with nonparametric heteroskedasticity," Journal of Econometrics, Elsevier, vol. 185(2), pages 409-419.
    18. Paciorek, Christopher J., 2007. "Computational techniques for spatial logistic regression with large data sets," Computational Statistics & Data Analysis, Elsevier, vol. 51(8), pages 3631-3653, May.
    19. Chib, Siddhartha & Greenberg, Edward, 2010. "Additive cubic spline regression with Dirichlet process mixture errors," Journal of Econometrics, Elsevier, vol. 156(2), pages 322-336, June.
    20. Jeong, Seonghyun & Park, Minjae & Park, Taeyoung, 2017. "Analysis of binary longitudinal data with time-varying effects," Computational Statistics & Data Analysis, Elsevier, vol. 112(C), pages 145-153.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:153:y:2009:i:2:p:155-173. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.