IDEAS home Printed from https://ideas.repec.org/p/ehl/lserod/57685.html
   My bibliography  Save this paper

Explaining the behavior of joint and marginal Monte Carlo estimators in latent variable models with independence assumptions

Author

Listed:
  • Vitoratou, Silia
  • Ntzoufras, Ioannis
  • Moustaki, Irini

Abstract

In latent variable models parameter estimation can be implemented by using the joint or the marginal likelihood, based on independence or conditional independence assumptions. The same dilemma occurs within the Bayesian framework with respect to the estimation of the Bayesian marginal (or integrated) likelihood, which is the main tool for model comparison and averaging. In most cases, the Bayesian marginal likelihood is a high dimensional integral that cannot be computed analytically and a plethora of methods based on Monte Carlo integration (MCI) are used for its estimation. In this work, it is shown that the joint MCI approach makes subtle use of the properties of the adopted model, leading to increased error and bias in finite settings. The sources and the components of the error associated with estimators under the two approaches are identified here and provided in exact forms. Additionally, the effect of the sample covariation on the Monte Carlo estimators is examined. In particular, even under independence assumptions the sample covariance will be close to (but not exactly) zero which surprisingly has a severe effect on the estimated values and their variability. To address this problem, an index of the sample's divergence from independence is introduced as a multivariate extension of covariance. The implications addressed here are important in the majority of practical problems appearing in Bayesian inference of multi-parameter models with analogous structures.

Suggested Citation

  • Vitoratou, Silia & Ntzoufras, Ioannis & Moustaki, Irini, 2016. "Explaining the behavior of joint and marginal Monte Carlo estimators in latent variable models with independence assumptions," LSE Research Online Documents on Economics 57685, London School of Economics and Political Science, LSE Library.
    • Handle: RePEc:ehl:lserod:57685
    as

    Download full text from publisher

    File URL: http://eprints.lse.ac.uk/57685/
    File Function: Open access version.
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Geweke, John & Zhou, Guofu, 1996. "Measuring the Pricing Error of the Arbitrage Pricing Theory," The Review of Financial Studies, Society for Financial Studies, vol. 9(2), pages 557-587.
    2. R. Bock & Murray Aitkin, 1981. "Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm," Psychometrika, Springer;The Psychometric Society, vol. 46(4), pages 443-459, December.
    3. Aguilar, Omar & West, Mike, 2000. "Bayesian Dynamic Factor Models and Portfolio Allocation," Journal of Business & Economic Statistics, American Statistical Association, vol. 18(3), pages 338-357, July.
    4. Seock-Ho Kim & Allan Cohen & Frank Baker & Michael Subkoviak & Tom Leonard, 1994. "An investigation of hierarchical Bayes procedures in item response theory," Psychometrika, Springer;The Psychometric Society, vol. 59(3), pages 405-421, September.
    5. Philippe Huber & Elvezio Ronchetti & Maria‐Pia Victoria‐Feser, 2004. "Estimation of generalized linear latent variable models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(4), pages 893-908, November.
    6. N. Friel & A. N. Pettitt, 2008. "Marginal likelihood estimation via power posteriors," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(3), pages 589-607, July.
    7. Koehler, Elizabeth & Brown, Elizabeth & Haneuse, Sebastien J.-P. A., 2009. "On the Assessment of Monte Carlo Error in Simulation-Based Statistical Analyses," The American Statistician, American Statistical Association, vol. 63(2), pages 155-162.
    8. Irini Moustaki & Martin Knott, 2000. "Generalized latent trait models," Psychometrika, Springer;The Psychometric Society, vol. 65(3), pages 391-411, September.
    9. Jones, Galin L. & Haran, Murali & Caffo, Brian S. & Neath, Ronald, 2006. "Fixed-Width Output Analysis for Markov Chain Monte Carlo," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1537-1547, December.
    10. Rabe-Hesketh, Sophia & Skrondal, Anders & Pickles, Andrew, 2005. "Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects," Journal of Econometrics, Elsevier, vol. 128(2), pages 301-323, October.
    11. Chib S. & Jeliazkov I., 2001. "Marginal Likelihood From the Metropolis-Hastings Output," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 270-281, March.
    12. Richard J. Patz & Brian W. Junker, 1999. "A Straightforward Approach to Markov Chain Monte Carlo Methods for Item Response Models," Journal of Educational and Behavioral Statistics, , vol. 24(2), pages 146-178, June.
    13. Robert Mislevy, 1986. "Bayes modal estimation in item response models," Psychometrika, Springer;The Psychometric Society, vol. 51(2), pages 177-195, June.
    14. Stephen Schilling & R. Bock, 2005. "High-dimensional maximum marginal likelihood item factor analysis by adaptive quadrature," Psychometrika, Springer;The Psychometric Society, vol. 70(3), pages 533-555, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bianconcini, Silvia & Cagnone, Silvia, 2012. "Estimation of generalized linear latent variable models via fully exponential Laplace approximation," Journal of Multivariate Analysis, Elsevier, vol. 112(C), pages 183-193.
    2. Li Cai, 2010. "High-dimensional Exploratory Item Factor Analysis by A Metropolis–Hastings Robbins–Monro Algorithm," Psychometrika, Springer;The Psychometric Society, vol. 75(1), pages 33-57, March.
    3. Silvia Cagnone & Paola Monari, 2013. "Latent variable models for ordinal data by using the adaptive quadrature approximation," Computational Statistics, Springer, vol. 28(2), pages 597-619, April.
    4. Sun-Joo Cho & Paul Boeck & Susan Embretson & Sophia Rabe-Hesketh, 2014. "Additive Multilevel Item Structure Models with Random Residuals: Item Modeling for Explanation and Item Generation," Psychometrika, Springer;The Psychometric Society, vol. 79(1), pages 84-104, January.
    5. Christopher J. Urban & Daniel J. Bauer, 2021. "A Deep Learning Algorithm for High-Dimensional Exploratory Item Factor Analysis," Psychometrika, Springer;The Psychometric Society, vol. 86(1), pages 1-29, March.
    6. Vassilis Vasdekis & Silvia Cagnone & Irini Moustaki, 2012. "A Composite Likelihood Inference in Latent Variable Models for Ordinal Longitudinal Responses," Psychometrika, Springer;The Psychometric Society, vol. 77(3), pages 425-441, July.
    7. Björn Andersson & Tao Xin, 2021. "Estimation of Latent Regression Item Response Theory Models Using a Second-Order Laplace Approximation," Journal of Educational and Behavioral Statistics, , vol. 46(2), pages 244-265, April.
    8. Haruhiko Ogasawara, 2013. "Asymptotic properties of the Bayes modal estimators of item parameters in item response theory," Computational Statistics, Springer, vol. 28(6), pages 2559-2583, December.
    9. Joshua C. C. Chan & Liana Jacobi & Dan Zhu, 2022. "An automated prior robustness analysis in Bayesian model comparison," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(3), pages 583-602, April.
    10. An, Xinming & Bentler, Peter M., 2012. "Efficient direct sampling MCEM algorithm for latent variable models with binary responses," Computational Statistics & Data Analysis, Elsevier, vol. 56(2), pages 231-244.
    11. Wu, Jianmin & Bentler, Peter M., 2013. "Limited information estimation in binary factor analysis: A review and extension," Computational Statistics & Data Analysis, Elsevier, vol. 57(1), pages 392-403.
    12. Yang Liu & Ji Seung Yang, 2018. "Bootstrap-Calibrated Interval Estimates for Latent Variable Scores in Item Response Theory," Psychometrika, Springer;The Psychometric Society, vol. 83(2), pages 333-354, June.
    13. Anders Skrondal & Sophia Rabe‐Hesketh, 2009. "Prediction in multilevel generalized linear models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(3), pages 659-687, June.
    14. Zhehan Jiang & Jonathan Templin, 2019. "Gibbs Samplers for Logistic Item Response Models via the Pólya–Gamma Distribution: A Computationally Efficient Data-Augmentation Strategy," Psychometrika, Springer;The Psychometric Society, vol. 84(2), pages 358-374, June.
    15. Minjeong Jeon & Frank Rijmen & Sophia Rabe-Hesketh, 2017. "A Variational Maximization–Maximization Algorithm for Generalized Linear Mixed Models with Crossed Random Effects," Psychometrika, Springer;The Psychometric Society, vol. 82(3), pages 693-716, September.
    16. Vock, David & Davidian, Marie & Tsiatis, Anastasios, 2014. "SNP_NLMM: A SAS Macro to Implement a Flexible Random Effects Density for Generalized Linear and Nonlinear Mixed Models," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 56(c02).
    17. Chan, Joshua C.C. & Grant, Angelia L., 2016. "Fast computation of the deviance information criterion for latent variable models," Computational Statistics & Data Analysis, Elsevier, vol. 100(C), pages 847-859.
    18. Andersson, Björn & Jin, Shaobo & Zhang, Maoxin, 2023. "Fast estimation of multiple group generalized linear latent variable models for categorical observed variables," Computational Statistics & Data Analysis, Elsevier, vol. 182(C).
    19. Javier Revuelta, 2008. "The generalized Logit-Linear Item Response Model for Binary-Designed Items," Psychometrika, Springer;The Psychometric Society, vol. 73(3), pages 385-405, September.
    20. Chib, Siddhartha & Nardari, Federico & Shephard, Neil, 2006. "Analysis of high dimensional multivariate stochastic volatility models," Journal of Econometrics, Elsevier, vol. 134(2), pages 341-371, October.

    More about this item

    Keywords

    Bayes factor; marginal likelihood; Monte Carlo integration;
    All these keywords.

    JEL classification:

    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ehl:lserod:57685. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: LSERO Manager (email available below). General contact details of provider: https://edirc.repec.org/data/lsepsuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.