IDEAS home Printed from https://ideas.repec.org/p/ecb/ecbwps/20101189.html
   My bibliography  Save this paper

Maximum likelihood estimation of factor models on data sets with arbitrary pattern of missing data

Author

Listed:
  • Bańbura, Marta
  • Modugno, Michele

Abstract

In this paper we propose a methodology to estimate a dynamic factor model on data sets with an arbitrary pattern of missing data. We modify the Expectation Maximisation (EM) algorithm as proposed for a dynamic factor model by Watson and Engle (1983) to the case with general pattern of missing data. We also extend the model to the case with serially correlated idiosyncratic component. The framework allows to handle efficiently and in an automatic manner sets of indicators characterized by different publication delays, frequencies and sample lengths. This can be relevant e.g. for young economies for which many indicators are compiled only since recently. We also show how to extract a model based news from a statistical data release within our framework and we derive the relationship between the news and the resulting forecast revision. This can be used for interpretation in e.g. nowcasting applications as it allows to determine the sign and size of a news as well as its contribution to the revision, in particular in case of simultaneous data releases. We evaluate the methodology in a Monte Carlo experiment and we apply it to nowcasting and backdating of euro area GDP. JEL Classification: C53, E37

Suggested Citation

  • Bańbura, Marta & Modugno, Michele, 2010. "Maximum likelihood estimation of factor models on data sets with arbitrary pattern of missing data," Working Paper Series 1189, European Central Bank.
    • Handle: RePEc:ecb:ecbwps:20101189
    Note: 810771
    as

    Download full text from publisher

    File URL: https://www.ecb.europa.eu//pub/pdf/scpwps/ecbwp1189.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Forni, Mario & Hallin, Marc & Lippi, Marco & Reichlin, Lucrezia, 2003. "Do financial variables help forecasting inflation and real activity in the euro area?," Journal of Monetary Economics, Elsevier, vol. 50(6), pages 1243-1255, September.
    2. Doz, Catherine & Giannone, Domenico & Reichlin, Lucrezia, 2011. "A two-step estimator for large approximate dynamic factor models based on Kalman filtering," Journal of Econometrics, Elsevier, vol. 164(1), pages 188-205, September.
    3. Catherine Doz & Domenico Giannone & Lucrezia Reichlin, 2012. "A Quasi–Maximum Likelihood Approach for Large, Approximate Dynamic Factor Models," The Review of Economics and Statistics, MIT Press, vol. 94(4), pages 1014-1024, November.
    4. Bańbura, Marta & Rünstler, Gerhard, 2011. "A look into the factor model black box: Publication lags and the role of hard and soft data in forecasting GDP," International Journal of Forecasting, Elsevier, vol. 27(2), pages 333-346.
    5. R. H. Shumway & D. S. Stoffer, 1982. "An Approach To Time Series Smoothing And Forecasting Using The Em Algorithm," Journal of Time Series Analysis, Wiley Blackwell, vol. 3(4), pages 253-264, July.
    6. Ben S. Bernanke & Jean Boivin & Piotr Eliasz, 2005. "Measuring the Effects of Monetary Policy: A Factor-Augmented Vector Autoregressive (FAVAR) Approach," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 120(1), pages 387-422.
    7. Mario Forni & Lucrezia Reichlin, 1998. "Let's Get Real: A Factor Analytical Approach to Disaggregated Business Cycle Dynamics," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 65(3), pages 453-473.
    8. Connor, Gregory & Korajczyk, Robert A., 1986. "Performance measurement with the arbitrage pricing theory : A new framework for analysis," Journal of Financial Economics, Elsevier, vol. 15(3), pages 373-394, March.
    9. Belviso Francesco & Milani Fabio, 2006. "Structural Factor-Augmented VARs (SFAVARs) and the Effects of Monetary Policy," The B.E. Journal of Macroeconomics, De Gruyter, vol. 6(3), pages 1-46, December.
    10. Guenter W. Beck & Kirstin Hubrich & Massimiliano Marcellino, 2009. "Regional inflation dynamics within and across euro area countries and a comparison with the United States [‘On the relevance and nature of regional inflation differentials: The case of Spain’, Banc," Economic Policy, CEPR, CESifo, Sciences Po;CES;MSH, vol. 24(57), pages 142-184.
    11. Cecilia Frale & Massimiliano Marcellino & Gian Luigi Mazzi & Tommaso Proietti, 2011. "EUROMIND: a monthly indicator of the euro area economic conditions," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(2), pages 439-470, April.
    12. repec:bla:ecpoli:v:24:y:2009:i::p:141-184 is not listed on IDEAS
    13. Bai, Jushan & Ng, Serena, 2007. "Determining the Number of Primitive Shocks in Factor Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 52-60, January.
    14. Lasse Bork, 2009. "Estimating US Monetary Policy Shocks Using a Factor-Augmented Vector Autoregression: An EM Algorithm Approach," CREATES Research Papers 2009-11, Department of Economics and Business Economics, Aarhus University.
    15. Forni, Mario & Reichlin, Lucrezia, 2001. "Federal policies and local economies: Europe and the US," European Economic Review, Elsevier, vol. 45(1), pages 109-134, January.
    16. Angelini, Elena & Henry, Jerome & Marcellino, Massimiliano, 2006. "Interpolation and backdating with a large information set," Journal of Economic Dynamics and Control, Elsevier, vol. 30(12), pages 2693-2724, December.
    17. M. Ayhan Kose & Christopher Otrok & Charles H. Whiteman, 2003. "International Business Cycles: World, Region, and Country-Specific Factors," American Economic Review, American Economic Association, vol. 93(4), pages 1216-1239, September.
    18. S. J. Koopman & J. Durbin, 2000. "Fast Filtering and Smoothing for Multivariate State Space Models," Journal of Time Series Analysis, Wiley Blackwell, vol. 21(3), pages 281-296, May.
    19. Filippo Altissimo & Riccardo Cristadoro & Mario Forni & Marco Lippi & Giovanni Veronese, 2010. "New Eurocoin: Tracking Economic Growth in Real Time," The Review of Economics and Statistics, MIT Press, vol. 92(4), pages 1024-1034, November.
    20. Durbin, James & Koopman, Siem Jan, 2012. "Time Series Analysis by State Space Methods," OUP Catalogue, Oxford University Press, edition 2, number 9780199641178.
    21. Roberto S. Mariano & Yasutomo Murasawa, 2003. "A new coincident index of business cycles based on monthly and quarterly series," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 18(4), pages 427-443.
    22. Romain Houssa & Lasse Bork & Hans Dewachter, 2008. "Identification of Macroeconomic Factors in Large Panels," Working Papers 1010, University of Namur, Department of Economics.
    23. Boivin, Jean & Ng, Serena, 2006. "Are more data always better for factor analysis?," Journal of Econometrics, Elsevier, vol. 132(1), pages 169-194, May.
    24. repec:hal:journl:peer-00844811 is not listed on IDEAS
    25. Jungbacker, B. & Koopman, S.J. & van der Wel, M., 2011. "Maximum likelihood estimation for dynamic factor models with missing data," Journal of Economic Dynamics and Control, Elsevier, vol. 35(8), pages 1358-1368, August.
    26. Jushan Bai, 2003. "Inferential Theory for Factor Models of Large Dimensions," Econometrica, Econometric Society, vol. 71(1), pages 135-171, January.
    27. Donald Rubin & Dorothy Thayer, 1982. "EM algorithms for ML factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 47(1), pages 69-76, March.
    28. Schumacher, Christian & Breitung, Jörg, 2008. "Real-time forecasting of German GDP based on a large factor model with monthly and quarterly data," International Journal of Forecasting, Elsevier, vol. 24(3), pages 386-398.
    29. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    30. repec:oup:ecpoli:v:24:y:2009:i::p:141-184 is not listed on IDEAS
    31. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    32. Watson, Mark W. & Engle, Robert F., 1983. "Alternative algorithms for the estimation of dynamic factor, mimic and varying coefficient regression models," Journal of Econometrics, Elsevier, vol. 23(3), pages 385-400, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Karim Barhoumi & Olivier Darné & Laurent Ferrara, 2014. "Dynamic factor models: A review of the literature," OECD Journal: Journal of Business Cycle Measurement and Analysis, OECD Publishing, Centre for International Research on Economic Tendency Surveys, vol. 2013(2), pages 73-107.
    2. Stock, J.H. & Watson, M.W., 2016. "Dynamic Factor Models, Factor-Augmented Vector Autoregressions, and Structural Vector Autoregressions in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 415-525, Elsevier.
    3. Catherine Doz & Peter Fuleky, 2019. "Dynamic Factor Models," Working Papers 2019-4, University of Hawaii Economic Research Organization, University of Hawaii at Manoa.
    4. Pilar Poncela & Esther Ruiz, 2016. "Small- Versus Big-Data Factor Extraction in Dynamic Factor Models: An Empirical Assessment," Advances in Econometrics, in: Dynamic Factor Models, volume 35, pages 401-434, Emerald Group Publishing Limited.
    5. Poncela, Pilar & Ruiz, Esther & Miranda, Karen, 2021. "Factor extraction using Kalman filter and smoothing: This is not just another survey," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1399-1425.
    6. Proietti, Tommaso, 2008. "Estimation of Common Factors under Cross-Sectional and Temporal Aggregation Constraints: Nowcasting Monthly GDP and its Main Components," MPRA Paper 6860, University Library of Munich, Germany.
    7. Banbura, Marta & Rünstler, Gerhard, 2011. "A look into the factor model black box: Publication lags and the role of hard and soft data in forecasting GDP," International Journal of Forecasting, Elsevier, vol. 27(2), pages 333-346, April.
    8. Rusnák, Marek, 2016. "Nowcasting Czech GDP in real time," Economic Modelling, Elsevier, vol. 54(C), pages 26-39.
    9. Marcellino, Massimiliano & Sivec, Vasja, 2016. "Monetary, fiscal and oil shocks: Evidence based on mixed frequency structural FAVARs," Journal of Econometrics, Elsevier, vol. 193(2), pages 335-348.
    10. Kihwan Kim & Norman Swanson, 2013. "Diffusion Index Model Specification and Estimation Using Mixed Frequency Datasets," Departmental Working Papers 201315, Rutgers University, Department of Economics.
    11. Alain Galli, 2018. "Which Indicators Matter? Analyzing the Swiss Business Cycle Using a Large-Scale Mixed-Frequency Dynamic Factor Model," Journal of Business Cycle Research, Springer;Centre for International Research on Economic Tendency Surveys (CIRET), vol. 14(2), pages 179-218, November.
    12. Matteo Barigozzi & Matteo Luciani, 2019. "Quasi Maximum Likelihood Estimation and Inference of Large Approximate Dynamic Factor Models via the EM algorithm," Papers 1910.03821, arXiv.org, revised Sep 2024.
    13. Tóth, Peter, 2014. "Malý dynamický faktorový model na krátkodobé prognózovanie slovenského HDP [A Small Dynamic Factor Model for the Short-Term Forecasting of Slovak GDP]," MPRA Paper 63713, University Library of Munich, Germany.
    14. Bae, Juhee, 2024. "Factor-augmented forecasting in big data," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1660-1688.
    15. Monica Defend & Aleksey Min & Lorenzo Portelli & Franz Ramsauer & Francesco Sandrini & Rudi Zagst, 2021. "Quantifying Drivers of Forecasted Returns Using Approximate Dynamic Factor Models for Mixed-Frequency Panel Data," Forecasting, MDPI, vol. 3(1), pages 1-35, February.
    16. Martin Solberger & Erik Spånberg, 2020. "Estimating a Dynamic Factor Model in EViews Using the Kalman Filter and Smoother," Computational Economics, Springer;Society for Computational Economics, vol. 55(3), pages 875-900, March.
    17. Kaufmann, Daniel & Scheufele, Rolf, 2017. "Business tendency surveys and macroeconomic fluctuations," International Journal of Forecasting, Elsevier, vol. 33(4), pages 878-893.
    18. Bańbura, Marta & Giannone, Domenico & Modugno, Michele & Reichlin, Lucrezia, 2013. "Now-Casting and the Real-Time Data Flow," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 195-237, Elsevier.
    19. repec:diw:diwwpp:dp1351 is not listed on IDEAS
    20. Romain Houssa & Lasse Bork & Hans Dewachter, 2008. "Identification of Macroeconomic Factors in Large Panels," Working Papers 1010, University of Namur, Department of Economics.
    21. Helmut Lütkepohl, 2014. "Structural Vector Autoregressive Analysis in a Data Rich Environment: A Survey," Discussion Papers of DIW Berlin 1351, DIW Berlin, German Institute for Economic Research.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • E37 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Forecasting and Simulation: Models and Applications

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecb:ecbwps:20101189. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Official Publications (email available below). General contact details of provider: https://edirc.repec.org/data/emieude.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.