IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1901.11355.html
   My bibliography  Save this paper

A dynamic factor model approach to incorporate Big Data in state space models for official statistics

Author

Listed:
  • Caterina Schiavoni
  • Franz Palm
  • Stephan Smeekes
  • Jan van den Brakel

Abstract

In this paper we consider estimation of unobserved components in state space models using a dynamic factor approach to incorporate auxiliary information from high-dimensional data sources. We apply the methodology to unemployment estimation as done by Statistics Netherlands, who uses a multivariate state space model to produce monthly figures for the unemployment using series observed with the labour force survey (LFS). We extend the model by including auxiliary series of Google Trends about job-search and economic uncertainty, and claimant counts, partially observed at higher frequencies. Our factor model allows for nowcasting the variable of interest, providing reliable unemployment estimates in real-time before LFS data become available.

Suggested Citation

  • Caterina Schiavoni & Franz Palm & Stephan Smeekes & Jan van den Brakel, 2019. "A dynamic factor model approach to incorporate Big Data in state space models for official statistics," Papers 1901.11355, arXiv.org, revised Feb 2020.
    • Handle: RePEc:arx:papers:1901.11355
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1901.11355
    File Function: Latest version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Gourieroux, Christian & Monfort, Alain & Trognon, Alain, 1984. "Pseudo Maximum Likelihood Methods: Theory," Econometrica, Econometric Society, vol. 52(3), pages 681-700, May.
    2. Pfeffermann, Danny, 1991. "Estimation and Seasonal Adjustment of Population Means Using Data from Repeated Surveys: Reply," Journal of Business & Economic Statistics, American Statistical Association, vol. 9(2), pages 177-177, April.
    3. Nikolaos Askitas & Klaus F. Zimmermann, 2009. "Google Econometrics and Unemployment Forecasting," Applied Economics Quarterly (formerly: Konjunkturpolitik), Duncker & Humblot, Berlin, vol. 55(2), pages 107-120.
    4. Pfeffermann, Danny & Feder, Moshe & Signorelli, David, 1998. "Estimation of Autocorrelations of Survey Errors with Application to Trend Estimation in Small Areas," Journal of Business & Economic Statistics, American Statistical Association, vol. 16(3), pages 339-348, July.
    5. Gourieroux, Christian & Monfort, Alain & Trognon, Alain, 1984. "Pseudo Maximum Likelihood Methods: Applications to Poisson Models," Econometrica, Econometric Society, vol. 52(3), pages 701-720, May.
    6. Doz, Catherine & Giannone, Domenico & Reichlin, Lucrezia, 2011. "A two-step estimator for large approximate dynamic factor models based on Kalman filtering," Journal of Econometrics, Elsevier, vol. 164(1), pages 188-205, September.
    7. Jushan Bai & Serena Ng, 2002. "Determining the Number of Factors in Approximate Factor Models," Econometrica, Econometric Society, vol. 70(1), pages 191-221, January.
    8. D’Amuri, Francesco & Marcucci, Juri, 2017. "The predictive power of Google searches in forecasting US unemployment," International Journal of Forecasting, Elsevier, vol. 33(4), pages 801-816.
    9. Moon, H.R. & Perron, B., 2012. "Beyond panel unit root tests: Using multiple testing to determine the nonstationarity properties of individual series in a panel," Journal of Econometrics, Elsevier, vol. 169(1), pages 29-33.
    10. Durbin, James & Koopman, Siem Jan, 2012. "Time Series Analysis by State Space Methods," OUP Catalogue, Oxford University Press, edition 2, number 9780199641178, Decembrie.
    11. Tanya Suhoy, 2009. "Query Indices and a 2008 Downturn: Israeli Data," Bank of Israel Working Papers 2009.06, Bank of Israel.
    12. repec:hal:journl:peer-00844811 is not listed on IDEAS
    13. Bai, Jushan, 2004. "Estimating cross-section common stochastic trends in nonstationary panel data," Journal of Econometrics, Elsevier, vol. 122(1), pages 137-183, September.
    14. Hyunyoung Choi & Hal Varian, 2012. "Predicting the Present with Google Trends," The Economic Record, The Economic Society of Australia, vol. 88(s1), pages 2-9, June.
    15. Bai, Jushan & Ng, Serena, 2008. "Forecasting economic time series using targeted predictors," Journal of Econometrics, Elsevier, vol. 146(2), pages 304-317, October.
    16. Hindrayanto, Irma & Koopman, Siem Jan & de Winter, Jasper, 2016. "Forecasting and nowcasting economic growth in the euro area using factor models," International Journal of Forecasting, Elsevier, vol. 32(4), pages 1284-1305.
    17. Elliott, Graham & Rothenberg, Thomas J & Stock, James H, 1996. "Efficient Tests for an Autoregressive Unit Root," Econometrica, Econometric Society, vol. 64(4), pages 813-836, July.
    18. Andrew Harvey & Chia‐Hui Chung, 2000. "Estimating the underlying change in unemployment in the UK," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 163(3), pages 303-309.
    19. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    20. Benedikt Maas, 2020. "Short‐term forecasting of the US unemployment rate," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(3), pages 394-411, April.
    21. Naccarato, Alessia & Falorsi, Stefano & Loriga, Silvia & Pierini, Andrea, 2018. "Combining official and Google Trends data to forecast the Italian youth unemployment rate," Technological Forecasting and Social Change, Elsevier, vol. 130(C), pages 114-122.
    22. Pfeffermann, Danny, 1991. "Estimation and Seasonal Adjustment of Population Means Using Data from Repeated Surveys," Journal of Business & Economic Statistics, American Statistical Association, vol. 9(2), pages 163-175, April.
    23. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Caterina Schiavoni & Siem Jan Koopman & Franz Palm & Stephan Smeekes & Jan van den Brakel, 2021. "Time-varying state correlations in state space models and their estimation via indirect inference," Tinbergen Institute Discussion Papers 21-020/III, Tinbergen Institute.
    2. Danny Pfeffermann, 2022. "Time series modelling of repeated survey data for estimation of finite population parameters," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(4), pages 1757-1777, October.
    3. Bantis, Evripidis & Clements, Michael P. & Urquhart, Andrew, 2023. "Forecasting GDP growth rates in the United States and Brazil using Google Trends," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1909-1924.
    4. Federico Bassetti & Giulia Carallo & Roberto Casarin, 2022. "First-order integer-valued autoregressive processes with Generalized Katz innovations," Papers 2202.02029, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bennedsen, Mikkel & Hillebrand, Eric & Koopman, Siem Jan, 2021. "Modeling, forecasting, and nowcasting U.S. CO2 emissions using many macroeconomic predictors," Energy Economics, Elsevier, vol. 96(C).
    2. Daniel Borup & Erik Christian Montes Schütte, 2022. "In Search of a Job: Forecasting Employment Growth Using Google Trends," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 40(1), pages 186-200, January.
    3. Cepni, Oguzhan & Güney, I. Ethem & Swanson, Norman R., 2019. "Nowcasting and forecasting GDP in emerging markets using global financial and macroeconomic diffusion indexes," International Journal of Forecasting, Elsevier, vol. 35(2), pages 555-572.
    4. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.
    5. Stig Vinther Møller & Thomas Pedersen & Erik Christian Montes Schütte & Allan Timmermann, 2024. "Search and Predictability of Prices in the Housing Market," Management Science, INFORMS, vol. 70(1), pages 415-438, January.
    6. Poncela, Pilar & Ruiz, Esther & Miranda, Karen, 2021. "Factor extraction using Kalman filter and smoothing: This is not just another survey," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1399-1425.
    7. Bantis, Evripidis & Clements, Michael P. & Urquhart, Andrew, 2023. "Forecasting GDP growth rates in the United States and Brazil using Google Trends," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1909-1924.
    8. Gutiérrez, Antonio, 2023. "La brecha de género en el emprendimiento y la cultura emprendedora: Evidencia con Google Trends [Entrepreneurship gender gap and entrepreneurial culture: Evidence from Google Trends]," MPRA Paper 115876, University Library of Munich, Germany.
    9. Marie Bessec, 2013. "Short‐Term Forecasts of French GDP: A Dynamic Factor Model with Targeted Predictors," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 32(6), pages 500-511, September.
    10. Erik Christian Montes Schütte, 2018. "In Search of a Job: Forecasting Employment Growth in the US using Google Trends," CREATES Research Papers 2018-25, Department of Economics and Business Economics, Aarhus University.
    11. Jan A. Brakel & Sabine Krieg, 2016. "Small area estimation with state space common factor models for rotating panels," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(3), pages 763-791, June.
    12. Mihaela, Simionescu, 2020. "Improving unemployment rate forecasts at regional level in Romania using Google Trends," Technological Forecasting and Social Change, Elsevier, vol. 155(C).
    13. Weigand Roland & Wanger Susanne & Zapf Ines, 2018. "Factor Structural Time Series Models for Official Statistics with an Application to Hours Worked in Germany," Journal of Official Statistics, Sciendo, vol. 34(1), pages 265-301, March.
    14. Krieg, Sabine & van den Brakel, Jan A., 2012. "Estimation of the monthly unemployment rate for six domains through structural time series modelling with cointegrated trends," Computational Statistics & Data Analysis, Elsevier, vol. 56(10), pages 2918-2933.
    15. Caio Gonçalves & Luna Hidalgo & Denise Silva & Jan van den Brakel, 2022. "Single‐month unemployment rate estimates for the Brazilian Labour Force Survey using state‐space models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(4), pages 1707-1732, October.
    16. Ardia, David & Bluteau, Keven & Boudt, Kris, 2019. "Questioning the news about economic growth: Sparse forecasting using thousands of news-based sentiment values," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1370-1386.
    17. repec:dau:papers:123456789/10079 is not listed on IDEAS
    18. Julius Stakenas, 2012. "Generating short-term forecasts of the Lithuanian GDP using factor models," Bank of Lithuania Working Paper Series 13, Bank of Lithuania.
    19. Caterina Schiavoni & Siem Jan Koopman & Franz Palm & Stephan Smeekes & Jan van den Brakel, 2021. "Time-varying state correlations in state space models and their estimation via indirect inference," Tinbergen Institute Discussion Papers 21-020/III, Tinbergen Institute.
    20. Oxana Babecka Kucharcukova & Jan Bruha, 2016. "Nowcasting the Czech Trade Balance," Working Papers 2016/11, Czech National Bank.
    21. Claudio Morana, 2014. "Factor Vector Autoregressive Estimation of Heteroskedastic Persistent and Non Persistent Processes Subject to Structural Breaks," Working Papers 273, University of Milano-Bicocca, Department of Economics, revised May 2014.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1901.11355. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.