IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/111431.html
   My bibliography  Save this paper

Large-scale generalized linear longitudinal data models with grouped patterns of unobserved heterogeneity

Author

Listed:
  • Ando, Tomohiro
  • Bai, Jushan

Abstract

This paper provides methods for flexibly capturing unobservable heterogeneity from longitudinal data in the context of an exponential family of distributions. The group memberships of individual units are left unspecified, and their heterogeneity is influenced by group-specific unobservable structures, as well as heterogeneous regression coefficients. We discuss a computationally efficient estimation method and derive the corresponding asymptotic theory. The established asymptotic theory includes verifying the uniform consistency of the estimated group membership. To test the heterogeneous regression coefficients within groups, we propose the Swamy-type test, which considers unobserved heterogeneity. We apply the proposed method to study the market structure of the taxi industry in New York City. Our method reveals interesting important insights from large-scale longitudinal data that consist of over 450 million data points.

Suggested Citation

  • Ando, Tomohiro & Bai, Jushan, 2021. "Large-scale generalized linear longitudinal data models with grouped patterns of unobserved heterogeneity," MPRA Paper 111431, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:111431
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/111431/1/MPRA_paper_111431.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ando, Tomohiro & Bai, Jushan, 2015. "A simple new test for slope homogeneity in panel data models with interactive effects," Economics Letters, Elsevier, vol. 136(C), pages 112-117.
    2. Fernández-Val, Iván & Weidner, Martin, 2016. "Individual and time effects in nonlinear panel models with large N, T," Journal of Econometrics, Elsevier, vol. 192(1), pages 291-312.
    3. Stéphane Bonhomme & Elena Manresa, 2015. "Grouped Patterns of Heterogeneity in Panel Data," Econometrica, Econometric Society, vol. 83(3), pages 1147-1184, May.
    4. Boneva, L. & Linton, O., 2017. "A Discrete Choice Model For Large Heterogeneous Panels with Interactive Fixed Effects with an Application to the Determinants of Corporate Bond Issuance," Cambridge Working Papers in Economics 1703, Faculty of Economics, University of Cambridge.
    5. Nan Kong & Andrew J. Schaefer & Brady Hunsaker & Mark S. Roberts, 2010. "Maximizing the Efficiency of the U.S. Liver Allocation System Through Region Design," Management Science, INFORMS, vol. 56(12), pages 2111-2122, December.
    6. Liangjun Su & Zhentao Shi & Peter C. B. Phillips, 2016. "Identifying Latent Structures in Panel Data," Econometrica, Econometric Society, vol. 84, pages 2215-2264, November.
    7. Hashem Pesaran, M. & Yamagata, Takashi, 2008. "Testing slope homogeneity in large panels," Journal of Econometrics, Elsevier, vol. 142(1), pages 50-93, January.
    8. Karyne B. Charbonneau, 2017. "Multiple fixed effects in binary response panel data models," Econometrics Journal, Royal Economic Society, vol. 20(3), pages 1-13, October.
    9. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    10. M. Hashem Pesaran, 2006. "Estimation and Inference in Large Heterogeneous Panels with a Multifactor Error Structure," Econometrica, Econometric Society, vol. 74(4), pages 967-1012, July.
    11. Tomohiro Ando & Jushan Bai, 2016. "Panel Data Models with Grouped Factor Structure Under Unknown Group Membership," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(1), pages 163-191, January.
    12. Tomohiro Ando & Jushan Bai, 2017. "Clustering Huge Number of Financial Time Series: A Panel Data Approach With High-Dimensional Predictors and Factor Structures," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 1182-1198, July.
    13. Moon, Hyungsik Roger & Shum, Matthew & Weidner, Martin, 2018. "Estimation of random coefficients logit demand models with interactive fixed effects," Journal of Econometrics, Elsevier, vol. 206(2), pages 613-644.
    14. Jushan Bai, 2009. "Panel Data Models With Interactive Fixed Effects," Econometrica, Econometric Society, vol. 77(4), pages 1229-1279, July.
    15. Chang Hee Park & Young-Hoon Park, 2016. "Investigating Purchase Conversion by Uncovering Online Visit Patterns," Marketing Science, INFORMS, vol. 35(6), pages 894-914, November.
    16. Wang, Wuyi & Su, Liangjun, 2021. "Identifying latent group structures in nonlinear panels," Journal of Econometrics, Elsevier, vol. 220(2), pages 272-295.
    17. Stephen L. France & Sanjoy Ghose, 2016. "An Analysis and Visualization Methodology for Identifying and Testing Market Structure," Marketing Science, INFORMS, vol. 35(1), pages 182-197, January.
    18. Wuyi Wang & Peter C. B. Phillips & Liangjun Su, 2018. "Homogeneity pursuit in panel data models: Theory and application," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 33(6), pages 797-815, September.
    19. Swamy, P A V B, 1970. "Efficient Inference in a Random Coefficient Regression Model," Econometrica, Econometric Society, vol. 38(2), pages 311-323, March.
    20. Lin Chang-Ching & Ng Serena, 2012. "Estimation of Panel Data Models with Parameter Heterogeneity when Group Membership is Unknown," Journal of Econometric Methods, De Gruyter, vol. 1(1), pages 42-55, August.
    21. Lena Boneva & Oliver Linton, 2017. "A discrete†choice model for large heterogeneous panels with interactive fixed effects with an application to the determinants of corporate bond issuance," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(7), pages 1226-1243, November.
    22. Bai, Jushan & Ng, Serena, 2013. "Principal components estimation and identification of static factors," Journal of Econometrics, Elsevier, vol. 176(1), pages 18-29.
    23. Heard, Nicholas A. & Holmes, Christopher C. & Stephens, David A., 2006. "A Quantitative Study of Gene Regulation Involved in the Immune Response of Anopheline Mosquitoes: An Application of Bayesian Hierarchical Clustering of Curves," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 18-29, March.
    24. Mark S. Handcock & Adrian E. Raftery & Jeremy M. Tantrum, 2007. "Model‐based clustering for social networks," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 170(2), pages 301-354, March.
    25. Ahlquist, John S. & Breunig, Christian, 2012. "Model-based Clustering and Typologies in the Social Sciences," Political Analysis, Cambridge University Press, vol. 20(1), pages 92-112, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wang, Yiren & Phillips, Peter C.B. & Su, Liangjun, 2024. "Panel data models with time-varying latent group structures," Journal of Econometrics, Elsevier, vol. 240(1).
    2. Ando, Tomohiro & Bai, Jushan & Li, Kunpeng, 2022. "Bayesian and maximum likelihood analysis of large-scale panel choice models with unobserved heterogeneity," Journal of Econometrics, Elsevier, vol. 230(1), pages 20-38.
    3. Denis Chetverikov & Elena Manresa, 2022. "Spectral and post-spectral estimators for grouped panel data models," Papers 2212.13324, arXiv.org, revised Dec 2022.
    4. Nibbering, D. & Paap, R., 2019. "Panel Forecasting with Asymmetric Grouping," Econometric Institute Research Papers EI-2019-30, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    5. Okui, Ryo & Wang, Wendun, 2021. "Heterogeneous structural breaks in panel data models," Journal of Econometrics, Elsevier, vol. 220(2), pages 447-473.
    6. Mehrabani, Ali, 2023. "Estimation and identification of latent group structures in panel data," Journal of Econometrics, Elsevier, vol. 235(2), pages 1464-1482.
    7. Su, Liangjun & Ju, Gaosheng, 2018. "Identifying latent grouped patterns in panel data models with interactive fixed effects," Journal of Econometrics, Elsevier, vol. 206(2), pages 554-573.
    8. Chen, Mingli & Fernández-Val, Iván & Weidner, Martin, 2021. "Nonlinear factor models for network and panel data," Journal of Econometrics, Elsevier, vol. 220(2), pages 296-324.
    9. Miao, Ke & Su, Liangjun & Wang, Wendun, 2020. "Panel threshold regressions with latent group structures," Journal of Econometrics, Elsevier, vol. 214(2), pages 451-481.
    10. Ye, Xiaoqing & Xu, Juan & Wu, Xiangjun, 2018. "Estimation of an unbalanced panel data Tobit model with interactive effects," Journal of choice modelling, Elsevier, vol. 28(C), pages 108-123.
    11. Wang, Wuyi & Su, Liangjun, 2021. "Identifying latent group structures in nonlinear panels," Journal of Econometrics, Elsevier, vol. 220(2), pages 272-295.
    12. Thibaut Lamadon & Elena Manresa & Stephane Bonhomme, 2016. "Discretizing Unobserved Heterogeneity," 2016 Meeting Papers 1536, Society for Economic Dynamics.
    13. Claudia Pigini & Alessandro Pionati & Francesco Valentini, 2023. "Specification testing with grouped fixed effects," Papers 2310.01950, arXiv.org.
    14. Gao, Jiti & Liu, Fei & Peng, Bin & Yan, Yayi, 2023. "Binary response models for heterogeneous panel data with interactive fixed effects," Journal of Econometrics, Elsevier, vol. 235(2), pages 1654-1679.
    15. Didier Nibbering & Richard Paap, 2024. "Forecasting carbon emissions using asymmetric grouping," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 43(6), pages 2228-2256, September.
    16. Yanbo Liu & Peter C. B. Phillips & Jun Yu, 2023. "A Panel Clustering Approach To Analyzing Bubble Behavior," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 64(4), pages 1347-1395, November.
    17. Yiren Wang & Liangjun Su & Yichong Zhang, 2022. "Low-rank Panel Quantile Regression: Estimation and Inference," Papers 2210.11062, arXiv.org.
    18. Jie Wei & Yonghui Zhang, 2022. "Panel Probit Models with Time‐Varying Individual Effects: Reestimating the Effects of Fertility on Female Labour Participation," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 84(4), pages 799-829, August.
    19. Yu, Lu & Gu, Jiaying & Volgushev, Stanislav, 2024. "Spectral clustering with variance information for group structure estimation in panel data," Journal of Econometrics, Elsevier, vol. 241(1).
    20. Leng, Xuan & Chen, Heng & Wang, Wendun, 2023. "Multi-dimensional latent group structures with heterogeneous distributions," Journal of Econometrics, Elsevier, vol. 233(1), pages 1-21.

    More about this item

    Keywords

    Clustering; Factor analysis; Generalized linear models; Longitudinal data; Unobserved heterogeneity.;
    All these keywords.

    JEL classification:

    • C33 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Models with Panel Data; Spatio-temporal Models
    • C38 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Classification Methdos; Cluster Analysis; Principal Components; Factor Analysis
    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:111431. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.