IDEAS home Printed from https://ideas.repec.org/p/kan/wpaper/202516.html
   My bibliography  Save this paper

Shrinkage Estimation and Identification of Latent Group Structures in Panel Data with Interactive Fixed Effects

Author

Listed:
  • Ali Mehrabani

    (Department of Economics, University of Kansas, Lawrence, KS 66045)

  • Shahnaz Parsaeian

    (Department of Economics, University of Kansas, Lawrence, KS 66045)

Abstract

This paper provides a framework for joint shrinkage estimation and identification of latent group structures in panel data models with interactive fixed effects and large number of explanatory variables. The latent structure of the model allows individuals to be classified into a number of groups where the number of groups and/or each individual’s group identity are unknown. A doubly penalized principal component estimation procedure using a pairwise fusion penalty and an adaptive LASSO (least absolute shrinkage and selection operator) penalty is introduced to detect the latent group structure and select the relevant regressors. To implement the proposed approach, an alternating direction method of multipliers algorithm has been developed. The proposed method is further illustrated by simulation studies and an empirical application of economic growth across various countries which demonstrate the good performance of the method.

Suggested Citation

  • Ali Mehrabani & Shahnaz Parsaeian, 2025. "Shrinkage Estimation and Identification of Latent Group Structures in Panel Data with Interactive Fixed Effects," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 202516, University of Kansas, Department of Economics.
  • Handle: RePEc:kan:wpaper:202516
    as

    Download full text from publisher

    File URL: https://kuwpaper.ku.edu/2025Papers/202516.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zou, Hui, 2006. "The Adaptive Lasso and Its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1418-1429, December.
    2. Vasilis Sarafidis & Neville Weber, 2015. "A Partially Heterogeneous Framework for Analyzing Panel Data," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 77(2), pages 274-296, April.
    3. Robert C. Feenstra & Robert Inklaar & Marcel P. Timmer, 2015. "The Next Generation of the Penn World Table," American Economic Review, American Economic Association, vol. 105(10), pages 3150-3182, October.
    4. Lu, Xun & Su, Liangjun, 2016. "Shrinkage estimation of dynamic panel data models with interactive fixed effects," Journal of Econometrics, Elsevier, vol. 190(1), pages 148-175.
    5. Jushan Bai & Serena Ng, 2002. "Determining the Number of Factors in Approximate Factor Models," Econometrica, Econometric Society, vol. 70(1), pages 191-221, January.
    6. Greenaway-McGrevy, Ryan & Han, Chirok & Sul, Donggyu, 2012. "Asymptotic distribution of factor augmented estimators for panel regression," Journal of Econometrics, Elsevier, vol. 169(1), pages 48-53.
    7. Su, Liangjun & Chen, Qihui, 2013. "Testing Homogeneity In Panel Data Models With Interactive Fixed Effects," Econometric Theory, Cambridge University Press, vol. 29(6), pages 1079-1135, December.
    8. Steve Bond & Asli Leblebicioglu & Fabio Schiantarelli, 2010. "Capital accumulation and growth: a new look at the empirical evidence," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(7), pages 1073-1099, November/.
    9. Hyungsik Roger Moon & Martin Weidner, 2015. "Linear Regression for Panel With Unknown Number of Factors as Interactive Fixed Effects," Econometrica, Econometric Society, vol. 83(4), pages 1543-1579, July.
    10. Wuyi Wang & Peter C. B. Phillips & Liangjun Su, 2018. "Homogeneity pursuit in panel data models: Theory and application," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 33(6), pages 797-815, September.
    11. Li, Liyao & Yang, Zhenlin, 2021. "Spatial dynamic panel data models with correlated random effects," Journal of Econometrics, Elsevier, vol. 221(2), pages 424-454.
    12. Xun Lu & Liangjun Su, 2017. "Determining the number of groups in latent panel structures with an application to income and democracy," Quantitative Economics, Econometric Society, vol. 8(3), pages 729-760, November.
    13. Liangjun Su & Zhentao Shi & Peter C. B. Phillips, 2016. "Identifying Latent Structures in Panel Data," Econometrica, Econometric Society, vol. 84, pages 2215-2264, November.
    14. Moon, Hyungsik Roger & Weidner, Martin, 2017. "Dynamic Linear Panel Regression Models With Interactive Fixed Effects," Econometric Theory, Cambridge University Press, vol. 33(1), pages 158-195, February.
    15. Bester, C. Alan & Hansen, Christian B., 2016. "Grouped effects estimators in fixed effects models," Journal of Econometrics, Elsevier, vol. 190(1), pages 197-208.
    16. Leeb, Hannes & Pötscher, Benedikt M., 2005. "Model Selection And Inference: Facts And Fiction," Econometric Theory, Cambridge University Press, vol. 21(1), pages 21-59, February.
    17. Robert J. Barro, 1991. "Economic Growth in a Cross Section of Countries," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 106(2), pages 407-443.
    18. Tomohiro Ando & Jushan Bai, 2016. "Panel Data Models with Grouped Factor Structure Under Unknown Group Membership," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(1), pages 163-191, January.
    19. Lin Chang-Ching & Ng Serena, 2012. "Estimation of Panel Data Models with Parameter Heterogeneity when Group Membership is Unknown," Journal of Econometric Methods, De Gruyter, vol. 1(1), pages 42-55, August.
    20. Chudik, Alexander & Pesaran, M. Hashem, 2015. "Common correlated effects estimation of heterogeneous dynamic panel data models with weakly exogenous regressors," Journal of Econometrics, Elsevier, vol. 188(2), pages 393-420.
    21. Liangjun Su & Xia Wang & Sainan Jin, 2019. "Sieve Estimation of Time-Varying Panel Data Models With Latent Structures," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 37(2), pages 334-349, April.
    22. Robert Tibshirani & Michael Saunders & Saharon Rosset & Ji Zhu & Keith Knight, 2005. "Sparsity and smoothness via the fused lasso," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(1), pages 91-108, February.
    23. Su, Liangjun & Jin, Sainan & Zhang, Yonghui, 2015. "Specification test for panel data models with interactive fixed effects," Journal of Econometrics, Elsevier, vol. 186(1), pages 222-244.
    24. M. Hashem Pesaran, 2006. "Estimation and Inference in Large Heterogeneous Panels with a Multifactor Error Structure," Econometrica, Econometric Society, vol. 74(4), pages 967-1012, July.
    25. Sun, Yixiao X, 2005. "Estimation and Inference in Panel Structure Models," University of California at San Diego, Economics Working Paper Series qt5tf1231k, Department of Economics, UC San Diego.
    26. Bai, Jushan & Li, Kunpeng, 2021. "Dynamic spatial panel data models with common shocks," Journal of Econometrics, Elsevier, vol. 224(1), pages 134-160.
    27. Markus Eberhardt & Francis Teal, 2011. "Econometrics For Grumblers: A New Look At The Literature On Cross‐Country Growth Empirics," Journal of Economic Surveys, Wiley Blackwell, vol. 25(1), pages 109-155, February.
    28. Degui Li & Junhui Qian & Liangjun Su, 2016. "Panel Data Models With Interactive Fixed Effects and Multiple Structural Breaks," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1804-1819, October.
    29. Su, Liangjun & Jin, Sainan, 2012. "Sieve estimation of panel data models with cross section dependence," Journal of Econometrics, Elsevier, vol. 169(1), pages 34-47.
    30. Qian, Junhui & Su, Liangjun, 2016. "Shrinkage estimation of common breaks in panel data models via adaptive group fused Lasso," Journal of Econometrics, Elsevier, vol. 191(1), pages 86-109.
    31. Okui, Ryo & Wang, Wendun, 2021. "Heterogeneous structural breaks in panel data models," Journal of Econometrics, Elsevier, vol. 220(2), pages 447-473.
    32. Liu, Ruiqi & Shang, Zuofeng & Zhang, Yonghui & Zhou, Qiankun, 2020. "Identification and estimation in panel models with overspecified number of groups," Journal of Econometrics, Elsevier, vol. 215(2), pages 574-590.
    33. Mehrabani, Ali, 2023. "Estimation and identification of latent group structures in panel data," Journal of Econometrics, Elsevier, vol. 235(2), pages 1464-1482.
    34. Su, Liangjun & Ju, Gaosheng, 2018. "Identifying latent grouped patterns in panel data models with interactive fixed effects," Journal of Econometrics, Elsevier, vol. 206(2), pages 554-573.
    35. Miao, Ke & Su, Liangjun & Wang, Wendun, 2020. "Panel threshold regressions with latent group structures," Journal of Econometrics, Elsevier, vol. 214(2), pages 451-481.
    36. Wang, Wei & Xiao, Zhijie & Ren, Yanyan & Yan, Xiaodong, 2023. "A bi-integrative analysis of two-dimensional heterogeneous panel data models," Economics Letters, Elsevier, vol. 230(C).
    37. Jushan Bai, 2009. "Panel Data Models With Interactive Fixed Effects," Econometrica, Econometric Society, vol. 77(4), pages 1229-1279, July.
    38. Ming Yuan & Yi Lin, 2006. "Model selection and estimation in regression with grouped variables," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(1), pages 49-67, February.
    39. Shujie Ma & Jian Huang, 2017. "A Concave Pairwise Fusion Approach to Subgroup Analysis," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(517), pages 410-423, January.
    40. Nazrul Islam, 1995. "Growth Empirics: A Panel Data Approach," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 110(4), pages 1127-1170.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Su, Liangjun & Ju, Gaosheng, 2018. "Identifying latent grouped patterns in panel data models with interactive fixed effects," Journal of Econometrics, Elsevier, vol. 206(2), pages 554-573.
    2. Pionati, Alessandro, 2025. "Latent grouped structures in panel data: a review," MPRA Paper 123954, University Library of Munich, Germany.
    3. Mehrabani, Ali, 2023. "Estimation and identification of latent group structures in panel data," Journal of Econometrics, Elsevier, vol. 235(2), pages 1464-1482.
    4. Wang, Yiren & Phillips, Peter C.B. & Su, Liangjun, 2024. "Panel data models with time-varying latent group structures," Journal of Econometrics, Elsevier, vol. 240(1).
    5. Huang, Wenxin & Jin, Sainan & Phillips, Peter C.B. & Su, Liangjun, 2021. "Nonstationary panel models with latent group structures and cross-section dependence," Journal of Econometrics, Elsevier, vol. 221(1), pages 198-222.
    6. Lu, Xun & Su, Liangjun, 2016. "Shrinkage estimation of dynamic panel data models with interactive fixed effects," Journal of Econometrics, Elsevier, vol. 190(1), pages 148-175.
    7. Paul Haimerl & Stephan Smeekes & Ines Wilms, 2025. "Estimation of Latent Group Structures in Time-Varying Panel Data Models," Papers 2503.23165, arXiv.org, revised Nov 2025.
    8. Miao, Ke & Su, Liangjun & Wang, Wendun, 2020. "Panel threshold regressions with latent group structures," Journal of Econometrics, Elsevier, vol. 214(2), pages 451-481.
    9. Wang, Wuyi & Su, Liangjun, 2021. "Identifying latent group structures in nonlinear panels," Journal of Econometrics, Elsevier, vol. 220(2), pages 272-295.
    10. Hong, Shengjie & Su, Liangjun & Jiang, Tao, 2023. "Profile GMM estimation of panel data models with interactive fixed effects," Journal of Econometrics, Elsevier, vol. 235(2), pages 927-948.
    11. Oguzhan Akgun & Ryo Okui, 2025. "Robust Inference Methods for Latent Group Panel Models under Possible Group Non-Separation," Papers 2511.18550, arXiv.org.
    12. Vasilis Sarafidis & Tom Wansbeek, 2020. "Celebrating 40 Years of Panel Data Analysis: Past, Present and Future," Monash Econometrics and Business Statistics Working Papers 6/20, Monash University, Department of Econometrics and Business Statistics.
    13. Katerina Chrysikou & George Kapetanios, 2024. "Heterogeneous Grouping Structures in Panel Data," Papers 2407.19509, arXiv.org.
    14. Okui, Ryo & Wang, Wendun, 2021. "Heterogeneous structural breaks in panel data models," Journal of Econometrics, Elsevier, vol. 220(2), pages 447-473.
    15. Su, Liangjun & Jin, Sainan & Zhang, Yonghui, 2015. "Specification test for panel data models with interactive fixed effects," Journal of Econometrics, Elsevier, vol. 186(1), pages 222-244.
    16. Wang, Xia & Jin, Sainan & Li, Yingxing & Qian, Junhui & Su, Liangjun, 2025. "On time-varying panel data models with time-varying interactive fixed effects," Journal of Econometrics, Elsevier, vol. 249(PB).
    17. Miao, Ke & Li, Kunpeng & Su, Liangjun, 2020. "Panel threshold models with interactive fixed effects," Journal of Econometrics, Elsevier, vol. 219(1), pages 137-170.
    18. Lumsdaine, Robin L. & Okui, Ryo & Wang, Wendun, 2023. "Estimation of panel group structure models with structural breaks in group memberships and coefficients," Journal of Econometrics, Elsevier, vol. 233(1), pages 45-65.
    19. Su, Liangjun & Wang, Wuyi & Xu, Xingbai, 2023. "Identifying latent group structures in spatial dynamic panels," Journal of Econometrics, Elsevier, vol. 235(2), pages 1955-1980.
    20. Saptorshee Kanto Chakraborty & Massimiliano Mazzanti, 2021. "Revisiting the literature on the dynamic Environmental Kuznets Curves using a latent structure approach," Economia Politica: Journal of Analytical and Institutional Economics, Springer;Fondazione Edison, vol. 38(3), pages 923-941, October.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C33 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Models with Panel Data; Spatio-temporal Models
    • C38 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Classification Methdos; Cluster Analysis; Principal Components; Factor Analysis
    • C51 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Construction and Estimation

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kan:wpaper:202516. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Professor Zongwu Cai (email available below). General contact details of provider: https://edirc.repec.org/data/deuksus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.