IDEAS home Printed from https://ideas.repec.org/p/hhs/gunwpe/0727.html
   My bibliography  Save this paper

Confidence Set for Group Membership

Author

Listed:
  • Dzemski, Andreas

    (Department of Economics, School of Business, Economics and Law, Göteborg University)

  • Okui, Ryo

    (Department of Economics, School of Business, Economics and Law, Göteborg University)

Abstract

We develop new procedures to quantify the statistical uncertainty from sorting units in panel data into groups using data-driven clustering algorithms. In our setting, each unit belongs to one of a finite number of latent groups and its regression curve is determined by which group it belongs to. Our main contribution is a new joint confidence set for group membership. Each element of the joint confidence set is a vector of possible group assignments for all units. The vector of true group memberships is contained in the confidence set with a pre-specified probability. The confidence set inverts a test for group membership. This test exploits a characterization of the true group memberships by a system of moment inequalities. Our procedure solves a high-dimensional one-sided testing problem and tests group membership simultaneously for all units. We also propose a procedure for identifying units for which group membership is obviously determined. These units can be ignored when computing critical values. We justify the joint confidence set under N, T → ∞ asymptotics where we allow T to be much smaller than N. Our arguments rely on the theory of self-normalized sums and high-dimensional central limit theorems. We contribute new theoretical results for testing problems with a large number of moment inequalities, including an anti-concentration inequality for the quasi-likelihood ratio (QLR) statistic. Monte Carlo results indicate that our confidence set has adequate coverage and is informative. We illustrate the practical relevance of our confidence set in two applications.

Suggested Citation

  • Dzemski, Andreas & Okui, Ryo, 2018. "Confidence Set for Group Membership," Working Papers in Economics 727, University of Gothenburg, Department of Economics.
    • Handle: RePEc:hhs:gunwpe:0727
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/2077/55922
    File Function: Full text
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ivan A. Canay & Azeem M. Shaikh, 2016. "Practical and theoretical advances in inference for partially identified models," CeMMAP working papers CWP05/16, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    2. Heckman, James & Singer, Burton, 1984. "A Method for Minimizing the Impact of Distributional Assumptions in Econometric Models for Duration Data," Econometrica, Econometric Society, vol. 52(2), pages 271-320, March.
    3. Wuyi Wang & Peter C. B. Phillips & Liangjun Su, 2018. "Homogeneity pursuit in panel data models: Theory and application," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 33(6), pages 797-815, September.
    4. Xun Lu & Liangjun Su, 2017. "Determining the number of groups in latent panel structures with an application to income and democracy," Quantitative Economics, Econometric Society, vol. 8(3), pages 729-760, November.
    5. Joseph P. Romano & Azeem M. Shaikh & Michael Wolf, 2014. "A Practical Two‐Step Method for Testing Moment Inequalities," Econometrica, Econometric Society, vol. 82, pages 1979-2002, September.
    6. Thibaut Lamadon & Elena Manresa & Stephane Bonhomme, 2016. "Discretizing Unobserved Heterogeneity," 2016 Meeting Papers 1536, Society for Economic Dynamics.
    7. Allen, Roy, 2018. "Testing moment inequalities: Selection versus recentering," Economics Letters, Elsevier, vol. 162(C), pages 124-126.
    8. Michael J. Grayling & Adrian Mander, 2015. "MVTNORM: Stata module to work with the multivariate normal and multivariate t distributions, with and without variable truncation," Statistical Software Components S458043, Boston College Department of Economics, revised 24 Dec 2021.
    9. Jinyong Hahn & Guido Kuersteiner, 2002. "Asymptotically Unbiased Inference for a Dynamic Panel Model with Fixed Effects when Both "n" and "T" Are Large," Econometrica, Econometric Society, vol. 70(4), pages 1639-1657, July.
    10. repec:hal:spmain:info:hdl:2441/eu4vqp9ompqllr09ij4j0h0h1 is not listed on IDEAS
    11. Tomohiro Ando & Jushan Bai, 2016. "Panel Data Models with Grouped Factor Structure Under Unknown Group Membership," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(1), pages 163-191, January.
    12. Lin Chang-Ching & Ng Serena, 2012. "Estimation of Panel Data Models with Parameter Heterogeneity when Group Membership is Unknown," Journal of Econometric Methods, De Gruyter, vol. 1(1), pages 1-14, August.
    13. Rosen, Adam M., 2008. "Confidence sets for partially identified parameters that satisfy a finite number of moment inequalities," Journal of Econometrics, Elsevier, vol. 146(1), pages 107-117, September.
    14. Donald W. K. Andrews & Gustavo Soares, 2010. "Inference for Parameters Defined by Moment Inequalities Using Generalized Moment Selection," Econometrica, Econometric Society, vol. 78(1), pages 119-157, January.
    15. Donald W. K. Andrews & Panle Jia Barwick, 2012. "Inference for Parameters Defined by Moment Inequalities: A Recommended Moment Selection Procedure," Econometrica, Econometric Society, vol. 80(6), pages 2805-2826, November.
    16. Hahn, Jinyong & Moon, Hyungsik Roger, 2010. "Panel Data Models With Finite Number Of Multiple Equilibria," Econometric Theory, Cambridge University Press, vol. 26(3), pages 863-881, June.
    17. David Neumark & William Wascher, 1992. "Employment Effects of Minimum and Subminimum Wages: Panel Data on State Minimum Wage Laws," ILR Review, Cornell University, ILR School, vol. 46(1), pages 55-81, October.
    18. Mayya Zhilova, 2015. "Simultaneous likelihood-based bootstrap confidence sets for a large number of models," SFB 649 Discussion Papers SFB649DP2015-031, Sonderforschungsbereich 649, Humboldt University, Berlin, Germany.
    19. Federico A. Bugni, 2010. "Bootstrap Inference in Partially Identified Models Defined by Moment Inequalities: Coverage of the Identified Set," Econometrica, Econometric Society, vol. 78(2), pages 735-753, March.
    20. Jiaying Gu & Stanislav Volgushev, 2018. "Panel Data Quantile Regression with Grouped Fixed Effects," Papers 1801.05041, arXiv.org, revised Aug 2018.
    21. Arindrajit Dube & T. William Lester & Michael Reich, 2010. "Minimum Wage Effects Across State Borders: Estimates Using Contiguous Counties," The Review of Economics and Statistics, MIT Press, vol. 92(4), pages 945-964, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andreas Dzemski & Ryo Okui, 2017. "Confidence set for group membership," Papers 1801.00332, arXiv.org, revised Nov 2023.
    2. Okui, Ryo & Wang, Wendun, 2021. "Heterogeneous structural breaks in panel data models," Journal of Econometrics, Elsevier, vol. 220(2), pages 447-473.
    3. Mehrabani, Ali, 2023. "Estimation and identification of latent group structures in panel data," Journal of Econometrics, Elsevier, vol. 235(2), pages 1464-1482.
    4. Hiroaki Kaido & Francesca Molinari & Jörg Stoye, 2019. "Confidence Intervals for Projections of Partially Identified Parameters," Econometrica, Econometric Society, vol. 87(4), pages 1397-1432, July.
    5. Ho, Kate & Rosen, Adam M., 2015. "Partial Identification in Applied Research: Benefits and Challenges," CEPR Discussion Papers 10883, C.E.P.R. Discussion Papers.
    6. Francesca Molinari, 2020. "Microeconometrics with Partial Identi?cation," CeMMAP working papers CWP15/20, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    7. Federico A. Bugni & Ivan A. Canay & Xiaoxia Shi, 2014. "Inference for functions of partially identified parameters in moment inequality models," CeMMAP working papers 22/14, Institute for Fiscal Studies.
    8. Lee, Sokbae & Song, Kyungchul & Whang, Yoon-Jae, 2018. "Testing For A General Class Of Functional Inequalities," Econometric Theory, Cambridge University Press, vol. 34(5), pages 1018-1064, October.
    9. Denis Chetverikov & Elena Manresa, 2022. "Spectral and post-spectral estimators for grouped panel data models," Papers 2212.13324, arXiv.org, revised Dec 2022.
    10. Sasaki, Yuya & Takahashi, Yuya & Xin, Yi & Hu, Yingyao, 2023. "Dynamic discrete choice models with incomplete data: Sharp identification," Journal of Econometrics, Elsevier, vol. 236(1).
    11. Ivan A. Canay & Azeem M. Shaikh, 2016. "Practical and theoretical advances in inference for partially identified models," CeMMAP working papers CWP05/16, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    12. Leng, Xuan & Chen, Heng & Wang, Wendun, 2023. "Multi-dimensional latent group structures with heterogeneous distributions," Journal of Econometrics, Elsevier, vol. 233(1), pages 1-21.
    13. Jorg Stoye, 2020. "A Simple, Short, but Never-Empty Confidence Interval for Partially Identified Parameters," Papers 2010.10484, arXiv.org, revised Dec 2020.
    14. Chen, Le-Yu & Szroeter, Jerzy, 2014. "Testing multiple inequality hypotheses: A smoothed indicator approach," Journal of Econometrics, Elsevier, vol. 178(P3), pages 678-693.
    15. Federico A. Bugni & Mehmet Caner & Anders Bredahl Kock & Soumendra Lahiri, 2016. "Inference in partially identified models with many moment inequalities using Lasso," CREATES Research Papers 2016-12, Department of Economics and Business Economics, Aarhus University.
    16. Zeng-Hua Lu & Alec Zuo, 2017. "Child disability, welfare payments, marital status and mothers’ labor supply: Evidence from Australia," Cogent Economics & Finance, Taylor & Francis Journals, vol. 5(1), pages 1339769-133, January.
    17. Francesca Molinari, 2019. "Econometrics with Partial Identification," CeMMAP working papers CWP25/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    18. Victor Chernozhukov & Denis Chetverikov & Kengo Kato, 2013. "Testing Many Moment Inequalities," CeMMAP working papers 65/13, Institute for Fiscal Studies.
    19. Donald S. Poskitt & Xueyan Zhao, 2023. "Bootstrap Hausdorff Confidence Regions for Average Treatment Effect Identified Sets," Monash Econometrics and Business Statistics Working Papers 9/23, Monash University, Department of Econometrics and Business Statistics.
    20. Arun G. Chandrasekhar & Victor Chernozhukov & Francesca Molinari & Paul Schrimpf, 2019. "Best Linear Approximations to Set Identified Functions: With an Application to the Gender Wage Gap," NBER Working Papers 25593, National Bureau of Economic Research, Inc.

    More about this item

    Keywords

    Panel data; grouped heterogeneity; clustering; confidence set; machine learning; moment inequalities; joint one-sided tests; self-normalized sums; high-dimensional CLT; anti-concentration for QLR;
    All these keywords.

    JEL classification:

    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models
    • C33 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Models with Panel Data; Spatio-temporal Models
    • C38 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Classification Methdos; Cluster Analysis; Principal Components; Factor Analysis

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hhs:gunwpe:0727. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Ann-Christin Räätäri Nyström (email available below). General contact details of provider: https://edirc.repec.org/data/naiguse.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.