IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2308.01596.html
   My bibliography  Save this paper

Individual Shrinkage for Random Effects

Author

Listed:
  • Raffaella Giacomini
  • Sokbae Lee
  • Silvia Sarpietro

Abstract

This paper develops a novel approach to random effects estimation and individual-level forecasting in micropanels, targeting individual accuracy rather than aggregate performance. The conventional shrinkage methods used in the literature, such as the James-Stein estimator and Empirical Bayes, target aggregate performance and can lead to inaccurate decisions at the individual level. We propose a complementary class of shrinkage estimators with individual weights (IW) that leverage an individual's own past history, instead of the cross-sectional dimension. This approach overcomes the "tyranny of the majority" inherent in existing methods, while relying on weaker assumptions. We discuss the theoretical optimality of IW and recommend using feasible weights determined through a Minimax Regret analysis in practice.

Suggested Citation

  • Raffaella Giacomini & Sokbae Lee & Silvia Sarpietro, 2023. "Individual Shrinkage for Random Effects," Papers 2308.01596, arXiv.org, revised May 2025.
  • Handle: RePEc:arx:papers:2308.01596
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2308.01596
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Fatih Guvenen & Fatih Karahan & Serdar Ozkan & Jae Song, 2015. "What Do Data on Millions of U.S. Workers Reveal about Life-Cycle Earnings Risk?," NBER Working Papers 20913, National Bureau of Economic Research, Inc.
    2. Thomas J. Kane & Douglas O. Staiger, 2008. "Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation," NBER Working Papers 14607, National Bureau of Economic Research, Inc.
    3. Martin Browning & Mette Ejrnæs & Javier Alvarez, 2010. "Modelling Income Processes with Lots of Heterogeneity," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 77(4), pages 1353-1381.
    4. Raj Chetty & Nathaniel Hendren, 2018. "The Impacts of Neighborhoods on Intergenerational Mobility I: Childhood Exposure Effects," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1107-1162.
    5. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    6. Fatih Guvenen & Fatih Karahan & Serdar Ozkan & Jae Song, 2021. "What Do Data on Millions of U.S. Workers Reveal About Lifecycle Earnings Dynamics?," Econometrica, Econometric Society, vol. 89(5), pages 2303-2339, September.
    7. M. Hashem Pesaran & Andreas Pick & Allan Timmermann, 2022. "Forecasting With Panel Data: Estimation Uncertainty Versus Parameter Heterogeneity," CESifo Working Paper Series 9690, CESifo.
    8. Costas Meghir & Luigi Pistaferri, 2004. "Income Variance Dynamics and Heterogeneity," Econometrica, Econometric Society, vol. 72(1), pages 1-32, January.
    9. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood," American Economic Review, American Economic Association, vol. 104(9), pages 2633-2679, September.
    10. Patrick Kline & Evan K Rose & Christopher R Walters, 2022. "Systemic Discrimination Among Large U.S. Employers [“Teachers and Student Achievement in the Chicago Public High Schools,”]," The Quarterly Journal of Economics, Oxford University Press, vol. 137(4), pages 1963-2036.
    11. Joshua D. Angrist & Peter D. Hull & Parag A. Pathak & Christopher R. Walters, 2017. "Leveraging Lotteries for School Value-Added: Testing and Estimation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(2), pages 871-919.
    12. Raj Chetty & Nathaniel Hendren, 2018. "The Impacts of Neighborhoods on Intergenerational Mobility II: County-Level Estimates," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1163-1228.
    13. Bradley Efron, 2016. "Empirical Bayes deconvolution estimates," Biometrika, Biometrika Trust, vol. 103(1), pages 1-20.
    14. Laura Liu & Hyungsik Roger Moon & Frank Schorfheide, 2020. "Forecasting With Dynamic Panel Data Models," Econometrica, Econometric Society, vol. 88(1), pages 171-201, January.
    15. James H. Stock & Mark W. Watson, 1998. "A Comparison of Linear and Nonlinear Univariate Models for Forecasting Macroeconomic Time Series," NBER Working Papers 6607, National Bureau of Economic Research, Inc.
    16. Gary Chamberlain & Keisuke Hirano, 1999. "Predictive Distributions based on Longitudinal Earnings Data," Annals of Economics and Statistics, GENES, issue 55-56, pages 211-242.
    17. Jiaying Gu & Roger Koenker, 2017. "Unobserved Heterogeneity in Income Dynamics: An Empirical Bayes Perspective," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 35(1), pages 1-16, January.
    18. Charles F. Manski, 2021. "Econometrics for Decision Making: Building Foundations Sketched by Haavelmo and Wald," Econometrica, Econometric Society, vol. 89(6), pages 2827-2853, November.
    19. Manuel Arellano & Stephen Bond, 1991. "Some Tests of Specification for Panel Data: Monte Carlo Evidence and an Application to Employment Equations," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 58(2), pages 277-297.
    20. Jason M. Fletcher & Leora I. Horwitz & Elizabeth Bradley, 2014. "Estimating the Value Added of Attending Physicians on Patient Outcomes," NBER Working Papers 20534, National Bureau of Economic Research, Inc.
    21. repec:adr:anecst:y:1999:i:55-56:p:08 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pietro Giorgio Lovaglio, 2025. "Cross‐Learning With Panel Data Modeling for Stacking and Forecast Time Series Employment in Europe," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 44(2), pages 753-780, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Raffaella Giacomini & Sokbae Lee & Silvia Sarpietro, 2023. "A Robust Method for Microforecasting and Estimation of Random Effects," Working Paper Series WP 2023-26, Federal Reserve Bank of Chicago.
    2. Bonhomme, Stéphane & Denis, Angela, 2024. "Estimating heterogeneous effects: Applications to labor economics," Labour Economics, Elsevier, vol. 91(C).
    3. Mookerjee, Sulagna & Slichter, David, 2023. "Test scores, schools, and the geography of economic opportunity," Journal of Urban Economics, Elsevier, vol. 137(C).
    4. John Carter Braxton & Kyle F. Herkenhoff & Jonathan Rothbaum & Lawrence Schmidt, 2021. "Changing Income Risk across the US Skill Distribution: Evidence from a Generalized Kalman Filter," Opportunity and Inclusive Growth Institute Working Papers 55, Federal Reserve Bank of Minneapolis.
    5. Tian Xie, 2025. "Automatic Inference for Value-Added Regressions," Papers 2503.19178, arXiv.org.
    6. Timothy B. Armstrong & Michal Kolesár & Mikkel Plagborg‐Møller, 2022. "Robust Empirical Bayes Confidence Intervals," Econometrica, Econometric Society, vol. 90(6), pages 2567-2602, November.
    7. Valentin Verdier, 2020. "Estimation and Inference for Linear Models with Two-Way Fixed Effects and Sparsely Matched Data," The Review of Economics and Statistics, MIT Press, vol. 102(1), pages 1-16, March.
    8. Botosaru, Irene, 2023. "Time-varying unobserved heterogeneity in earnings shocks," Journal of Econometrics, Elsevier, vol. 235(2), pages 1378-1393.
    9. Patrick Kline & Christopher Walters, 2021. "Reasonable Doubt: Experimental Detection of Job‐Level Employment Discrimination," Econometrica, Econometric Society, vol. 89(2), pages 765-792, March.
    10. Bar, M.; & Bakx, P.; & Wouterse, B.; & van Doorslaer, Eddy.;, 2022. "Estimating the health value added by nursing homes," Health, Econometrics and Data Group (HEDG) Working Papers 22/12, HEDG, c/o Department of Economics, University of York.
    11. Bär, Marlies & Bakx, Pieter & Wouterse, Bram & van Doorslaer, Eddy, 2022. "Estimating the health value added by nursing homes," Journal of Economic Behavior & Organization, Elsevier, vol. 203(C), pages 1-23.
    12. Michael Geruso & Timothy J. Layton & Jacob Wallace, 2023. "What Difference Does a Health Plan Make? Evidence from Random Plan Assignment in Medicaid," American Economic Journal: Applied Economics, American Economic Association, vol. 15(3), pages 341-379, July.
    13. Kline, Patrick & Walters, Christopher, 2019. "Audits as Evidence: Experiments, Ensembles, and Enforcement," Institute for Research on Labor and Employment, Working Paper Series qt3z72m9kn, Institute of Industrial Relations, UC Berkeley.
    14. Christine Mulhern & Isaac M. Opper, 2021. "Measuring and Summarizing the Multiple Dimensions of Teacher Effectiveness," CESifo Working Paper Series 9263, CESifo.
    15. Bruhn, Jesse & Imberman, Scott & Winters, Marcus, 2022. "Regulatory arbitrage in teacher hiring and retention: Evidence from Massachusetts Charter Schools," Journal of Public Economics, Elsevier, vol. 215(C).
    16. Theloudis, Alexandros, 2021. "Consumption inequality across heterogeneous families," European Economic Review, Elsevier, vol. 136(C).
    17. Naven, Matthew, 2019. "Human-Capital Formation During Childhood and Adolescence: Evidence from School Quality and Postsecondary Success in California," MPRA Paper 97716, University Library of Munich, Germany.
    18. Christian Posso & Jorge Tamayo & Arlen Guarin & Estefania Saravia, 2024. "Luck of the Draw: The Causal Effect of Physicians on Birth Outcomes," Borradores de Economia 1269, Banco de la Republica de Colombia.
    19. M. Hashem Pesaran & Liying Yang, 2024. "Heterogeneous autoregressions in short T panel data models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 39(7), pages 1359-1378, November.
    20. Lleras-Muney, Adriana & Price, Joseph & Yue, Dahai, 2022. "The association between educational attainment and longevity using individual-level data from the 1940 census," Journal of Health Economics, Elsevier, vol. 84(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2308.01596. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.