IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1901.01898.html
   My bibliography  Save this paper

Shrinkage for Categorical Regressors

Author

Listed:
  • Phillip Heiler
  • Jana Mareckova

Abstract

This paper introduces a flexible regularization approach that reduces point estimation risk of group means stemming from e.g. categorical regressors, (quasi-)experimental data or panel data models. The loss function is penalized by adding weighted squared l2-norm differences between group location parameters and informative first-stage estimates. Under quadratic loss, the penalized estimation problem has a simple interpretable closed-form solution that nests methods established in the literature on ridge regression, discretized support smoothing kernels and model averaging methods. We derive risk-optimal penalty parameters and propose a plug-in approach for estimation. The large sample properties are analyzed in an asymptotic local to zero framework by introducing a class of sequences for close and distant systems of locations that is sufficient for describing a large range of data generating processes. We provide the asymptotic distributions of the shrinkage estimators under different penalization schemes. The proposed plug-in estimator uniformly dominates the ordinary least squares in terms of asymptotic risk if the number of groups is larger than three. Monte Carlo simulations reveal robust improvements over standard methods in finite samples. Real data examples of estimating time trends in a panel and a difference-in-differences study illustrate potential applications.

Suggested Citation

  • Phillip Heiler & Jana Mareckova, 2019. "Shrinkage for Categorical Regressors," Papers 1901.01898, arXiv.org.
  • Handle: RePEc:arx:papers:1901.01898
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1901.01898
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Robert Tibshirani & Michael Saunders & Saharon Rosset & Ji Zhu & Keith Knight, 2005. "Sparsity and smoothness via the fused lasso," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(1), pages 91-108, February.
    2. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    3. Uri Gneezy & Aldo Rustichini, 2000. "Pay Enough or Don't Pay at All," The Quarterly Journal of Economics, Oxford University Press, vol. 115(3), pages 791-810.
    4. Peter Hall & Jeff Racine & Qi Li, 2004. "Cross-Validation and the Estimation of Conditional Probability Densities," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 1015-1026, December.
    5. Peter Hall & Qi Li & Jeffrey S. Racine, 2007. "Nonparametric Estimation of Regression Functions in the Presence of Irrelevant Regressors," The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 784-789, November.
    6. Uri Gneezy & Stephan Meier & Pedro Rey-Biel, 2011. "When and Why Incentives (Don't) Work to Modify Behavior," Journal of Economic Perspectives, American Economic Association, vol. 25(4), pages 191-210, Fall.
    7. Leeb, Hannes & Potscher, Benedikt M., 2008. "Sparse estimators and the oracle property, or the return of Hodges' estimator," Journal of Econometrics, Elsevier, vol. 142(1), pages 201-211, January.
    8. Gerhard Tutz & Margret-Ruth Oelker, 2017. "Modelling Clustered Heterogeneity: Fixed Effects, Random Effects and Mixtures," International Statistical Review, International Statistical Institute, vol. 85(2), pages 204-227, August.
    9. Liu, Chu-An, 2015. "Distribution theory of the least squares averaging estimator," Journal of Econometrics, Elsevier, vol. 186(1), pages 142-159.
    10. Hansen, Bruce E., 2016. "Efficient shrinkage in parametric models," Journal of Econometrics, Elsevier, vol. 190(1), pages 115-132.
    11. Ouyang, Desheng & Li, Qi & Racine, Jeffrey S., 2009. "Nonparametric Estimation Of Regression Functions With Discrete Regressors," Econometric Theory, Cambridge University Press, vol. 25(1), pages 1-42, February.
    12. Claeskens,Gerda & Hjort,Nils Lid, 2008. "Model Selection and Model Averaging," Cambridge Books, Cambridge University Press, number 9780521852258, January.
    13. Hansen, Bruce E. & Racine, Jeffrey S., 2012. "Jackknife model averaging," Journal of Econometrics, Elsevier, vol. 167(1), pages 38-46.
    14. Li, Qi & Racine, Jeff, 2003. "Nonparametric estimation of distributions with categorical and continuous data," Journal of Multivariate Analysis, Elsevier, vol. 86(2), pages 266-292, August.
    15. Liang, Hua & Zou, Guohua & Wan, Alan T. K. & Zhang, Xinyu, 2011. "Optimal Weight Choice for Frequentist Model Average Estimators," Journal of the American Statistical Association, American Statistical Association, vol. 106(495), pages 1053-1066.
    16. Xu Cheng & Zhipeng Liao & Ruoyao Shi, 2013. "Uniform Asymptotic Risk of Averaging GMM Estimator Robust to Misspecification, Second Version," PIER Working Paper Archive 15-017, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 25 Mar 2015.
    17. Hjort N.L. & Claeskens G., 2003. "Frequentist Model Average Estimators," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 879-899, January.
    18. Bruce E. Hansen, 2014. "Model averaging, asymptotic risk, and regressor groups," Quantitative Economics, Econometric Society, vol. 5(3), pages 495-530, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Heiler, Phillip & Mareckova, Jana, 2021. "Shrinkage for categorical regressors," Journal of Econometrics, Elsevier, vol. 223(1), pages 161-189.
    2. Kitagawa, Toru & Muris, Chris, 2016. "Model averaging in semiparametric estimation of treatment effects," Journal of Econometrics, Elsevier, vol. 193(1), pages 271-289.
    3. Ruoyao Shi, 2021. "An Averaging Estimator for Two Step M Estimation in Semiparametric Models," Working Papers 202105, University of California at Riverside, Department of Economics.
    4. Chu-An Liu & Biing-Shen Kuo & Wen-Jen Tsay, 2017. "Autoregressive Spectral Averaging Estimator," IEAS Working Paper : academic research 17-A013, Institute of Economics, Academia Sinica, Taipei, Taiwan.
    5. Ruoyao Shi & Zhipeng Liao, 2018. "An Averaging GMM Estimator Robust to Misspecification," Working Papers 201803, University of California at Riverside, Department of Economics.
    6. Giuseppe Luca & Jan R. Magnus & Franco Peracchi, 2023. "Weighted-Average Least Squares (WALS): Confidence and Prediction Intervals," Computational Economics, Springer;Society for Computational Economics, vol. 61(4), pages 1637-1664, April.
    7. Qingfeng Liu & Ryo Okui & Arihiro Yoshimura, 2016. "Generalized Least Squares Model Averaging," Econometric Reviews, Taylor & Francis Journals, vol. 35(8-10), pages 1692-1752, December.
    8. Liao, Jun & Zou, Guohua, 2020. "Corrected Mallows criterion for model averaging," Computational Statistics & Data Analysis, Elsevier, vol. 144(C).
    9. De Luca, Giuseppe & Magnus, Jan R. & Peracchi, Franco, 2018. "Weighted-average least squares estimation of generalized linear models," Journal of Econometrics, Elsevier, vol. 204(1), pages 1-17.
    10. Edvard Bakhitov, 2020. "Frequentist Shrinkage under Inequality Constraints," Papers 2001.10586, arXiv.org.
    11. Shangwei Zhao & Aman Ullah & Xinyu Zhang, 2018. "A Class of Model Averaging Estimators," Working Paper series 18-11, Rimini Centre for Economic Analysis.
    12. Sun, Yuying & Zhang, Xinyu & Wan, Alan T.K. & Wang, Shouyang, 2022. "Model averaging for interval-valued data," European Journal of Operational Research, Elsevier, vol. 301(2), pages 772-784.
    13. Aman Ullah & Xinyu Zhang, 2015. "Grouped Model Averaging for Finite Sample Size," Working Papers 201501, University of California at Riverside, Department of Economics.
    14. Giuseppe De Luca & Jan Magnus & Franco Peracchi, 2022. "Asymptotic properties of the weighted average least squares (WALS) estimator," Tinbergen Institute Discussion Papers 22-022/III, Tinbergen Institute.
    15. Martins, Luis F. & Gabriel, Vasco J., 2014. "Linear instrumental variables model averaging estimation," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 709-724.
    16. Ryan Greenaway-McGrevy & Kade Sorensen, 2021. "A spatial model averaging approach to measuring house prices," Journal of Spatial Econometrics, Springer, vol. 2(1), pages 1-32, December.
    17. Lu, Xun & Su, Liangjun, 2015. "Jackknife model averaging for quantile regressions," Journal of Econometrics, Elsevier, vol. 188(1), pages 40-58.
    18. Raffaella Giacomini & Toru Kitagawa & Alessio Volpicella, 2017. "Uncertain identification," CeMMAP working papers CWP18/17, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    19. Shou-Yung Yin & Chu-An Liu & Chang-Ching Lin, 2021. "Focused Information Criterion and Model Averaging for Large Panels With a Multifactor Error Structure," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(1), pages 54-68, January.
    20. Boot, Tom, 2023. "Joint inference based on Stein-type averaging estimators in the linear regression model," Journal of Econometrics, Elsevier, vol. 235(2), pages 1542-1563.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1901.01898. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.