IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1512.06290.html

On the Non-Asymptotic Properties of Regularized M-estimators

Author

Listed:
  • Demian Pouzo

Abstract

We propose a general framework for regularization in M-estimation problems under time dependent (absolutely regular-mixing) data which encompasses many of the existing estimators. We derive non-asymptotic concentration bounds for the regularized M-estimator. Our results exhibit a variance-bias trade-off, with the variance term being governed by a novel measure of the complexity of the parameter set. We also show that the mixing structure affect the variance term by scaling the number of observations; depending on the decay rate of the mixing coefficients, this scaling can even affect the asymptotic behavior. Finally, we propose a data-driven method for choosing the tuning parameters of the regularized estimator which yield the same (up to constants) concentration bound as one that optimally balances the (squared) bias and variance terms. We illustrate the results with several canonical examples.

Suggested Citation

  • Demian Pouzo, 2015. "On the Non-Asymptotic Properties of Regularized M-estimators," Papers 1512.06290, arXiv.org, revised Oct 2016.
    • Handle: RePEc:arx:papers:1512.06290
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1512.06290
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Peter Bickel & Bo Li & Alexandre Tsybakov & Sara Geer & Bin Yu & Teófilo Valdés & Carlos Rivero & Jianqing Fan & Aad Vaart, 2006. "Regularization in statistics," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 15(2), pages 271-344, September.
    2. Koenker, Roger & Xiao, Zhijie, 2006. "Quantile Autoregression," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 980-990, September.
    3. Wei Biao Wu & Keming Yu & Gautam Mitra, 2008. "Kernel Conditional Quantile Estimation for Stationary Processes with Application to Conditional Value-at-Risk," Journal of Financial Econometrics, Oxford University Press, vol. 6(2), pages 253-270, Spring.
    4. Koenker, Roger W & Bassett, Gilbert, Jr, 1978. "Regression Quantiles," Econometrica, Econometric Society, vol. 46(1), pages 33-50, January.
    5. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107016064, January.
    6. Brendan K. Beare, 2010. "Copulas and Temporal Dependence," Econometrica, Econometric Society, vol. 78(1), pages 395-410, January.
    7. Chen, Xiaohong & Hansen, Lars Peter & Carrasco, Marine, 2010. "Nonlinearity and temporal dependence," Journal of Econometrics, Elsevier, vol. 155(2), pages 155-169, April.
    8. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107627314, January.
    9. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107016040, January.
    10. Xiaohong Chen & Timothy Christensen, 2013. "Optimal Sup-norm Rates, Adaptivity and Inference in Nonparametric Instrumental Variables Estimation," Cowles Foundation Discussion Papers 1923R, Cowles Foundation for Research in Economics, Yale University, revised Apr 2015.
    11. Xiaohong Chen & Wei Biao Wu & Yanping Yi, 2009. "Efficient Estimation of Copula-based Semiparametric Markov Models," Cowles Foundation Discussion Papers 1691, Cowles Foundation for Research in Economics, Yale University, revised Mar 2009.
    12. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107638105, January.
    13. Koenker, Roger, 2004. "Quantile regression for longitudinal data," Journal of Multivariate Analysis, Elsevier, vol. 91(1), pages 74-89, October.
    14. Jianqing Fan & Jinchi Lv & Lei Qi, 2011. "Sparse High-Dimensional Models in Economics," Annual Review of Economics, Annual Reviews, vol. 3(1), pages 291-317, September.
    15. Roger Koenker & Kevin F. Hallock, 2001. "Quantile Regression," Journal of Economic Perspectives, American Economic Association, vol. 15(4), pages 143-156, Fall.
    16. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107016057, January.
    17. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2011. "Inference for High-Dimensional Sparse Econometric Models," Papers 1201.0220, arXiv.org.
    18. Alexandre Belloni & Victor Chernozhukov, 2009. "L1-Penalized quantile regression in high-dimensional sparse models," CeMMAP working papers CWP10/09, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    19. repec:hum:wpaper:sfb649dp2011-048 is not listed on IDEAS
    20. Richard Blundell & Xiaohong Chen & Dennis Kristensen, 2007. "Semi-Nonparametric IV Estimation of Shape-Invariant Engel Curves," Econometrica, Econometric Society, vol. 75(6), pages 1613-1669, November.
    21. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107674165, January.
    22. Song, Song & Bickel, Peter J., 2011. "Large vector auto regressions," SFB 649 Discussion Papers 2011-048, Humboldt University Berlin, Collaborative Research Center 649: Economic Risk.
    23. Horowitz, Joel L., 2014. "Adaptive nonparametric instrumental variables estimation: Empirical choice of the regularization parameter," Journal of Econometrics, Elsevier, vol. 180(2), pages 158-173.
    24. Xiaohong Chen & Xiaotong Shen, 1998. "Sieve Extremum Estimates for Weakly Dependent Data," Econometrica, Econometric Society, vol. 66(2), pages 289-314, March.
    25. Beare, Brendan K., 2012. "Archimedean Copulas And Temporal Dependence," Econometric Theory, Cambridge University Press, vol. 28(6), pages 1165-1185, December.
    26. Xiaohong Chen & Norman R. Swanson (ed.), 2013. "Recent Advances and Future Directions in Causality, Prediction, and Specification Analysis," Springer Books, Springer, edition 127, number 978-1-4614-1653-1, March.
    27. Xiaohong Chen & Demian Pouzo, 2012. "Estimation of Nonparametric Conditional Moment Models With Possibly Nonsmooth Generalized Residuals," Econometrica, Econometric Society, vol. 80(1), pages 277-321, January.
    28. Lan Wang & Yichao Wu & Runze Li, 2012. "Quantile Regression for Analyzing Heterogeneity in Ultra-High Dimension," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(497), pages 214-222, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Karun Adusumilli & Taisuke Otsu, 2015. "Nonparametric instrumental regression with errors in variables," STICERD - Econometrics Paper Series /2015/585, Suntory and Toyota International Centres for Economics and Related Disciplines, LSE.
    2. Kaspar Wüthrich, 2020. "A Comparison of Two Quantile Models With Endogeneity," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(2), pages 443-456, April.
    3. Carlos A. Manzanares & Ying Jiang & Patrick Bajari, 2015. "Improving Policy Functions in High-Dimensional Dynamic Games," NBER Working Papers 21124, National Bureau of Economic Research, Inc.
    4. Liu, Xiaochun, 2019. "On tail fatness of macroeconomic dynamics," Journal of Macroeconomics, Elsevier, vol. 62(C).
    5. Elsby, Michael W.L. & Hobijn, Bart & Şahin, Ayşegül, 2015. "On the importance of the participation margin for labor market fluctuations," Journal of Monetary Economics, Elsevier, vol. 72(C), pages 64-82.
    6. Wen Xu, 2016. "Estimation of Dynamic Panel Data Models with Stochastic Volatility Using Particle Filters," Econometrics, MDPI, vol. 4(4), pages 1-13, October.
    7. Özgür Orhangazi & A. Erinç Yeldan, 2021. "The Re‐making of the Turkish Crisis," Development and Change, International Institute of Social Studies, vol. 52(3), pages 460-503, May.
    8. Alessandra Bonfiglioli & Rosario Crinò & Gino Gancia, 2018. "Firms and Economic Performance: A View from Trade," Working Papers 1047, Barcelona School of Economics.
    9. Guriev, Sergei & Treisman, Daniel, 2020. "A theory of informational autocracy," Journal of Public Economics, Elsevier, vol. 186(C).
    10. Makoto Shimoji, 2016. "Rationalizable Persuasion," Discussion Papers 16/08, Department of Economics, University of York.
    11. Ivan Balbuzanov, 2019. "Lies and consequences," International Journal of Game Theory, Springer;Game Theory Society, vol. 48(4), pages 1203-1240, December.
    12. Raphael Corbi & Fabio Miessi Sanches, 2022. "Church Competition, Religious Subsidies and the Rise of Evangelicalism: a Dynamic Structural Analysis," Working Papers, Department of Economics 2022_09, University of São Paulo (FEA-USP).
    13. Shirai, Daichi, 2016. "Persistence and Amplification of Financial Frictions," MPRA Paper 72187, University Library of Munich, Germany.
    14. Haiwen Zhou, 2018. "Impact of international trade on unemployment under oligopoly," The Journal of International Trade & Economic Development, Taylor & Francis Journals, vol. 27(4), pages 365-379, May.
    15. Lefgren, Lars J. & Stoddard, Olga B. & Stovall, John E., 2021. "Rationalizing self-defeating behaviors: Theory and evidence," Journal of Health Economics, Elsevier, vol. 76(C).
    16. Liang, Yan, 2022. "Impact of financial development on outsourcing and aggregate productivity," Journal of Development Economics, Elsevier, vol. 154(C).
    17. Jerzmanowski, Michal & Tamura, Robert, 2019. "Directed technological change & cross-country income differences: A quantitative analysis," Journal of Development Economics, Elsevier, vol. 141(C).
    18. Andrzej Rzonca & Piotr Cizkowicz, 2014. "The effects of unconventional monetary policy: what do central banks not include in their models? / Skutki niekonwencjonalnej polityki pieniê¿nej: czego banki centralne nie uwzglêdniaj¹w swoich modela," mBank - CASE Seminar Proceedings 131, CASE-Center for Social and Economic Research.
    19. L. A. Franzoni, 2016. "Optimal liability design under risk and ambiguity," Working Papers wp1048, Dipartimento Scienze Economiche, Universita' di Bologna.
    20. Kajal Lahiri & Chuanming Gao & Bernard Wixon, 2020. "Value of Sample Separation Information in a Sequential Probit Model," Arthaniti: Journal of Economic Theory and Practice, , vol. 19(2), pages 151-176, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1512.06290. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.