IDEAS home Printed from https://ideas.repec.org/p/crs/wpaper/2010-40.html
   My bibliography  Save this paper

Pac-Bayesian Bounds for Sparse Regression Estimation with Exponential Weights

Author

Listed:
  • Pierre Alquier

    (Crest)

  • Karim Lounici

    (Crest)

Abstract

We consider the sparse regression model where the number of parametersp is larger than the sample size n. The difficulty when consideringhigh-dimensional problems is to propose estimators achieving a good compromisebetween statistical and computational performances. The BIC estimatorfor instance performs well from the statistical point of view [11] but can onlybe computed for values of p of at most a few tens. The Lasso estimator issolution of a convex minimization problem, hence computable for large valueof p. However stringent conditions on the design are required to establish fastrates of convergence for this estimator. Dalalyan and Tsybakov [19] proposea method achieving a good compromise between the statistical and computationalaspects of the problem. Their estimator can be computed for reasonablylarge p and satisfies nice statistical properties under weak assumptions on thedesign. However, [19] proposes sparsity oracle inequalities in expectation forthe empirical excess risk only. In this paper, we propose an aggregation proceduresimilar to that of [19] but with improved statistical performances. Ourmain theoretical result is a sparsity oracle inequality in probability for the trueexcess risk for a version of exponential weight estimator. We also propose aMCMC method to compute our estimator for reasonably large values of p.

Suggested Citation

  • Pierre Alquier & Karim Lounici, 2010. "Pac-Bayesian Bounds for Sparse Regression Estimation with Exponential Weights," Working Papers 2010-40, Center for Research in Economics and Statistics.
  • Handle: RePEc:crs:wpaper:2010-40
    as

    Download full text from publisher

    File URL: http://crest.science/RePEc/wpstorage/2010-40.pdf
    File Function: Crest working paper version
    Download Restriction: no

    References listed on IDEAS

    as
    1. Laurent Gobillon & Thierry Magnac & Harris Selod, 2011. "The effect of location on finding a job in the Paris region," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(7), pages 1079-1112, November.
    2. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    3. Joel A. Elvery, 2009. "The Impact of Enterprise Zones on Resident Employment," Economic Development Quarterly, , vol. 23(1), pages 44-59, February.
    4. Raphael W. Bostic & Allen C. Prohofsky, 2006. "Enterprise Zones and Individual Welfare: A Case Study of California," Journal of Regional Science, Wiley Blackwell, vol. 46(2), pages 175-203.
    5. Peter S. Fisher, 1997. "Tax and spending incentives and enterprise zones," New England Economic Review, Federal Reserve Bank of Boston, issue Mar, pages 109-138.
    6. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 605-654.
    7. Jörn-Steffen Pischke & Alan Manning, 2006. "Comprehensive versus Selective Schooling in England in Wales: What Do We Know?," NBER Working Papers 12176, National Bureau of Economic Research, Inc.
    8. Boarnet, Marlon G. & Bogart, William T., 1996. "Enterprise Zones and Employment: Evidence from New Jersey," Journal of Urban Economics, Elsevier, vol. 40(2), pages 198-215, September.
    9. Gobillon, Laurent & Magnac, Thierry & Selod, Harris, 2012. "Do unemployed workers benefit from enterprise zones? The French experience," Journal of Public Economics, Elsevier, pages 881-892.
    10. repec:spr:portec:v:1:y:2002:i:2:d:10.1007_s10258-002-0010-3 is not listed on IDEAS
    11. Altonji, Joseph G & Segal, Lewis M, 1996. "Small-Sample Bias in GMM Estimation of Covariance Structures," Journal of Business & Economic Statistics, American Statistical Association, vol. 14(3), pages 353-366, July.
    12. O'Keefe, Suzanne, 2004. "Job creation in California's enterprise zones: a comparison using a propensity score matching model," Journal of Urban Economics, Elsevier, vol. 55(1), pages 131-150, January.
    13. Bondonio, Daniele & Engberg, John, 2000. "Enterprise zones and local employment: evidence from the states' programs," Regional Science and Urban Economics, Elsevier, vol. 30(5), pages 519-549, September.
    14. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    15. Heckman, J.J. & Hotz, V.J., 1988. "Choosing Among Alternative Nonexperimental Methods For Estimating The Impact Of Social Programs: The Case Of Manpower Training," University of Chicago - Economics Research Center 88-12, Chicago - Economics Research Center.
    16. Mauer, David C. & Ott, Steven H., 1999. "On the Optimal Structure of Government Subsidies for Enterprise Zones and Other Locational Development Programs," Journal of Urban Economics, Elsevier, vol. 45(3), pages 421-450, May.
    17. Richard Blundell & Monica Costa Dias & Costas Meghir & John Van Reenen, 2004. "Evaluating the Employment Impact of a Mandatory Job Search Program," Journal of the European Economic Association, MIT Press, vol. 2(4), pages 569-606, June.
    18. Laurent Gobillon & Harris Selod, 2007. "Les déterminants locaux du chômage en région parisienne," Economie & Prévision, La Documentation Française, pages 19-38.
    19. Cynthia L. Rogers & Jill L. Tao, 2004. "Quasi-Experimental Analysis of Targeted Economic Development Programs: Lessons from Florida," Economic Development Quarterly, , vol. 18(3), pages 269-285, August.
    20. Hanson, Andrew, 2009. "Local employment, poverty, and property value effects of geographically-targeted tax incentives: An instrumental variables approach," Regional Science and Urban Economics, Elsevier, vol. 39(6), pages 721-731, November.
    21. Papke, Leslie E., 1994. "Tax policy and urban development : Evidence from the Indiana enterprise zone program," Journal of Public Economics, Elsevier, pages 37-49.
    22. Bondonio, Daniele & Greenbaum, Robert T., 2007. "Do local tax incentives affect economic growth? What mean impacts miss in the analysis of enterprise zone policies," Regional Science and Urban Economics, Elsevier, vol. 37(1), pages 121-136, January.
    23. Matias Busso & Patrick Kline, 2008. "Do Local Economic Development Programs Work? Evidence from the Federal Empowerment Zone Program," Cowles Foundation Discussion Papers 1639, Cowles Foundation for Research in Economics, Yale University.
    24. Neumark, David & Kolko, Jed, 2010. "Do enterprise zones create jobs? Evidence from California's enterprise zone program," Journal of Urban Economics, Elsevier, vol. 68(1), pages 1-19, July.
    25. Sylvie Blasco & François Fontaine, 2010. "Étudier le non-recours à l'assurance chômage," Revue économique, Presses de Sciences-Po, vol. 61(5), pages 933-943.
    26. Ridder, Geert & Tunali, Insan, 1999. "Stratified partial likelihood estimation," Journal of Econometrics, Elsevier, vol. 92(2), pages 193-232, October.
    27. Papke, Leslie E., 1994. "Tax policy and urban development : Evidence from the Indiana enterprise zone program," Journal of Public Economics, Elsevier, pages 37-49.
    28. Roland Rathelot & Patrick Sillard, 2008. "Zones Franches Urbaines : quels effets sur l'emploi salarié et les créations d'établissements ?," Économie et Statistique, Programme National Persée, vol. 415(1), pages 81-96.
    29. Richard Blundell & Monica Costa Dias & Costas Meghir & John Van Reenen, 2001. "Evaluating the employment impact of a mandatory job search assistance program," IFS Working Papers W01/20, Institute for Fiscal Studies.
    Full references (including those not matched with items on IDEAS)

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:crs:wpaper:2010-40. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Sri Srikandan). General contact details of provider: http://edirc.repec.org/data/crestfr.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.