Advanced Search
MyIDEAS: Login to save this paper or follow this series

Model selection and error estimation

Contents:

Author Info

  • Peter L. Bartlett
  • Stéphane Boucheron
  • Gábor Lugosi

    ()

Abstract

We study model selection strategies based on penalized empirical loss minimization. We point out a tight relationship between error estimation and data-based complexity penalization: any good error estimate may be converted into a data-based penalty function and the performance of the estimate is governed by the quality of the error estimate. We consider several penalty functions, involving error estimates on independent test data, empirical {\sc vc} dimension, empirical {\sc vc} entropy, and margin-based quantities. We also consider the maximal difference between the error on the first half of the training data and the second half, and the expected maximal discrepancy, a closely related capacity estimate that can be calculated by Monte Carlo integration. Maximal discrepancy penalty functions are appealing for pattern classification problems, since their computation is equivalent to empirical risk minimization over the training data with some labels flipped.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://www.econ.upf.edu/docs/papers/downloads/508.pdf
File Function: Whole Paper
Download Restriction: no

Bibliographic Info

Paper provided by Department of Economics and Business, Universitat Pompeu Fabra in its series Economics Working Papers with number 508.

as in new window
Length:
Date of creation: Oct 2000
Date of revision:
Handle: RePEc:upf:upfgen:508

Contact details of provider:
Web page: http://www.econ.upf.edu/

Related research

Keywords: Complexity regularization; model selection; error estimation; concentration of measure;

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

No references listed on IDEAS
You can help add them by filling out this form.

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Alessio Sancetta, 2010. "Bootstrap model selection for possibly dependent and heterogeneous data," Annals of the Institute of Statistical Mathematics, Springer, vol. 62(3), pages 515-546, June.
  2. Daudin, Jean-Jacques & Mary-Huard, Tristan, 2008. "Estimation of the conditional risk in classification: The swapping method," Computational Statistics & Data Analysis, Elsevier, vol. 52(6), pages 3220-3232, February.
  3. Olivier Bousquet, 2003. "New approaches to statistical learning theory," Annals of the Institute of Statistical Mathematics, Springer, vol. 55(2), pages 371-389, June.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:upf:upfgen:508. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ().

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.