IDEAS home Printed from https://ideas.repec.org/a/inm/ormoor/v46y2021i3p912-945.html
   My bibliography  Save this article

Statistical Query Algorithms for Mean Vector Estimation and Stochastic Convex Optimization

Author

Listed:
  • Vitaly Feldman

    (Apple, Inc., Cupertino, California 95014)

  • Cristóbal Guzmán

    (Institute for Mathematical and Computational Engineering, Facultad de Matemáticas & Escuela de Ingeniería, Pontificia Universidad Católica de Chile, Region Metropolitana, 3580000 Santiago, Chile)

  • Santosh Vempala

    (ANID – Millennium Science Initiative Program – Millennium Nucleus Center for the Discovery of Structures in Complex Data, 7820244 Santiago, Chile; School of Computer Science, Georgia Institute of Technology, Atlanta, Georgia 30332)

Abstract

Stochastic convex optimization, by which the objective is the expectation of a random convex function, is an important and widely used method with numerous applications in machine learning, statistics, operations research, and other areas. We study the complexity of stochastic convex optimization given only statistical query (SQ) access to the objective function. We show that well-known and popular first-order iterative methods can be implemented using only statistical queries. For many cases of interest, we derive nearly matching upper and lower bounds on the estimation (sample) complexity, including linear optimization in the most general setting. We then present several consequences for machine learning, differential privacy, and proving concrete lower bounds on the power of convex optimization–based methods. The key ingredient of our work is SQ algorithms and lower bounds for estimating the mean vector of a distribution over vectors supported on a convex body in R d . This natural problem has not been previously studied, and we show that our solutions can be used to get substantially improved SQ versions of Perceptron and other online algorithms for learning halfspaces.

Suggested Citation

  • Vitaly Feldman & Cristóbal Guzmán & Santosh Vempala, 2021. "Statistical Query Algorithms for Mean Vector Estimation and Stochastic Convex Optimization," Mathematics of Operations Research, INFORMS, vol. 46(3), pages 912-945, August.
    • Handle: RePEc:inm:ormoor:v:46:y:2021:i:3:p:912-945
    DOI: 10.1287/moor.2020.1111
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/moor.2020.1111
    Download Restriction: no
    File URL: https://libkey.io/10.1287/moor.2020.1111?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormoor:v:46:y:2021:i:3:p:912-945. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.