IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1710.04273.html

Stochastic Gradient Descent in Continuous Time: A Central Limit Theorem

Author

Listed:
  • Justin Sirignano
  • Konstantinos Spiliopoulos

Abstract

Stochastic gradient descent in continuous time (SGDCT) provides a computationally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance. The SGDCT algorithm follows a (noisy) descent direction along a continuous stream of data. The parameter updates occur in continuous time and satisfy a stochastic differential equation. This paper analyzes the asymptotic convergence rate of the SGDCT algorithm by proving a central limit theorem (CLT) for strongly convex objective functions and, under slightly stronger conditions, for non-convex objective functions as well. An $L^{p}$ convergence rate is also proven for the algorithm in the strongly convex case. The mathematical analysis lies at the intersection of stochastic analysis and statistical learning.

Suggested Citation

  • Justin Sirignano & Konstantinos Spiliopoulos, 2017. "Stochastic Gradient Descent in Continuous Time: A Central Limit Theorem," Papers 1710.04273, arXiv.org, revised Jun 2019.
    • Handle: RePEc:arx:papers:1710.04273
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1710.04273
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. V. Basawa, Ishwar & Prakasa Rao, B. L. S., 1980. "Asymptotic inference for stochastic processes," Stochastic Processes and their Applications, Elsevier, vol. 10(3), pages 221-254, October.
    2. Veretennikov, A. Yu., 1997. "On polynomial mixing bounds for stochastic differential equations," Stochastic Processes and their Applications, Elsevier, vol. 70(1), pages 115-127, October.
    3. Elerain, Ola & Chib, Siddhartha & Shephard, Neil, 2001. "Likelihood Inference for Discretely Observed Nonlinear Diffusions," Econometrica, Econometric Society, vol. 69(4), pages 959-993, July.
    4. Yacine Ait-Sahalia, 2002. "Maximum Likelihood Estimation of Discretely Sampled Diffusions: A Closed-form Approximation Approach," Econometrica, Econometric Society, vol. 70(1), pages 223-262, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mark Trede & Bernd Wilfling, 2007. "Estimating exchange rate dynamics with diffusion processes: an application to Greek EMU data," Empirical Economics, Springer, vol. 33(1), pages 23-39, July.
    2. Kristensen, Dennis & Shin, Yongseok, 2012. "Estimation of dynamic models with nonparametric simulated maximum likelihood," Journal of Econometrics, Elsevier, vol. 167(1), pages 76-94.
    3. Radu Tunaru, 2015. "Model Risk in Financial Markets:From Financial Engineering to Risk Management," World Scientific Books, World Scientific Publishing Co. Pte. Ltd., number 9524, March.
    4. Stan Hurn & J.Jeisman & K.A. Lindsay, 2006. "Teaching an old dog new tricks: Improved estimation of the parameters of SDEs by numerical solution of the Fokker-Planck equation," Stan Hurn Discussion Papers 2006-01, School of Economics and Finance, Queensland University of Technology.
    5. Siddhartha Chib & Michael K Pitt & Neil Shephard, 2004. "Likelihood based inference for diffusion driven models," OFRC Working Papers Series 2004fe17, Oxford Financial Research Centre.
    6. Stan Hurn & J.Jeisman & K.A. Lindsay, 2006. "Seeing the Wood for the Trees: A Critical Evaluation of Methods to Estimate the Parameters of Stochastic Differential Equations. Working paper #2," NCER Working Paper Series 2, National Centre for Econometric Research.
    7. Hermann Singer, 2011. "Continuous-discrete state-space modeling of panel data with nonlinear filter algorithms," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 95(4), pages 375-413, December.
    8. A. Hurn & J. Jeisman & K. Lindsay, 2007. "Teaching an Old Dog New Tricks: Improved Estimation of the Parameters of Stochastic Differential Equations by Numerical Solution of the Fokker-Planck Equation," NCER Working Paper Series 9, National Centre for Econometric Research.
    9. Choi, Seungmoon, 2013. "Closed-form likelihood expansions for multivariate time-inhomogeneous diffusions," Journal of Econometrics, Elsevier, vol. 174(2), pages 45-65.
    10. J. Jimenez & R. Biscay & T. Ozaki, 2005. "Inference Methods for Discretely Observed Continuous-Time Stochastic Volatility Models: A Commented Overview," Asia-Pacific Financial Markets, Springer;Japanese Association of Financial Economics and Engineering, vol. 12(2), pages 109-141, June.
    11. Zongwu Cai & Yongmiao Hong, 2013. "Some Recent Developments in Nonparametric Finance," Working Papers 2013-10-14, Wang Yanan Institute for Studies in Economics (WISE), Xiamen University.
    12. Chernov, Mikhail & Ronald Gallant, A. & Ghysels, Eric & Tauchen, George, 2003. "Alternative models for stock price dynamics," Journal of Econometrics, Elsevier, vol. 116(1-2), pages 225-257.
    13. esposito, francesco paolo & cummins, mark, 2015. "Filtering and likelihood estimation of latent factor jump-diffusions with an application to stochastic volatility models," MPRA Paper 64987, University Library of Munich, Germany.
    14. Helle Sørensen, 2002. "Parametric Inference for Diffusion Processes Observed at Discrete Points in Time: a Survey," Discussion Papers 02-08, University of Copenhagen. Department of Economics.
    15. Peter C.B. Phillips & Jun Yu, 2005. "A Two-Stage Realized Volatility Approach to the Estimation for Diffusion Processes from Discrete Observations," Cowles Foundation Discussion Papers 1523, Cowles Foundation for Research in Economics, Yale University.
    16. Griffin, J.E. & Steel, M.F.J., 2006. "Inference with non-Gaussian Ornstein-Uhlenbeck processes for stochastic volatility," Journal of Econometrics, Elsevier, vol. 134(2), pages 605-644, October.
    17. Kristensen, Dennis, 2010. "Pseudo-maximum likelihood estimation in two classes of semiparametric diffusion models," Journal of Econometrics, Elsevier, vol. 156(2), pages 239-259, June.
    18. Christopher S. Jones, 2003. "Nonlinear Mean Reversion in the Short-Term Interest Rate," The Review of Financial Studies, Society for Financial Studies, vol. 16(3), pages 793-843, July.
    19. Bandi, Federico M. & Phillips, Peter C.B., 2007. "A simple approach to the parametric estimation of potentially nonstationary diffusions," Journal of Econometrics, Elsevier, vol. 137(2), pages 354-395, April.
    20. Bakshi, Gurdip & Ju, Nengjiu & Ou-Yang, Hui, 2006. "Estimation of continuous-time models with an application to equity volatility dynamics," Journal of Financial Economics, Elsevier, vol. 82(1), pages 227-249, October.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1710.04273. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.