IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1710.04273.html
   My bibliography  Save this paper

Stochastic Gradient Descent in Continuous Time: A Central Limit Theorem

Author

Listed:
  • Justin Sirignano
  • Konstantinos Spiliopoulos

Abstract

Stochastic gradient descent in continuous time (SGDCT) provides a computationally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance. The SGDCT algorithm follows a (noisy) descent direction along a continuous stream of data. The parameter updates occur in continuous time and satisfy a stochastic differential equation. This paper analyzes the asymptotic convergence rate of the SGDCT algorithm by proving a central limit theorem (CLT) for strongly convex objective functions and, under slightly stronger conditions, for non-convex objective functions as well. An $L^{p}$ convergence rate is also proven for the algorithm in the strongly convex case. The mathematical analysis lies at the intersection of stochastic analysis and statistical learning.

Suggested Citation

  • Justin Sirignano & Konstantinos Spiliopoulos, 2017. "Stochastic Gradient Descent in Continuous Time: A Central Limit Theorem," Papers 1710.04273, arXiv.org, revised Jun 2019.
    • Handle: RePEc:arx:papers:1710.04273
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1710.04273
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. V. Basawa, Ishwar & Prakasa Rao, B. L. S., 1980. "Asymptotic inference for stochastic processes," Stochastic Processes and their Applications, Elsevier, vol. 10(3), pages 221-254, October.
    2. Elerain, Ola & Chib, Siddhartha & Shephard, Neil, 2001. "Likelihood Inference for Discretely Observed Nonlinear Diffusions," Econometrica, Econometric Society, vol. 69(4), pages 959-993, July.
    3. Veretennikov, A. Yu., 1997. "On polynomial mixing bounds for stochastic differential equations," Stochastic Processes and their Applications, Elsevier, vol. 70(1), pages 115-127, October.
    4. Yacine Ait-Sahalia, 2002. "Maximum Likelihood Estimation of Discretely Sampled Diffusions: A Closed-form Approximation Approach," Econometrica, Econometric Society, vol. 70(1), pages 223-262, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mark Trede & Bernd Wilfling, 2007. "Estimating exchange rate dynamics with diffusion processes: an application to Greek EMU data," Empirical Economics, Springer, vol. 33(1), pages 23-39, July.
    2. Kristensen, Dennis & Shin, Yongseok, 2012. "Estimation of dynamic models with nonparametric simulated maximum likelihood," Journal of Econometrics, Elsevier, vol. 167(1), pages 76-94.
    3. Stan Hurn & J.Jeisman & K.A. Lindsay, 2006. "Teaching an old dog new tricks: Improved estimation of the parameters of SDEs by numerical solution of the Fokker-Planck equation," Stan Hurn Discussion Papers 2006-01, School of Economics and Finance, Queensland University of Technology.
    4. Siddhartha Chib & Michael K Pitt & Neil Shephard, 2004. "Likelihood based inference for diffusion driven models," OFRC Working Papers Series 2004fe17, Oxford Financial Research Centre.
    5. Stan Hurn & J.Jeisman & K.A. Lindsay, 2006. "Seeing the Wood for the Trees: A Critical Evaluation of Methods to Estimate the Parameters of Stochastic Differential Equations. Working paper #2," NCER Working Paper Series 2, National Centre for Econometric Research.
    6. Czellar, Veronika & Karolyi, G. Andrew & Ronchetti, Elvezio, 2007. "Indirect robust estimation of the short-term interest rate process," Journal of Empirical Finance, Elsevier, vol. 14(4), pages 546-563, September.
    7. Hermann Singer, 2011. "Continuous-discrete state-space modeling of panel data with nonlinear filter algorithms," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 95(4), pages 375-413, December.
    8. Chernov, Mikhail & Ronald Gallant, A. & Ghysels, Eric & Tauchen, George, 2003. "Alternative models for stock price dynamics," Journal of Econometrics, Elsevier, vol. 116(1-2), pages 225-257.
    9. Peter C.B. Phillips & Jun Yu, 2005. "A Two-Stage Realized Volatility Approach to the Estimation for Diffusion Processes from Discrete Observations," Cowles Foundation Discussion Papers 1523, Cowles Foundation for Research in Economics, Yale University.
    10. A. S. Hurn & J. I. Jeisman & K. A. Lindsay, 0. "Seeing the Wood for the Trees: A Critical Evaluation of Methods to Estimate the Parameters of Stochastic Differential Equations," Journal of Financial Econometrics, Oxford University Press, vol. 5(3), pages 390-455.
    11. Kristensen, Dennis, 2010. "Pseudo-maximum likelihood estimation in two classes of semiparametric diffusion models," Journal of Econometrics, Elsevier, vol. 156(2), pages 239-259, June.
    12. Christopher S. Jones, 2003. "Nonlinear Mean Reversion in the Short-Term Interest Rate," The Review of Financial Studies, Society for Financial Studies, vol. 16(3), pages 793-843, July.
    13. Bandi, Federico M. & Phillips, Peter C.B., 2007. "A simple approach to the parametric estimation of potentially nonstationary diffusions," Journal of Econometrics, Elsevier, vol. 137(2), pages 354-395, April.
    14. Peter C.B.Phillips & Jun Yu, "undated". "Maximum Likelihood and Gaussian Estimation of Continuous Time Models in Finance," Working Papers CoFie-08-2009, Singapore Management University, Sim Kee Boon Institute for Financial Economics.
    15. Filipović, Damir & Mayerhofer, Eberhard & Schneider, Paul, 2013. "Density approximations for multivariate affine jump-diffusion processes," Journal of Econometrics, Elsevier, vol. 176(2), pages 93-111.
    16. Konstantinos Kalogeropoulos & Gareth O. Roberts & Petros Dellaportas, 2007. "Inference for stochastic volatility models using time change transformations," Papers 0711.1594, arXiv.org.
    17. Griffin, J.E. & Steel, M.F.J., 2006. "Inference with non-Gaussian Ornstein-Uhlenbeck processes for stochastic volatility," Journal of Econometrics, Elsevier, vol. 134(2), pages 605-644, October.
    18. Michael Sørensen, 2008. "Parametric inference for discretely sampled stochastic differential equations," CREATES Research Papers 2008-18, Department of Economics and Business Economics, Aarhus University.
    19. Kristensen, Dennis, 2004. "Estimation in two classes of semiparametric diffusion models," LSE Research Online Documents on Economics 24739, London School of Economics and Political Science, LSE Library.
    20. Radu Tunaru, 2015. "Model Risk in Financial Markets:From Financial Engineering to Risk Management," World Scientific Books, World Scientific Publishing Co. Pte. Ltd., number 9524, April.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1710.04273. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.