  Applying standard Markov chain Monte Carlo (MCMC) algorithms to large data
sets is computationally expensive. Both the calculation of the acceptance
probability and the creation of informed proposals usually require an iteration
through the whole data set. The recently proposed stochastic gradient Langevin
dynamics (SGLD) method circumvents this problem by generating proposals which
are only based on a subset of the data, by skipping the accept-reject step and
by using decreasing step-sizes sequence $(\delta_m)_{m \geq 0}$.
  %Under appropriate Lyapunov conditions, We provide in this article a rigorous
mathematical framework for analysing this algorithm. We prove that, under
verifiable assumptions, the algorithm is consistent, satisfies a central limit
theorem (CLT) and its asymptotic bias-variance decomposition can be
characterized by an explicit functional of the step-sizes sequence
$(\delta_m)_{m \geq 0}$. We leverage this analysis to give practical
recommendations for the notoriously difficult tuning of this algorithm: it is
asymptotically optimal to use a step-size sequence of the type $\delta_m \asymp
m^{-1/3}$, leading to an algorithm whose mean squared error (MSE) decreases at
rate $\mathcal{O}(m^{-1/3})$

Machine Learning