Nonasymptotic estimates for Stochastic Gradient Langevin Dynamics under local conditions in nonconvex optimization

Within the context of empirical risk minimization, see Raginsky, Rakhlin, and Telgarsky (2017), we are concerned with a non-asymptotic analysis of sampling algorithms used in optimization. In particular, we obtain non-asymptotic error bounds for a popular class of algorithms called Stochastic Gradient Langevin Dynamics (SGLD). These results are derived in Wasserstein-1 and Wasserstein-2 distances in the absence of log-concavity of the target distribution. More precisely, the stochastic gradient $H(\theta, x)$ is assumed to be locally Lipschitz continuous in both variables, and furthermore, the dissipativity condition is relaxed by removing its uniform dependence in $x$. This relaxation allows us to present two key paradigms within the framework of scalable posterior sampling for Bayesian inference and of nonconvex optimization; namely, examples from minibatch logistic regression and from variational inference are given by providing theoretical guarantees for the sampling behaviour of the algorithm.