On stochastic proximal gradient algorithms

We study a stochastic version of the proximal gradient algorithm where the gradient is analytically intractable, and is approximated by Monte Carlo simulation. We derive a non-asymptotic bound on the convergence rate, and we derive conditions involving the Monte Carlo batch-size and the step-size of the algorithm under which convergence is guaranteed. In particular, we show that the stochastic proximal gradient algorithm achieves the same convergence rate as its deterministic counterpart. We extend the analysis to a stochastic version of the fast iterative shrinkage-thresholding of \cite{becketteboulle09}, whereby the gradient is approximated by Monte Carlo simulation. Contrary to the deterministic setting where the fast iterative shinkage-thresholding is known to converge faster than the proximal gradient algorithm, our results suggest that in the stochastic setting, the acceleration scheme yields no improvement. To illustrate, we apply the algorithms to the estimation of network structures from multiple realizations of a Gibbs measure.