Analysis Techniques for Adaptive Online Learning

We survey tools for the analysis of Follow-The-Regularized-Leader (FTRL) and Dual Averaging algorithms when the regularizer (prox-function) is chosen adaptively based on the data. Adaptivity can be used to prove regret bounds that hold on every round (rather than a specific final round T), and also allows for data-dependent regret bounds as in AdaGrad-style algorithms. We present results from a large number of prior works in a unified manner, using a modular analysis that isolates the key arguments in easily re-usable lemmas. Our results include the first fully-general analysis of the FTRL-Proximal algorithm (a close relative of mirror descent), supporting arbitrary norms and non-smooth regularizers.