Online Optimization in Dynamic Environments

Online optimization methods are often designed to have a total accumulated loss comparable to that achievable by some comparator, such as a batch algorithm with access to all the data and infinite computational resources. In many settings, this comparator is allowed to vary with time, and the associated "tracking regret" bounds scale with the overall variation of the comparator sequence. However, in practical scenarios ranging from motion imagery to network analysis, the environment is nonstationary and comparator sequences with small variation are quite weak, resulting in large losses. This paper describes a "dynamic mirror descent" method which addresses this challenge, yielding low regrets bounds for comparators with small deviations from a given dynamical model. This approach is then used within a broader class of online learning methods to simultaneously track the best dynamical model and form predictions based on that model. This concept is demonstrated empirically in the context of sequential compressed sensing of a dynamic scene, solar flare detection from satellite data with missing elements, and tracking a dynamic social network.