Lookahead Optimizer: k steps forward, 1 step back

19 July 2019

Jimmy Ba

Papers citing "Lookahead Optimizer: k steps forward, 1 step back"

47 / 347 papers shown

Title
Entropic gradient descent algorithms and wide flat minima Fabrizio Pittorino C. Lucibello Christoph Feinauer Gabriele Perugini Carlo Baldassi Elizaveta Demyanenko R. Zecchina ODL MLT 25 33 0 14 Jun 2020
VirTex: Learning Visual Representations from Textual Annotations Karan Desai Justin Johnson SSL VLM 24 432 0 11 Jun 2020
sEMG Gesture Recognition with a Simple Model of Attention David Josephs Carson Drake Andrew Heroy John Santerre 12 47 0 05 Jun 2020
CoolMomentum: A Method for Stochastic Optimization by Langevin Dynamics with Simulated Annealing O. Borysenko M. Byshkin ODL 9 14 0 29 May 2020
Adaptive Transformers for Learning Multimodal Representations Prajjwal Bhargava 14 4 0 15 May 2020
Neural Networks Versus Conventional Filters for Inertial-Sensor-based Attitude Estimation Daniel Weber C. Gühmann Thomas Seel 6 34 0 14 May 2020
2kenize: Tying Subword Sequences for Chinese Script Conversion Pranav A Isabelle Augenstein 19 1 0 07 May 2020
BlackBox: Generalizable Reconstruction of Extremal Values from Incomplete Spatio-Temporal Data T. Ivek Domagoj Vlah 40 4 0 30 Apr 2020
How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking Nicola De Cao M. Schlichtkrull Wilker Aziz Ivan Titov 17 89 0 30 Apr 2020
Multi-view Self-Constructing Graph Convolutional Networks with Adaptive Class Weighting Loss for Semantic Segmentation Qinghui Liu Michael C. Kampffmeyer Robert Jenssen Arnt-Børre Salberg SSL 27 35 0 21 Apr 2020
An Adaptive Intelligence Algorithm for Undersampled Knee MRI Reconstruction Nicola Pezzotti Sahar Yousefi M. Elmahdy J. V. Gemert C. Schulke ... Sergey Kastryulin B. Lelieveldt M. Osch E. Weerdt Marius Staring 14 97 0 15 Apr 2020
An Evaluation of DNN Architectures for Page Segmentation of Historical Newspapers Bernhard Liebl M. Burghardt SSeg 14 11 0 15 Apr 2020
Self6D: Self-Supervised Monocular 6D Object Pose Estimation Gu Wang Fabian Manhardt Jianzhun Shao Xiangyang Ji Nassir Navab Federico Tombari SSL MDE 24 133 0 14 Apr 2020
Detached Error Feedback for Distributed SGD with Random Sparsification An Xu Heng-Chiao Huang 36 9 0 11 Apr 2020
Applying Cyclical Learning Rate to Neural Machine Translation Choon Meng Lee Jianfeng Liu Wei Peng ODL 11 2 0 06 Apr 2020
Multi-Plateau Ensemble for Endoscopic Artefact Segmentation and Detection Suyog Jadhav Udbhav Bamba Arnav Chavan Rishabh Tiwari A. Raj 14 3 0 23 Mar 2020
Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation Huiyu Wang Yukun Zhu Bradley Green Hartwig Adam Alan Yuille Liang-Chieh Chen 3DPC 22 656 0 17 Mar 2020
Encoder-Decoder Based Convolutional Neural Networks with Multi-Scale-Aware Modules for Crowd Counting Pongpisit Thanasutives Ken-ichi Fukui M. Numao B. Kijsirikul 24 63 0 12 Mar 2020
Flexible numerical optimization with ensmallen Ryan R. Curtin Marcus Edel Rahul Prabhu S. Basak Zhihao Lou Conrad Sanderson 14 1 0 09 Mar 2020
Train-by-Reconnect: Decoupling Locations of Weights from their Values Yushi Qiu R. Suda 13 0 0 05 Mar 2020
Colored Noise Injection for Training Adversarially Robust Neural Networks Evgenii Zheltonozhskii Chaim Baskin Yaniv Nemcovsky Brian Chmiel A. Mendelson A. Bronstein AAML 17 5 0 04 Mar 2020
3D dynamic hand gestures recognition using the Leap Motion sensor and convolutional neural networks Katia Lupinetti A. Ranieri F. Giannini M. Monti SLR 8 27 0 03 Mar 2020
A New Dataset, Poisson GAN and AquaNet for Underwater Object Grabbing Chongwei Liu Zhihui Wang Shijie Wang Tao Tang Yulong Tao Caifei Yang Haojie Li Xing Liu Xin-Yue Fan 16 48 0 03 Mar 2020
Iterative Averaging in the Quest for Best Test Error Diego Granziol Xingchen Wan Samuel Albanie Stephen J. Roberts 8 3 0 02 Mar 2020
Adaptive Federated Optimization Sashank J. Reddi Zachary B. Charles Manzil Zaheer Zachary Garrett Keith Rush Jakub Konecný Sanjiv Kumar H. B. McMahan FedML 12 1,389 0 29 Feb 2020
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence Nicolas Loizou Sharan Vaswani I. Laradji Simon Lacoste-Julien 27 181 0 24 Feb 2020
From English To Foreign Languages: Transferring Pre-trained Language Models Ke M. Tran 22 47 0 18 Feb 2020
Meta-learning Extractors for Music Source Separation David Samuel Aditya Ganeshan Jason Naradowsky 21 60 0 17 Feb 2020
LaProp: Separating Momentum and Adaptivity in Adam Liu Ziyin Zhikang T.Wang Masahito Ueda ODL 6 18 0 12 Feb 2020
Evolutionary Neural Architecture Search for Retinal Vessel Segmentation Zhun Fan Jiahong Wei Guijie Zhu Jiajie Mo Wenji Li 24 8 0 18 Jan 2020
Gradient descent with momentum --- to accelerate or to super-accelerate? Goran Nakerst John Brennan M. Haque ODL 10 15 0 17 Jan 2020
Fine-grained Image Classification and Retrieval by Combining Visual and Locally Pooled Textual Features Andrés Mafla S. Dey Ali Furkan Biten Lluís Gómez Dimosthenis Karatzas 8 26 0 14 Jan 2020
CProp: Adaptive Learning Rate Scaling from Past Gradient Conformity Konpat Preechakul B. Kijsirikul ODL 25 3 0 24 Dec 2019
Pyramid Convolutional RNN for MRI Image Reconstruction Eric Z. Chen Puyang Wang Xiao Chen Terrence Chen Shanhui Sun 13 41 0 02 Dec 2019
Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models Giannis Daras Augustus Odena Han Zhang A. Dimakis 29 54 0 27 Nov 2019
Merging Deterministic Policy Gradient Estimations with Varied Bias-Variance Tradeoff for Effective Deep Reinforcement Learning Gang Chen 20 4 0 24 Nov 2019
Weakly Supervised Multi-Task Learning for Cell Detection and Segmentation Alireza Chamanzar Yao Nie 16 53 0 27 Oct 2019
Filterbank design for end-to-end speech separation Manuel Pariente Samuele Cornell Antoine Deleforge Emmanuel Vincent 18 69 0 23 Oct 2019
SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum Jianyu Wang Vinayak Tantia Nicolas Ballas Michael G. Rabbat 4 200 0 01 Oct 2019
MGBPv2: Scaling Up Multi-Grid Back-Projection Networks Pablo Navarrete Michelini Wenbin Chen Hanwen Liu Dan Zhu 13 7 0 27 Sep 2019
Improving Federated Learning Personalization via Model Agnostic Meta Learning Yihan Jiang Jakub Konecný Keith Rush Sreeram Kannan FedML 6 586 0 27 Sep 2019
Deep Prediction of Investor Interest: a Supervised Clustering Approach Baptiste Barreau Laurent Carlier D. Challet 10 1 0 11 Sep 2019
Kinematic Single Vehicle Trajectory Prediction Baselines and Applications with the NGSIM Dataset Jean Pierre Mercat N. Zoghby G. Sandou D. Beauvois Guillermo Pita Gil AI4TS 20 17 0 29 Aug 2019
Mish: A Self Regularized Non-Monotonic Activation Function Diganta Misra 14 677 0 23 Aug 2019
Use What You Have: Video Retrieval Using Representations From Collaborative Experts Yang Liu Samuel Albanie Arsha Nagrani Andrew Zisserman 34 387 0 31 Jul 2019
An Adaptive Remote Stochastic Gradient Method for Training Neural Networks Yushu Chen Hao Jing Wenlai Zhao Zhiqiang Liu H. Fu Lián Qiao Wei Xue Guangwen Yang ODL 19 2 0 04 May 2019
Neutron: An Implementation of the Transformer Translation Model and its Variants Hongfei Xu Qiuhui Liu 27 19 0 18 Mar 2019