v1v2v3v4 (latest)

On the Variance of the Adaptive Learning Rate and Beyond

8 August 2019

Xiaodong Liu

ArXiv (abs)PDF HTML Github (2548★)

Papers citing "On the Variance of the Adaptive Learning Rate and Beyond"

50 / 864 papers shown

Title
Introducing VaDA: Novel Image Segmentation Model for Maritime Object Segmentation Using New Dataset Yongjin Kim Jinbum Park Sanha Kang Hanguen Kim 108 1 0 12 Jul 2024
LETS-C: Leveraging Text Embedding for Time Series Classification Rachneet Kaur Zhen Zeng T. Balch Manuela Veloso AI4TS 73 0 0 09 Jul 2024
Latent Space Imaging Matheus Souza Yidan Zheng Kaizhang Kang Yogeshwar Nath Mishra Qiang Fu Wolfgang Heidrich 138 0 0 09 Jul 2024
Stepping on the Edge: Curvature Aware Learning Rate Tuners Vincent Roulet Atish Agarwala Jean-Bastien Grill Grzegorz Swirszcz Mathieu Blondel Fabian Pedregosa 99 3 0 08 Jul 2024
Simplifying Deep Temporal Difference Learning Matteo Gallici Mattie Fellows Benjamin Ellis B. Pou Ivan Masmitja Jakob Foerster Mario Martin OffRL 165 26 0 05 Jul 2024
Eyes on the Game: Deciphering Implicit Human Signals to Infer Human Proficiency, Trust, and Intent Nikhil Hulle Stéphane Aroca-Ouellette Anthony J. Ries Jake Brawer Katharina von der Wense Alessandro Roncone 40 1 0 03 Jul 2024
Towards Deep Active Learning in Avian Bioacoustics Lukas Rauch Denis Huseljic Moritz Wirth J. Decke Bernhard Sick Christoph Scholz 62 4 0 26 Jun 2024
METRIK: Measurement-Efficient Randomized Controlled Trials using Transformers with Input Masking S. Lala Niraj K. Jha 54 0 0 24 Jun 2024
Inferring stochastic low-rank recurrent neural networks from neural data Matthijs Pals A Erdem Sağtekin Felix Pei Manuel Gloeckler Jakob H Macke 599 7 0 24 Jun 2024
Consistency Models Made Easy Zhengyang Geng Ashwini Pokle William Luo Justin Lin J. Zico Kolter 110 35 0 20 Jun 2024
A Unified View of Abstract Visual Reasoning Problems Mikołaj Małkiński Jacek Mańdziuk 70 0 0 16 Jun 2024
Optimizing Automatic Speech Assessment: W-RankSim Regularization and Hybrid Feature Fusion Strategies Chung-Wen Wu Berlin Chen 65 1 0 16 Jun 2024
When Will Gradient Regularization Be Harmful? Yang Zhao Hao Zhang Xiuyuan Hu AI4CE 65 1 0 14 Jun 2024
Why Warmup the Learning Rate? Underlying Mechanisms and Improvements Dayal Singh Kalra M. Barkeshli 125 11 0 13 Jun 2024
Optimal Recurrent Network Topologies for Dynamical Systems Reconstruction Christoph Jürgen Hemmer Manuel Brenner Florian Hess Daniel Durstewitz 104 4 0 07 Jun 2024
A Diffusion Model Framework for Unsupervised Neural Combinatorial Optimization Sebastian Sanokowski Sepp Hochreiter Sebastian Lehner 113 23 0 03 Jun 2024
Improving Generalization and Convergence by Enhancing Implicit Regularization Mingze Wang Haotian He Jinbo Wang Zilin Wang Guanhua Huang Feiyu Xiong Zhiyu Li E. Weinan Lei Wu 96 8 0 31 May 2024
AdaFisher: Adaptive Second Order Optimization via Fisher Information Damien Martins Gomes Yanlei Zhang Eugene Belilovsky Guy Wolf Mahdi S. Hosseini ODL 216 3 0 26 May 2024
Distilling Diffusion Models into Conditional GANs Minguk Kang Richard Zhang Connelly Barnes Sylvain Paris Suha Kwak Jaesik Park Eli Shechtman Jun-Yan Zhu Taesung Park 115 45 0 09 May 2024
Annot-Mix: Learning with Noisy Class Labels from Multiple Annotators via a Mixup Extension M. Herde Lukas Lührs Denis Huseljic Bernhard Sick 128 3 0 06 May 2024
Toward end-to-end interpretable convolutional neural networks for waveform signals Linh Vu Thu Tran Wern-Han Lim Raphael Phan 35 1 0 03 May 2024
Image segmentation of treated and untreated tumor spheroids by Fully Convolutional Networks Matthias Streller S. Michlíková Willy Ciecior Katharina Lönnecke L. Kunz-Schughart Steffen Lange Anja Voss-Böhme 123 1 0 02 May 2024
LidaRF: Delving into Lidar for Neural Radiance Field on Street Scenes Shanlin Sun Bingbing Zhuang Ziyu Jiang Buyu Liu Xiaohui Xie Manmohan Chandraker 144 3 0 01 May 2024
A Comprehensive Survey for Hyperspectral Image Classification: The Evolution from Conventional to Transformers Muhammad Ahmad Salvatore Distifano Adil Mehmood Khan Manuel Mazzara Chenyu Li Jing Yao Hao Li Jagannath Aryal Gemine Vivone Danfeng Hong 130 7 0 23 Apr 2024
FisheyeDetNet: 360° Surround view Fisheye Camera based Object Detection System for Autonomous Driving Ganesh Sistu S. Yogamani 85 0 0 20 Apr 2024
Revisiting Noise Resilience Strategies in Gesture Recognition: Short-Term Enhancement in Surface Electromyographic Signal Analysis Weiyu Guo Ziyue Qiao Ying Sun Hui Xiong 44 1 0 17 Apr 2024
GeoReF: Geometric Alignment Across Shape Variation for Category-level Object Pose Refinement Linfang Zheng Tze Ho Elden Tse Chen Wang Yinghan Sun Hua Chen A. Leonardis Wei Zhang 92 4 0 17 Apr 2024
Fast Fishing: Approximating BAIT for Efficient and Scalable Deep Active Image Classification Denis Huseljic Paul Hahn M. Herde Lukas Rauch Bernhard Sick 108 2 0 13 Apr 2024
Probing the 3D Awareness of Visual Foundation Models Mohamed El Banani Amit Raj Kevis-Kokitsi Maninis Abhishek Kar Yuanzhen Li Michael Rubinstein Deqing Sun Leonidas Guibas Justin Johnson Varun Jampani 101 86 0 12 Apr 2024
Greedy-DiM: Greedy Algorithms for Unreasonably Effective Face Morphs Zander W. Blasingame Chen Liu 102 6 0 09 Apr 2024
RaSim: A Range-aware High-fidelity RGB-D Data Simulation Pipeline for Real-world Applications Xingyu Liu Chenyangguang Zhang Gu Wang Ruida Zhang Xiangyang Ji 69 1 0 05 Apr 2024
VF-NeRF: Viewshed Fields for Rigid NeRF Registration Leo Segre S. Avidan 127 0 0 04 Apr 2024
Improving Line Search Methods for Large Scale Neural Network Training Philip Kenneweg Tristan Kenneweg Barbara Hammer ODL 53 3 0 27 Mar 2024
Faster Convergence for Transformer Fine-tuning with Line Search Methods Philip Kenneweg Leonardo Galli Tristan Kenneweg Barbara Hammer ODL 63 2 0 27 Mar 2024
Integrative Graph-Transformer Framework for Histopathology Whole Slide Image Representation and Classification Zhan Shi Jingwei Zhang Jun Kong Fusheng Wang MedIm 96 5 0 26 Mar 2024
Predicting Perceived Gloss: Do Weak Labels Suffice? Julia Guerrero-Viu J. Daniel Subias Ana Serrano Katherine R. Storrs Roland W. Fleming B. Masiá Diego F. F. Gutierrez 77 2 0 26 Mar 2024
On permutation-invariant neural networks Masanari Kimura Ryotaro Shimizu Yuki Hirakawa Ryosuke Goto Yuki Saito OOD AAML 94 12 0 26 Mar 2024
Bidirectional Consistency Models Liangchen Li Jiajun He DiffM 153 15 0 26 Mar 2024
PathoTune: Adapting Visual Foundation Model to Pathological Specialists Jiaxuan Lu Fang Yan Xiaofan Zhang Yue Gao Shaoting Zhang VLM LM&MA MedIm 80 7 0 25 Mar 2024
TexTile: A Differentiable Metric for Texture Tileability Carlos Rodriguez-Pardo Dan Casas Elena Garces Jorge López-Moreno DiffM 81 4 0 19 Mar 2024
A Hybrid Transformer-Sequencer approach for Age and Gender classification from in-wild facial images Aakash Singh V. K. Singh 50 5 0 19 Mar 2024
Towards Understanding the Relationship between In-context Learning and Compositional Generalization Sungjun Han Sebastian Padó CoGe 67 2 0 18 Mar 2024
Biophysics Informed Pathological Regularisation for Brain Tumour Segmentation Lipei Zhang Yanqi Cheng Lihao Liu Carola-Bibiane Schönlieb Angelica I Aviles-Rivero AI4CE 89 10 0 14 Mar 2024
Lightning NeRF: Efficient Hybrid Scene Representation for Autonomous Driving Junyi Cao Zhichao Li Naiyan Wang Chao Ma 89 8 0 09 Mar 2024
MamMIL: Multiple Instance Learning for Whole Slide Images with State Space Models Zijie Fang Yifeng Wang Zhi Wang Jian Zhang Xiangyang Ji Yongbing Zhang Mamba 81 7 0 08 Mar 2024
Shuffling Momentum Gradient Algorithm for Convex Optimization Trang H. Tran Quoc Tran-Dinh Lam M. Nguyen 55 2 0 05 Mar 2024
SGD with Partial Hessian for Deep Neural Networks Optimization Ying Sun Hongwei Yong Lei Zhang ODL 51 0 0 05 Mar 2024
Fast, Scale-Adaptive, and Uncertainty-Aware Downscaling of Earth System Model Fields with Generative Machine Learning P. Hess Michael Aich Baoxiang Pan Niklas Boers AI4Cl 107 2 0 05 Mar 2024
EEE-QA: Exploring Effective and Efficient Question-Answer Representations Zhanghao Hu Yijun Yang Junjie Xu Yifu Qiu Pinzhen Chen 65 0 0 04 Mar 2024
MPIPN: A Multi Physics-Informed PointNet for solving parametric acoustic-structure systems Chu Wang Jinhong Wu Yanzhi Wang Zhijian Zha Qi Zhou PINN 46 3 0 02 Mar 2024