v1v2v3v4 (latest)

Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs

Neural Information Processing Systems (NeurIPS), 2018

27 February 2018

Dmitry Vetrov

Papers citing "Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs"

50 / 547 papers shown

Title
End-to-End Training of CNN Ensembles for Person Re-IdentificationPattern Recognition (Pattern Recognit.), 2020 Ayse Serbetci Y. S. Akgul 134 24 0 03 Oct 2020
Adversarial Training with Stochastic Weight AverageInternational Conference on Information Photonics (ICIP), 2020 Joong-won Hwang Youngwan Lee Sungchan Oh Yuseok Bae OOD AAML 147 11 0 21 Sep 2020
S-SGD: Symmetrical Stochastic Gradient Descent with Weight Noise Injection for Reaching Flat Minima Wonyong Sung Iksoo Choi Jinhwan Park Seokhyun Choi Sungho Shin ODL 122 8 0 05 Sep 2020
Optimizing Mode Connectivity via Neuron AlignmentNeural Information Processing Systems (NeurIPS), 2020 N. Joseph Tatro Pin-Yu Chen Payel Das Igor Melnyk P. Sattigeri Rongjie Lai MoMe 621 92 0 05 Sep 2020
FedBE: Making Bayesian Model Ensemble Applicable to Federated LearningInternational Conference on Learning Representations (ICLR), 2020 Hong-You Chen Wei-Lun Chao FedML 265 309 0 04 Sep 2020
Robust, Accurate Stochastic Optimization for Variational InferenceNeural Information Processing Systems (NeurIPS), 2020 Akash Kumar Dhaka Alejandro Catalina Michael Riis Andersen Maans Magnusson Jonathan H. Huggins Aki Vehtari 141 35 0 01 Sep 2020
What is being transferred in transfer learning?Neural Information Processing Systems (NeurIPS), 2020 Behnam Neyshabur Hanie Sedghi Chiyuan Zhang 317 582 0 26 Aug 2020
Low-loss connection of weight vectors: distribution-based approaches Ivan Anokhin Dmitry Yarotsky 3DV 166 4 0 03 Aug 2020
Neural networks with late-phase weightsInternational Conference on Learning Representations (ICLR), 2020 J. Oswald Seijin Kobayashi Alexander Meulemans Christian Henning Benjamin Grewe João Sacramento 248 38 0 25 Jul 2020
Backdoor Learning: A SurveyIEEE Transactions on Neural Networks and Learning Systems (IEEE TNNLS), 2020 Yiming Li Yong Jiang Zhifeng Li Shutao Xia AAML 493 729 0 17 Jul 2020
Data-driven effective model shows a liquid-like deep learningPhysical Review Research (PRResearch), 2020 Wenxuan Zou Haiping Huang 199 2 0 16 Jul 2020
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers Robin M. Schmidt Frank Schneider Philipp Hennig ODL 687 185 0 03 Jul 2020
The Global Landscape of Neural Networks: An Overview Tian Ding Dawei Li Shiyu Liang Tian Ding R. Srikant 186 92 0 02 Jul 2020
Taming GANs with Lookahead-Minmax Tatjana Chavdarova Matteo Pagliardini Sebastian U. Stich François Fleuret Martin Jaggi GAN 184 29 0 25 Jun 2020
Uncertainty-Aware (UNA) Bases for Deep Bayesian Regression Using Multi-Headed Auxiliary Networks Sujay Thakur Cooper Lorsung Yaniv Yacoby Finale Doshi-Velez Weiwei Pan BDL UQCV 205 4 0 21 Jun 2020
Collective Learning by Ensembles of Altruistic Diversifying Neural Networks Benjamin Brazowski E. Schneidman FedML 109 5 0 20 Jun 2020
Directional Pruning of Deep Neural Networks Shih-Kang Chao Zhanyu Wang Yue Xing Guang Cheng ODL 207 35 0 16 Jun 2020
Depth Uncertainty in Neural Networks Javier Antorán J. Allingham José Miguel Hernández-Lobato UQCV OOD BDL 329 113 0 15 Jun 2020
On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them Chen Liu Mathieu Salzmann Tao Lin Ryota Tomioka Sabine Süsstrunk AAML 284 90 0 15 Jun 2020
Understanding Global Loss Landscape of One-hidden-layer ReLU Networks, Part 2: Experiments and Analysis Bo Liu 60 1 0 15 Jun 2020
Beyond Random Matrix Theory for Deep Networks Diego Granziol 333 17 0 13 Jun 2020
A benchmark study on reliable molecular supervised learning via Bayesian learning Doyeong Hwang Grace Lee Hanseok Jo Seyoul Yoon Seongok Ryu 132 9 0 12 Jun 2020
Is the Skip Connection Provable to Reform the Neural Network Loss Landscape?International Joint Conference on Artificial Intelligence (IJCAI), 2020 Lifu Wang Bo Shen Ningrui Zhao Zhiyuan Zhang 151 16 0 10 Jun 2020
Extrapolation for Large-batch Training in Deep LearningInternational Conference on Machine Learning (ICML), 2020 Tao Lin Lingjing Kong Sebastian U. Stich Martin Jaggi 214 40 0 10 Jun 2020
Isotropic SGD: a Practical Approach to Bayesian Posterior Sampling Giulio Franzese Rosa Candela Dimitrios Milios Maurizio Filippone Pietro Michiardi 83 1 0 09 Jun 2020
A Variational View on Bootstrap Ensembles as Bayesian Inference Dimitrios Milios Pietro Michiardi Maurizio Filippone 89 1 0 08 Jun 2020
Efficient AutoML Pipeline Search with Matrix and Tensor Factorization Chengrun Yang Jicong Fan Ziyang Wu Madeleine Udell 130 9 0 07 Jun 2020
Deep Ensembles on a Fixed Memory Budget: One Wide Network or Several Thinner Ones? Nadezhda Chirkova E. Lobacheva Dmitry Vetrov OOD MoE 93 9 0 14 May 2020
The critical locus of overparameterized neural networks Y. Cooper UQCV 183 11 0 08 May 2020
Bridging Mode Connectivity in Loss Landscapes and Adversarial RobustnessInternational Conference on Learning Representations (ICLR), 2020 Pu Zhao Pin-Yu Chen Payel Das Karthikeyan N. Ramamurthy Xue Lin AAML 424 204 0 30 Apr 2020
Masking as an Efficient Alternative to Finetuning for Pretrained Language ModelsConference on Empirical Methods in Natural Language Processing (EMNLP), 2020 Mengjie Zhao Tao Lin Fei Mi Martin Jaggi Hinrich Schütze 221 124 0 26 Apr 2020
Climate Adaptation: Reliably Predicting from Imbalanced Satellite Data Ruchit Rawal Prabhu Pradhan 72 2 0 26 Apr 2020
Towards Deep Learning Models Resistant to Large Perturbations Amirreza Shaeiri Rozhin Nobahari M. Rohban OOD AAML 151 14 0 30 Mar 2020
SuperNet -- An efficient method of neural networks ensembling Ludwik Bukowski W. Dzwinel 66 2 0 29 Mar 2020
Piecewise linear activations substantially shape the loss surfaces of neural networksInternational Conference on Learning Representations (ICLR), 2020 Fengxiang He Bohan Wang Dacheng Tao ODL 146 33 0 27 Mar 2020
Auto-Ensemble: An Adaptive Learning Rate Scheduling based Deep Learning Model EnsemblingIEEE Access (IEEE Access), 2020 Jun Yang Fei Wang 200 39 0 25 Mar 2020
Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network LossesNeural Computation (Neural Comput.), 2020 Charles G. Frye James B. Simon Neha S. Wadia A. Ligeralde M. DeWeese K. Bouchard ODL 133 3 0 23 Mar 2020
Safe Crossover of Neural Networks Through Neuron AlignmentAnnual Conference on Genetic and Evolutionary Computation (GECCO), 2020 Thomas Uriot Dario Izzo 197 15 0 23 Mar 2020
Interference and Generalization in Temporal Difference LearningInternational Conference on Machine Learning (ICML), 2020 Emmanuel Bengio Joelle Pineau Doina Precup 165 65 0 13 Mar 2020
Wide-minima Density Hypothesis and the Explore-Exploit Learning Rate ScheduleJournal of machine learning research (JMLR), 2020 Nikhil Iyer V. Thejas Nipun Kwatra Ramachandran Ramjee Muthian Sivathanu 275 33 0 09 Mar 2020
Some Geometrical and Topological Properties of DNNs' Decision Boundaries Bo Liu Mengya Shen AAML 188 3 0 07 Mar 2020
Explaining Knowledge Distillation by Quantifying the KnowledgeComputer Vision and Pattern Recognition (CVPR), 2020 Feng He Zhefan Rao Yilan Chen Quanshi Zhang 182 136 0 07 Mar 2020
Dropout Strikes Back: Improved Uncertainty Estimation via Diversity SamplingInternational Joint Conference on the Analysis of Images, Social Networks and Texts (AISNT), 2020 Kirill Fedyanin Evgenii Tsymbalov Maxim Panov UQCV 156 7 0 06 Mar 2020
Iterative Averaging in the Quest for Best Test ErrorJournal of machine learning research (JMLR), 2020 Diego Granziol Xingchen Wan Samuel Albanie Stephen J. Roberts 229 3 0 02 Mar 2020
Bayesian Deep Learning and a Probabilistic Perspective of GeneralizationNeural Information Processing Systems (NeurIPS), 2020 A. Wilson Pavel Izmailov UQCV BDL OOD 603 730 0 20 Feb 2020
BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong LearningInternational Conference on Learning Representations (ICLR), 2020 Yeming Wen Dustin Tran Jimmy Ba OOD FedML UQCV 407 534 0 17 Feb 2020
Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep LearningInternational Conference on Learning Representations (ICLR), 2020 Arsenii Ashukha Alexander Lyzhov Dmitry Molchanov Dmitry Vetrov UQCV FedML 459 343 0 15 Feb 2020
LaProp: Separating Momentum and Adaptivity in Adam Liu Ziyin Zhikang T.Wang Masahito Ueda ODL 184 20 0 12 Feb 2020
Understanding Global Loss Landscape of One-hidden-layer ReLU Networks, Part 1: Theory Bo Liu FAtt MLT 189 1 0 12 Feb 2020
A study of local optima for learning feature interactions using neural networksIEEE International Joint Conference on Neural Network (IJCNN), 2020 Yangzi Guo Adrian Barbu 205 1 0 11 Feb 2020