Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs

27 February 2018

Dmitry Vetrov

Papers citing "Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs"

50 / 145 papers shown

Title
Low-Loss Space in Neural Networks is Continuous and Fully Connected Yongding Tian Zaid Al-Ars Maksim Kitsak P. Hofstee 3DPC 26 0 0 05 May 2025
The effect of the number of parameters and the number of local feature patches on loss landscapes in distributed quantum neural networks Yoshiaki Kawase 73 0 0 27 Apr 2025
Dynamic Fisher-weighted Model Merging via Bayesian Optimization Sanwoo Lee Jiahao Liu Qifan Wang J. Wang Xunliang Cai Yunfang Wu MoMe 133 0 0 26 Apr 2025
Adiabatic Fine-Tuning of Neural Quantum States Enables Detection of Phase Transitions in Weight Space Vinicius Hernandes Thomas Spriggs Saqar Khaleefah E. Greplova 47 0 0 21 Mar 2025
Self-Ensembling Gaussian Splatting for Few-Shot Novel View Synthesis Chen Zhao Xuan Wang Tong Zhang Saqib Javed Mathieu Salzmann 3DGS 185 0 0 13 Mar 2025
Analyzing the Role of Permutation Invariance in Linear Mode Connectivity Keyao Zhan Puheng Li Lei Wu MoMe 79 0 0 13 Mar 2025
SplatPose: Geometry-Aware 6-DoF Pose Estimation from Single RGB Image via 3D Gaussian Splatting Linqi Yang Xiongwei Zhao Qihao Sun Ke Wang Ao Chen Peng Kang 3DGS 78 2 0 07 Mar 2025
High-dimensional manifold of solutions in neural networks: insights from statistical physics Enrico M. Malatesta 46 4 0 20 Feb 2025
Unveiling Mode Connectivity in Graph Neural Networks Bingheng Li Z. Chen Haoyu Han Shenglai Zeng J. Liu Jiliang Tang 45 0 0 18 Feb 2025
SuperMerge: An Approach For Gradient-Based Model Merging Haoyu Yang Zheng Zhang Saket Sathe MoMe 125 0 0 17 Feb 2025
CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling Kaiyuan Zhang Siyuan Cheng Guangyu Shen Bruno Ribeiro Shengwei An Pin-Yu Chen X. Zhang Ninghui Li 92 1 0 28 Jan 2025
Meta Curvature-Aware Minimization for Domain Generalization Z. Chen Yiwen Ye Feilong Tang Yongsheng Pan Yong-quan Xia BDL 191 1 0 16 Dec 2024
TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensembling Yury Gorishniy Akim Kotelnikov Artem Babenko LMTD MoE 89 6 0 31 Oct 2024
Sampling from Bayesian Neural Network Posteriors with Symmetric Minibatch Splitting Langevin Dynamics Daniel Paulin P. Whalley Neil K. Chada B. Leimkuhler BDL 43 4 0 14 Oct 2024
Uncovering, Explaining, and Mitigating the Superficial Safety of Backdoor Defense Rui Min Zeyu Qin Nevin L. Zhang Li Shen Minhao Cheng AAML 31 4 0 13 Oct 2024
QT-DoG: Quantization-aware Training for Domain Generalization Saqib Javed Hieu Le Mathieu Salzmann OOD MQ 28 1 0 08 Oct 2024
Input Space Mode Connectivity in Deep Neural Networks Jakub Vrabel Ori Shem-Ur Yaron Oz David Krueger 48 1 0 09 Sep 2024
Information-Theoretic Progress Measures reveal Grokking is an Emergent Phase Transition Kenzo Clauw S. Stramaglia Daniele Marinazzo 50 3 0 16 Aug 2024
Network Fission Ensembles for Low-Cost Self-Ensembles Hojung Lee Jong-Seok Lee UQCV 54 0 0 05 Aug 2024
Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergence Shuya Feng Meisam Mohammady Hanbin Hong Shenao Yan Ashish Kundu Binghui Wang Yuan Hong FedML 36 3 0 20 Jul 2024
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models Jinliang Lu Ziliang Pang Min Xiao Yaochen Zhu Rui Xia Jiajun Zhang MoMe 38 18 0 08 Jul 2024
Harmony in Diversity: Merging Neural Networks with Canonical Correlation Analysis Stefan Horoi Albert Manuel Orozco Camacho Eugene Belilovsky Guy Wolf FedML MoMe 29 9 0 07 Jul 2024
Merging Improves Self-Critique Against Jailbreak Attacks Victor Gallego AAML MoMe 38 3 0 11 Jun 2024
Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models Chia-Yi Hsu Yu-Lin Tsai Chih-Hsun Lin Pin-Yu Chen Chia-Mu Yu Chun-ying Huang 44 32 0 27 May 2024
Linear Combination of Saved Checkpoints Makes Consistency and Diffusion Models Better En-hao Liu Junyi Zhu Zinan Lin Xuefei Ning Shuaiqi Wang ... Sergey Yekhanin Guohao Dai Huazhong Yang Yu-Xiang Wang Yu Wang MoMe 55 4 0 02 Apr 2024
Arcee's MergeKit: A Toolkit for Merging Large Language Models Charles Goddard Shamane Siriwardhana Malikeh Ehghaghi Luke Meyers Vladimir Karpukhin Brian Benedict Mark McQuade Jacob Solawetz MoMe KELM 84 77 0 20 Mar 2024
MedMerge: Merging Models for Effective Transfer Learning to Medical Imaging Tasks Ibrahim Almakky Santosh Sanjeev Anees Ur Rehman Hashmi Mohammad Areeb Qazi Mohammad Yaqub Mohammad Yaqub FedML MoMe 82 3 0 18 Mar 2024
Federated Learning over Connected Modes Dennis Grinwald Philipp Wiesner Shinichi Nakajima FedML 45 0 0 05 Mar 2024
Merging Text Transformer Models from Different Initializations Neha Verma Maha Elbayad MoMe 56 7 0 01 Mar 2024
Towards Meta-Pruning via Optimal Transport Alexander Theus Olin Geimer Friedrich Wicke Thomas Hofmann Sotiris Anagnostidis Sidak Pal Singh MoMe 16 3 0 12 Feb 2024
Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching: With Insights into Other Permutation Search Methods Akira Ito Masanori Yamada Atsutoshi Kumagai MoMe 61 5 0 06 Feb 2024
Stochastic Subnetwork Annealing: A Regularization Technique for Fine Tuning Pruned Subnetworks Tim Whitaker Darrell Whitley 27 0 0 16 Jan 2024
Train ñ Trade: Foundations of Parameter Markets Tzu-Heng Huang Harit Vishwakarma Frederic Sala AIFin 24 2 0 07 Dec 2023
Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also How Timm Hess Tinne Tuytelaars Gido M. van de Ven 36 7 0 08 Nov 2023
Equivariant Deep Weight Space Alignment Aviv Navon Aviv Shamsian Ethan Fetaya Gal Chechik Nadav Dym Haggai Maron 23 21 0 20 Oct 2023
Transformer Fusion with Optimal Transport Moritz Imfeld Jacopo Graldi Marco Giordano Thomas Hofmann Sotiris Anagnostidis Sidak Pal Singh ViT MoMe 22 16 0 09 Oct 2023
Geodesic Mode Connectivity Charlie Tan Theodore Long Sarah Zhao Rudolf Laine 11 2 0 24 Aug 2023
Layer-wise Linear Mode Connectivity Linara Adilova Maksym Andriushchenko Michael Kamp Asja Fischer Martin Jaggi FedML FAtt MoMe 28 15 0 13 Jul 2023
On The Impact of Machine Learning Randomness on Group Fairness Prakhar Ganesh Hong Chang Martin Strobel Reza Shokri FaML 23 30 0 09 Jul 2023
Domain Aligned Prefix Averaging for Domain Generalization in Abstractive Summarization Pranav Ajit Nair Sukomal Pal Pradeepika Verm MoMe 32 2 0 26 May 2023
How to escape sharp minima with random perturbations Kwangjun Ahn Ali Jadbabaie S. Sra ODL 29 6 0 25 May 2023
Mode Connectivity in Auction Design Christoph Hertrich Yixin Tao László A. Végh 16 1 0 18 May 2023
Inductive biases in deep learning models for weather prediction Jannik Thümmel Matthias Karlbauer S. Otte C. Zarfl Georg Martius ... Thomas Scholten Ulrich Friedrich V. Wulfmeyer B. Goswami Martin Volker Butz AI4CE 38 5 0 06 Apr 2023
On the Variance of Neural Network Training with respect to Test Sets and Distributions Keller Jordan OOD 16 10 0 04 Apr 2023
Achieving a Better Stability-Plasticity Trade-off via Auxiliary Networks in Continual Learning Sang-Ho Kim Lorenzo Noci Antonio Orvieto Thomas Hofmann CLL 22 34 0 16 Mar 2023
Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection Zhen Cheng Fei Zhu Xu-Yao Zhang Cheng-Lin Liu MoMe OODD 40 11 0 02 Mar 2023
DART: Diversify-Aggregate-Repeat Training Improves Generalization of Neural Networks Samyak Jain Sravanti Addepalli P. Sahu Priyam Dey R. Venkatesh Babu MoMe OOD 35 20 0 28 Feb 2023
Knowledge is a Region in Weight Space for Fine-tuned Language Models Almog Gueta Elad Venezian Colin Raffel Noam Slonim Yoav Katz Leshem Choshen 26 49 0 09 Feb 2023
Towards Inference Efficient Deep Ensemble Learning Ziyue Li Kan Ren Yifan Yang Xinyang Jiang Yuqing Yang Dongsheng Li BDL 21 12 0 29 Jan 2023
On the Lipschitz Constant of Deep Networks and Double Descent Matteo Gamba Hossein Azizpour Marten Bjorkman 19 7 0 28 Jan 2023