v1v2v3v4 (latest)

Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs

Neural Information Processing Systems (NeurIPS), 2018

27 February 2018

Dmitry Vetrov

Papers citing "Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs"

50 / 546 papers shown

Title
CopRA: A Progressive LoRA Training Strategy Zhan Zhuang Xiequn Wang Yulong Zhang Wei Li Yu Zhang Ying Wei 232 1 0 30 Oct 2024
Model merging with SVD to tie the KnotsInternational Conference on Learning Representations (ICLR), 2024 George Stoica Pratik Ramesh B. Ecsedi Leshem Choshen Judy Hoffman MoMe 234 40 0 25 Oct 2024
Scalable Data Ablation Approximations for Language Models through Modular Training and MergingConference on Empirical Methods in Natural Language Processing (EMNLP), 2024 Clara Na Ian H. Magnusson A. Jha Tom Sherborne Emma Strubell Jesse Dodge Pradeep Dasigi MoMe 157 7 0 21 Oct 2024
Unconstrained Model Merging for Enhanced LLM Reasoning Yiming Zhang Baoyi He Shengyu Zhang Yuhao Fu Qi Zhou ... Guanghan Ning Linyi Li Chunlin Ji Leilei Gan Hongxia Yang MoMe 171 5 0 17 Oct 2024
LoRA Soups: Merging LoRAs for Practical Skill Composition TasksInternational Conference on Computational Linguistics (COLING), 2024 Akshara Prabhakar Yuanzhi Li Karthik Narasimhan Sham Kakade Eran Malach Samy Jelassi MoMe 289 28 0 16 Oct 2024
Exploring Model Kinship for Merging Large Language Models Yedi Hu Yunzhi Yao Ningyu Zhang Shumin Deng Ningyu Zhang MoMe 409 1 0 16 Oct 2024
Deep Model Merging: The Sister of Neural Network Interpretability -- A Survey A. Khan Todd Nief Nathaniel Hudson Mansi Sakarvadia Daniel Grzenda Aswathy Ajith Jordan Pettyjohn Kyle Chard Ian Foster MoMe 135 1 0 16 Oct 2024
Sampling from Bayesian Neural Network Posteriors with Symmetric Minibatch Splitting Langevin DynamicsInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2024 Daniel Paulin Peter Whalley Neil K. Chada Benedict Leimkuhler BDL 355 6 0 14 Oct 2024
Uncovering, Explaining, and Mitigating the Superficial Safety of Backdoor DefenseNeural Information Processing Systems (NeurIPS), 2024 Rui Min Zeyu Qin Nevin L. Zhang Li Shen Minhao Cheng AAML 387 8 0 13 Oct 2024
Uncertainty-Aware Optimal Treatment Selection for Clinical Time Series Thomas Schwarz Cecilia Casolo Niki Kilbertus CML 249 0 0 11 Oct 2024
Boosting Deep Ensembles with Learning Rate Tuning Hongpeng Jin Yanzhao Wu 177 2 0 10 Oct 2024
Decouple-Then-Merge: Finetune Diffusion Models as Multi-Task LearningComputer Vision and Pattern Recognition (CVPR), 2024 Qianli Ma Xuefei Ning Dongrui Liu Li Niu Linfeng Zhang MoMe 259 2 0 09 Oct 2024
QT-DoG: Quantization-aware Training for Domain Generalization Saqib Javed Hieu Le Mathieu Salzmann OOD MQ 296 6 0 08 Oct 2024
What Matters for Model Merging at Scale? Prateek Yadav Tu Vu Jonathan Lai Alexandra Chronopoulou Manaal Faruqui Joey Tianyi Zhou Tsendsuren Munkhdalai MoMe 238 41 0 04 Oct 2024
Parameter Competition Balancing for Model MergingNeural Information Processing Systems (NeurIPS), 2024 Guodong DU Junlin Lee Jing Li Runhua Jiang Yifei Guo ... Hanting Liu Sim Kuan Goh Jing Li Daojing He Min Zhang MoMe 195 40 0 03 Oct 2024
Foldable SuperNets: Scalable Merging of Transformers with Different Initializations and Tasks Edan Kinderman Itay Hubara Haggai Maron Daniel Soudry MoMe 308 3 0 02 Oct 2024
Input Space Mode Connectivity in Deep Neural NetworksInternational Conference on Learning Representations (ICLR), 2024 Jakub Vrabel Ori Shem-Ur Yaron Oz David Krueger 302 1 0 09 Sep 2024
Weight Scope Alignment: A Frustratingly Easy Method for Model MergingEuropean Conference on Artificial Intelligence (ECAI), 2024 Yichu Xu Xin-Chun Li Le Gan De-Chuan Zhan MoMe 275 2 0 22 Aug 2024
SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models Anke Tang Li Shen Yong Luo Shuai Xie Han Hu Lefei Zhang Di Lin Dacheng Tao MoMe 240 9 0 19 Aug 2024
Activated Parameter Locating via Causal Intervention for Model Merging Fanshuang Kong Richong Zhang Ziqiao Wang MoMe 146 3 0 18 Aug 2024
Learning to Explore for Stochastic Gradient MCMCInternational Conference on Machine Learning (ICML), 2024 Seunghyun Kim Seohyeon Jung Seonghyeon Kim Juho Lee BDL 230 1 0 17 Aug 2024
Information-Theoretic Progress Measures reveal Grokking is an Emergent Phase Transition Kenzo Clauw S. Stramaglia Daniele Marinazzo 156 7 0 16 Aug 2024
Low-Cost Self-Ensembles Based on Multi-Branch Transformation and Grouped Convolution Hojung Lee Jong-Seok Lee 3DV 179 1 0 05 Aug 2024
Network Fission Ensembles for Low-Cost Self-EnsemblesPattern Recognition Letters (PR), 2024 Hojung Lee Jong-Seok Lee UQCV 477 2 0 05 Aug 2024
Enhancing material property prediction with ensemble deep graph convolutional networks Chowdhury Mohammad Abid Rahman Ghadendra B. Bhandari Nasser M. Nasrabadi Aldo H. Romero P. Gyawali AI4CE 168 9 0 26 Jul 2024
Flatness-aware Sequential Learning Generates Resilient Backdoors Hoang Pham The-Anh Ta Anh Tran Khoa D. Doan FedML AAML 198 1 0 20 Jul 2024
Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergence Shuya Feng Meisam Mohammady Hanbin Hong Shenao Yan Ashish Kundu Binghui Wang Yuan Hong FedML 322 4 0 20 Jul 2024
Training-Free Model Merging for Multi-target Domain Adaptation Wenyi Li Huan-ang Gao Mingju Gao Beiwen Tian Rong Zhi Hao Zhao MoMe 212 10 0 18 Jul 2024
Exploring End-to-end Differentiable Neural Charged Particle Tracking -- A Loss Landscape Perspective T. Kortus Ralf Keidel N.R. Gauger 392 0 0 18 Jul 2024
Seq-to-Final: A Benchmark for Tuning from Sequential Distributions to a Final Time Point Christina X. Ji Ahmed M. Alaa David Sontag OOD 202 0 0 12 Jul 2024
Deep Adversarial Defense Against Multilevel-Lp Attacks Ren Wang Yuxuan Li Alfred Hero AAML 176 1 0 12 Jul 2024
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models Jinliang Lu Ziliang Pang Min Xiao Yaochen Zhu Rui Xia Jiajun Zhang MoMe 324 45 0 08 Jul 2024
Harmony in Diversity: Merging Neural Networks with Canonical Correlation Analysis Stefan Horoi Albert Manuel Orozco Camacho Eugene Belilovsky Guy Wolf FedML MoMe 190 11 0 07 Jul 2024
Randomized Physics-Informed Neural Networks for Bayesian Data Assimilation Yifei Zong D. Barajas-Solano A. Tartakovsky 153 5 0 05 Jul 2024
Face Reconstruction Transfer Attack as Out-of-Distribution Generalization Yoon Gyo Jung Jaewoo Park Xingbo Dong Hojin Park Andrew Beng Jin Teoh Octavia Camps AAML 222 1 0 02 Jul 2024
Reevaluating Theoretical Analysis Methods for Optimization in Deep Learning Hoang Tran Qinzi Zhang Ashok Cutkosky 269 4 0 01 Jul 2024
Adaptive Stochastic Weight Averaging Caglar Demir Arnab Sharma Axel-Cyrille Ngonga Ngomo MoMe 205 5 0 27 Jun 2024
MD tree: a model-diagnostic tree grown on loss landscape Yefan Zhou Jianlong Chen Qinxue Cao Konstantin Schürholt Yaoqing Yang 271 2 0 24 Jun 2024
Landscaping Linear Mode Connectivity Sidak Pal Singh Linara Adilova Michael Kamp Asja Fischer Bernhard Scholkopf Thomas Hofmann 326 9 0 24 Jun 2024
WATT: Weight Average Test-Time Adaptation of CLIP David Osowiechi Mehrdad Noori G. A. V. Hakim Moslem Yazdanpanah Ali Bahri Milad Cheraghalikhani Sahar Dastani Farzad Beizaee Ismail Ben Ayed Christian Desrosiers VLM 206 21 0 19 Jun 2024
Composite Concept Extraction through BackdooringInternational Conference on Pattern Recognition (ICPR), 2024 Banibrata Ghosh Haripriya Harikumar Khoa D. Doan Svetha Venkatesh Santu Rana 232 0 0 19 Jun 2024
Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging Zhenyi Lu Chenghao Fan Wei Wei Xiaoye Qu Dangyang Chen Yu Cheng MoMe 214 86 0 17 Jun 2024
Tilt and Average : Geometric Adjustment of the Last Layer for RecalibrationInternational Conference on Machine Learning (ICML), 2024 Gyusang Cho Chan-Hyun Youn 214 0 0 14 Jun 2024
Diffusion Soup: Model Merging for Text-to-Image Diffusion Models Benjamin Biggs Arjun Seshadri Yang Zou Achin Jain Aditya Golatkar Yusheng Xie Alessandro Achille Ashwin Swaminathan Stefano Soatto MoMe DiffM 190 20 0 12 Jun 2024
Merging Improves Self-Critique Against Jailbreak Attacks Victor Gallego AAML MoMe 277 7 0 11 Jun 2024
The Expanding Scope of the Stability Gap: Unveiling its Presence in Joint Incremental Learning of Homogeneous Tasks Sandesh Kamath Albin Soutif--Cormerais Joost van de Weijer Bogdan Raducanu 211 5 0 07 Jun 2024
FusionBench: A Unified Library and Comprehensive Benchmark for Deep Model Fusion Anke Tang Li Shen Yong Luo Enneng Yang Di Lin Dacheng Tao Bo Du Dacheng Tao ELM MoMe VLM 322 38 0 05 Jun 2024
On the Limitations of Fractal Dimension as a Measure of Generalization Charlie Tan Inés García-Redondo Qiquan Wang M. Bronstein Anthea Monod AI4CE 130 2 0 04 Jun 2024
On the Use of Anchoring for Training Vision Models V. Narayanaswamy Kowshik Thopalli Rushil Anirudh Yamen Mubarka W. Sakla Jayaraman J. Thiagarajan 283 1 0 01 Jun 2024
Improving Generalization and Convergence by Enhancing Implicit Regularization Mingze Wang Haotian He Jinbo Wang Zilin Wang Guanhua Huang Feiyu Xiong Zhiyu Li E. Weinan Lei Wu 213 10 0 31 May 2024