Learning Neural Network Subspaces

20 February 2021

Papers citing "Learning Neural Network Subspaces"

50 / 67 papers shown

Title
Boosting-inspired online learning with transfer for railway maintenance Diogo Risca Afonso Lourenço Goreti Marreiros 18 0 0 11 Apr 2025
You Only Debias Once: Towards Flexible Accuracy-Fairness Trade-offs at Inference Time Xiaotian Han Tianlong Chen Kaixiong Zhou Zhimeng Jiang Zhangyang Wang Xia Hu 156 0 0 10 Mar 2025
SplatPose: Geometry-Aware 6-DoF Pose Estimation from Single RGB Image via 3D Gaussian Splatting Linqi Yang Xiongwei Zhao Qihao Sun Ke Wang Ao Chen Peng Kang 3DGS 78 2 0 07 Mar 2025
Orthogonal Calibration for Asynchronous Federated Learning Jiayun Zhang Shuheng Li Haiyu Huang Xiaofan Yu Rajesh K. Gupta Jingbo Shang FedML 60 0 0 21 Feb 2025
CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling Kaiyuan Zhang Siyuan Cheng Guangyu Shen Bruno Ribeiro Shengwei An Pin-Yu Chen X. Zhang Ninghui Li 92 1 0 28 Jan 2025
Parameter-Efficient Interventions for Enhanced Model Merging Marcin Osial Daniel Marczak Bartosz Zieliñski MoMe 84 1 0 22 Dec 2024
Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learning Anthony Kobanda Rémy Portelas Odalric-Ambrym Maillard Ludovic Denoyer OffRL CLL 77 0 0 19 Dec 2024
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 35 0 0 11 Nov 2024
Collective Model Intelligence Requires Compatible Specialization Jyothish Pari Samy Jelassi Pulkit Agrawal MoMe 46 1 0 04 Nov 2024
Local Superior Soups: A Catalyst for Model Merging in Cross-Silo Federated Learning Minghui Chen Meirui Jiang Xin Zhang Qi Dou Zehua Wang Xiaoxiao Li MoMe FedML 45 2 0 31 Oct 2024
Glider: Global and Local Instruction-Driven Expert Router Pingzhi Li Prateek Yadav Jaehong Yoon Jie Peng Yi-Lin Sung Mohit Bansal Tianlong Chen MoMe MoE 27 1 0 09 Oct 2024
QT-DoG: Quantization-aware Training for Domain Generalization Saqib Javed Hieu Le Mathieu Salzmann OOD MQ 28 1 0 08 Oct 2024
A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning Prateek Yadav Colin Raffel Mohammed Muqeeth Lucas Page-Caccia Haokun Liu Tianlong Chen Mohit Bansal Leshem Choshen Alessandro Sordoni MoMe 43 21 0 13 Aug 2024
Learning Scalable Model Soup on a Single GPU: An Efficient Subspace Training Strategy Tao Li Weisen Jiang Fanghui Liu X. Huang James T. Kwok MoMe 55 1 0 04 Jul 2024
Towards Scalable and Versatile Weight Space Learning Konstantin Schurholt Michael W. Mahoney Damian Borth 40 15 0 14 Jun 2024
FDLoRA: Personalized Federated Learning of Large Language Model via Dual LoRA Tuning Jiaxing Qi Zhongzhi Luan Shaohan Huang Carol J. Fung Hailong Yang Depei Qian 32 12 0 12 Jun 2024
Simultaneous linear connectivity of neural networks modulo permutation Ekansh Sharma Devin Kwok Tom Denton Daniel M. Roy David Rolnick Gintare Karolina Dziugaite 189 7 0 09 Apr 2024
Federated Learning over Connected Modes Dennis Grinwald Philipp Wiesner Shinichi Nakajima FedML 45 0 0 05 Mar 2024
Improving Group Connectivity for Generalization of Federated Deep Learning Zexi Li Jie Lin Zhiqi Li Didi Zhu Chao Wu AI4CE FedML 38 0 0 29 Feb 2024
How Good is a Single Basin? Kai Lion Lorenzo Noci Thomas Hofmann Gregor Bachmann UQCV 23 2 0 05 Feb 2024
Bayesian Semi-structured Subspace Inference Daniel Dold David Rügamer Beate Sick Oliver Durr BDL 24 1 0 23 Jan 2024
Asynchronous Local-SGD Training for Language Modeling Bo Liu Rachita Chhaparia Arthur Douillard Satyen Kale Andrei A. Rusu Jiajun Shen Arthur Szlam MarcÁurelio Ranzato FedML 33 10 0 17 Jan 2024
Merging by Matching Models in Task Parameter Subspaces Derek Tam Mohit Bansal Colin Raffel MoMe 21 10 0 07 Dec 2023
DiLoCo: Distributed Low-Communication Training of Language Models Arthur Douillard Qixuang Feng Andrei A. Rusu Rachita Chhaparia Yani Donchev A. Kuncoro MarcÁurelio Ranzato Arthur Szlam Jiajun Shen 56 31 0 14 Nov 2023
MIST: Defending Against Membership Inference Attacks Through Membership-Invariant Subspace Training Jiacheng Li Ninghui Li Bruno Ribeiro 30 2 0 02 Nov 2023
Improving generalization in large language models by learning prefix subspaces Louis Falissard Vincent Guigue Laure Soulier 11 1 0 24 Oct 2023
Revisiting Deep Ensemble for Out-of-Distribution Detection: A Loss Landscape Perspective Kun Fang Qinghua Tao Xiaolin Huang Jie-jin Yang OODD 40 2 0 22 Oct 2023
Deep Model Fusion: A Survey Weishi Li Yong Peng Miao Zhang Liang Ding Han Hu Li Shen FedML MoMe 28 51 0 27 Sep 2023
Lightweight Adaptation of Neural Language Models via Subspace Embedding Amit Kumar Jaiswal Haiming Liu 23 2 0 16 Aug 2023
Layer-wise Linear Mode Connectivity Linara Adilova Maksym Andriushchenko Michael Kamp Asja Fischer Martin Jaggi FedML FAtt MoMe 28 15 0 13 Jul 2023
Improving Online Continual Learning Performance and Stability with Temporal Ensembles Albin Soutif--Cormerais Antonio Carta Joost van de Weijer CLL 10 10 0 29 Jun 2023
Slimmable Encoders for Flexible Split DNNs in Bandwidth and Resource Constrained IoT Systems Juliano S. Assine José Cândido Silveira Santos Filho Eduardo Valle Marco Levorato 19 2 0 22 Jun 2023
Traversing Between Modes in Function Space for Fast Ensembling Eunggu Yun Hyungi Lee G. Nam Juho Lee UQCV 24 2 0 20 Jun 2023
Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning Moonseok Choi Hyungi Lee G. Nam Juho Lee 27 2 0 24 May 2023
Transferring Learning Trajectories of Neural Networks Daiki Chijiwa 17 2 0 23 May 2023
Subspace-Configurable Networks Dong Wang O. Saukh Xiaoxi He Lothar Thiele OOD 30 0 0 22 May 2023
Sparsified Model Zoo Twins: Investigating Populations of Sparsified Neural Network Models D. Honegger Konstantin Schurholt Damian Borth 22 4 0 26 Apr 2023
DART: Diversify-Aggregate-Repeat Training Improves Generalization of Neural Networks Samyak Jain Sravanti Addepalli P. Sahu Priyam Dey R. Venkatesh Babu MoMe OOD 35 20 0 28 Feb 2023
Image Reconstruction via Deep Image Prior Subspaces Riccardo Barbano Javier Antorán Johannes Leuschner José Miguel Hernández-Lobato Bangti Jin vZeljko Kereta 31 1 0 20 Feb 2023
Editing Models with Task Arithmetic Gabriel Ilharco Marco Tulio Ribeiro Mitchell Wortsman Suchin Gururangan Ludwig Schmidt Hannaneh Hajishirzi Ali Farhadi KELM MoMe MU 45 424 0 08 Dec 2022
Building a Subspace of Policies for Scalable Continual Learning Jean-Baptiste Gaya T. Doan Lucas Page-Caccia Laure Soulier Ludovic Denoyer Roberta Raileanu CLL 29 29 0 18 Nov 2022
Mechanistic Mode Connectivity Ekdeep Singh Lubana Eric J. Bigelow Robert P. Dick David M. Krueger Hidenori Tanaka 27 45 0 15 Nov 2022
lo-fi: distributed fine-tuning without communication Mitchell Wortsman Suchin Gururangan Shen Li Ali Farhadi Ludwig Schmidt Michael G. Rabbat Ari S. Morcos 24 24 0 19 Oct 2022
Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task models Nikolaos Dimitriadis P. Frossard Franccois Fleuret 16 25 0 18 Oct 2022
Multiple Modes for Continual Learning Siddhartha Datta N. Shadbolt CLL MoMe 41 2 0 29 Sep 2022
Model Zoos: A Dataset of Diverse Populations of Neural Network Models Konstantin Schurholt Diyar Taskiran Boris Knyazev Xavier Giró-i-Nieto Damian Borth 52 29 0 29 Sep 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 252 313 0 11 Sep 2022
SPIN: An Empirical Evaluation on Sharing Parameters of Isotropic Networks Chien-Yu Lin Anish K. Prabhu Thomas Merth Sachin Mehta Anurag Ranjan Maxwell Horton Mohammad Rastegari 16 2 0 21 Jul 2022
CoSCL: Cooperation of Small Continual Learners is Stronger than a Big One Liyuan Wang Xingxing Zhang Qian Li Jun Zhu Yi Zhong CLL 25 46 0 13 Jul 2022
On the Subspace Structure of Gradient-Based Meta-Learning Gustaf Tegnér Alfredo Reichlin Hang Yin Mårten Björkman Danica Kragic 29 0 0 08 Jul 2022