Supermasks in Superposition

26 June 2020

Papers citing "Supermasks in Superposition"

44 / 44 papers shown

Title
Achieving Upper Bound Accuracy of Joint Training in Continual Learning Saleh Momeni Bing Liu CLL 82 1 0 17 Feb 2025
The impact of allocation strategies in subset learning on the expressive power of neural networks Ofir Schlisselberg Ran Darshan 91 0 0 10 Feb 2025
Modular Training of Neural Networks aids Interpretability Satvik Golechha Maheep Chaudhary Joan Velja Alessandro Abate Nandi Schoots 74 0 0 04 Feb 2025
TinySubNets: An efficient and low capacity continual learning strategy Marcin Pietroñ Kamil Faber Dominik Zurek Roberto Corizzo 73 0 0 14 Dec 2024
More Experts Than Galaxies: Conditionally-overlapping Experts With Biologically-Inspired Fixed Routing Sagi Shaier Francisco Pereira K. Wense Lawrence E Hunter Matt Jones MoE 41 0 0 10 Oct 2024
Self-Masking Networks for Unsupervised Adaptation Alfonso Taboada Warmerdam Mathilde Caron Yuki M. Asano 36 1 0 11 Sep 2024
Buffer-based Gradient Projection for Continual Federated Learning Shenghong Dai Jy-yong Sohn Yicong Chen S. Alam Ravikumar Balakrishnan Suman Banerjee N. Himayat Kangwook Lee FedML 71 2 0 03 Sep 2024
Mitigating the Stability-Plasticity Dilemma in Adaptive Train Scheduling with Curriculum-Driven Continual DQN Expansion Achref Jaziri Etienne Kunzel Visvanathan Ramesh CLL 45 0 0 19 Aug 2024
Learning System Dynamics without Forgetting Xikun Zhang Dongjin Song Yushan Jiang Yixin Chen Dacheng Tao AI4CE 35 2 0 30 Jun 2024
Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue System Chen Chen Ruizhe Li Yuchen Hu Yuanyuan Chen Chengwei Qin Qiang Zhang CLL 37 1 0 16 May 2024
Understanding Multimodal Deep Neural Networks: A Concept Selection View Chenming Shang Hengyuan Zhang Hao Wen Yujiu Yang 38 5 0 13 Apr 2024
MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting Computation in Superposition Nicolas Menet Michael Hersche G. Karunaratne Luca Benini Abu Sebastian Abbas Rahimi 28 13 0 05 Dec 2023
Continual Learning of Diffusion Models with Generative Distillation Sergi Masip Pau Rodriguez Tinne Tuytelaars Gido M. van de Ven VLM DiffM 21 7 0 23 Nov 2023
Density Distribution-based Learning Framework for Addressing Online Continual Learning Challenges Shilin Zhang Jiahui Wang CLL 16 0 0 22 Nov 2023
Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs Yu-xin Zhang Lirui Zhao Mingbao Lin Yunyun Sun Yiwu Yao Xingjia Han Jared Tanner Shiwei Liu Rongrong Ji SyDa 32 40 0 13 Oct 2023
Lightweight Learner for Shared Knowledge Lifelong Learning Yunhao Ge Yuecheng Li Di Wu Ao Xu Adam M. Jones ... Gozde Sahin Shuo Ni Kiran Lekkala S. Sontakke Laurent Itti CLL 19 10 0 24 May 2023
Quantum Ridgelet Transform: Winning Lottery Ticket of Neural Networks with Quantum Computation H. Yamasaki Sathyawageeswar Subramanian Satoshi Hayakawa Sho Sonoda MLT 28 4 0 27 Jan 2023
A Distinct Unsupervised Reference Model From The Environment Helps Continual Learning Seyyed Amirhossein Ameli Kalkhoran Mohammadamin Banayeeanzade Mahdi Samiei M. Baghshah BDL CLL 21 0 0 11 Jan 2023
You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets Tianjin Huang Tianlong Chen Meng Fang Vlado Menkovski Jiaxu Zhao ... Yulong Pei D. Mocanu Zhangyang Wang Mykola Pechenizkiy Shiwei Liu GNN 34 14 0 28 Nov 2022
Cooperative data-driven modeling Aleksandr Dekhovich O. T. Turan Jiaxiang Yi Miguel A. Bessa CLL KELM AI4CE 19 5 0 23 Nov 2022
Continual Learning of Natural Language Processing Tasks: A Survey Zixuan Ke Bin Liu KELM CLL VLM 19 67 0 23 Nov 2022
Building a Subspace of Policies for Scalable Continual Learning Jean-Baptiste Gaya T. Doan Lucas Page-Caccia Laure Soulier Ludovic Denoyer Roberta Raileanu CLL 24 29 0 18 Nov 2022
A Theoretical Study on Solving Continual Learning Gyuhak Kim Changnan Xiao Tatsuya Konishi Zixuan Ke Bin Liu CLL 19 70 0 04 Nov 2022
Is Multi-Task Learning an Upper Bound for Continual Learning? Zihao Wu Huy Tran Hamed Pirsiavash Soheil Kolouri SSL CLL 20 4 0 26 Oct 2022
A Multi-Head Model for Continual Learning via Out-of-Distribution Replay Gyuhak Kim Zixuan Ke Bin Liu VLM CLL OODD 15 29 0 20 Aug 2022
Continual Prune-and-Select: Class-incremental learning with specialized subnetworks Aleksandr Dekhovich David Tax M. Sluiter Miguel A. Bessa CLL 18 19 0 09 Aug 2022
Continual Learning for Multivariate Time Series Tasks with Variable Input Dimensions Vibhor Gupta Jyoti Narwariya Pankaj Malhotra L. Vig Gautam M. Shroff AI4TS 19 20 0 14 Mar 2022
Memory Efficient Continual Learning with Transformers B. Ermiş Giovanni Zappella Martin Wistuba Aditya Rawal Cédric Archambeau CLL 19 42 0 09 Mar 2022
Continual Learning with Invertible Generative Models Jary Pomponi Simone Scardapane A. Uncini BDL 37 3 0 11 Feb 2022
Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments A. Iyer Karan Grewal Akash Velu Lucas O. Souza Jérémy Forest Subutai Ahmad AI4CE 16 41 0 31 Dec 2021
Generative Kernel Continual learning Mohammad Mahdi Derakhshani Xiantong Zhen Ling Shao Cees G. M. Snoek BDL VLM 33 0 0 26 Dec 2021
Neural Network Module Decomposition and Recomposition Hiroaki Kingetsu Kenichi Kobayashi Taiji Suzuki 11 10 0 25 Dec 2021
Hidden-Fold Networks: Random Recurrent Residuals Using Sparse Supermasks Ángel López García-Arias Masanori Hashimoto Masato Motomura Jaehoon Yu 23 5 0 24 Nov 2021
Avoiding Forgetting and Allowing Forward Transfer in Continual Learning via Sparse Networks Ghada Sokar D. Mocanu Mykola Pechenizkiy CLL 21 8 0 11 Oct 2021
Powerpropagation: A sparsity inducing weight reparameterisation Jonathan Richard Schwarz Siddhant M. Jayakumar Razvan Pascanu P. Latham Yee Whye Teh 87 54 0 01 Oct 2021
Generative Feature-driven Image Replay for Continual Learning Kevin Thandiackal Tiziano Portenier Andrea Giovannini M. Gabrani O. Goksel CLL VLM DiffM 13 9 0 09 Jun 2021
Structured Ensembles: an Approach to Reduce the Memory Footprint of Ensemble Methods Jary Pomponi Simone Scardapane A. Uncini UQCV 33 7 0 06 May 2021
Efficient Feature Transformations for Discriminative and Generative Continual Learning Vinay K. Verma Kevin J Liang Nikhil Mehta Piyush Rai Lawrence Carin CLL 30 76 0 25 Mar 2021
Recent Advances on Neural Network Pruning at Initialization Huan Wang Can Qin Yue Bai Yulun Zhang Yun Fu CVBM 23 64 0 11 Mar 2021
Sparse Training Theory for Scalable and Efficient Agents D. Mocanu Elena Mocanu T. Pinto Selima Curci Phuong H. Nguyen M. Gibescu D. Ernst Z. Vale 45 17 0 02 Mar 2021
Posterior Meta-Replay for Continual Learning Christian Henning Maria R. Cervera Francesco DÁngelo J. Oswald Regina Traber Benjamin Ehret Seijin Kobayashi Benjamin Grewe João Sacramento CLL BDL 51 54 0 01 Mar 2021
BinPlay: A Binary Latent Autoencoder for Generative Replay Continual Learning Kamil Deja Pawel Wawrzyñski Daniel Marczak Wojciech Masarczyk Tomasz Trzciñski SyDa 11 9 0 25 Nov 2020
Energy-Based Models for Continual Learning Shuang Li Yilun Du Gido M. van de Ven Igor Mordatch 16 42 0 24 Nov 2020
Linear Mode Connectivity in Multitask and Continual Learning Seyed Iman Mirzadeh Mehrdad Farajtabar Dilan Görür Razvan Pascanu H. Ghasemzadeh CLL 29 137 0 09 Oct 2020