v1v2v3 (latest)

Editing Models with Task Arithmetic

International Conference on Learning Representations (ICLR), 2022

8 December 2022

ArXiv (abs)PDF HTML HuggingFace (7 upvotes)

Papers citing "Editing Models with Task Arithmetic"

50 / 525 papers shown

Title
Towards Modular LLMs by Building and Reusing a Library of LoRAsInternational Conference on Machine Learning (ICML), 2024 O. Ostapenko Zhan Su Edoardo Ponti Laurent Charlin Nicolas Le Roux Matheus Pereira Lucas Caccia Alessandro Sordoni MoMe 237 51 0 18 May 2024
A safety realignment framework via subspace-oriented model fusion for large language modelsKnowledge-Based Systems (KBS), 2024 Xin Yi Shunfan Zheng Linlin Wang Xiaoling Wang Xiaoling Wang 209 40 0 15 May 2024
Localizing Task Information for Improved Model Merging and CompressionInternational Conference on Machine Learning (ICML), 2024 Ke Wang Nikolaos Dimitriadis Guillermo Ortiz-Jimenez Franccois Fleuret Pascal Frossard MoMe 275 86 0 13 May 2024
Zero-Shot Tokenizer TransferNeural Information Processing Systems (NeurIPS), 2024 Benjamin Minixhofer Edoardo Ponti Ivan Vulić VLM 262 25 0 13 May 2024
Erasing Concepts from Text-to-Image Diffusion Models with Few-shot Unlearning Masane Fuchi Tomohiro Takagi DiffM VLM 252 25 0 12 May 2024
To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language ModelsInternational Conference on Machine Learning (ICML), 2024 George-Octavian Barbulescu Peter Triantafillou MU 329 32 0 06 May 2024
Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuningInternational Conference on Machine Learning (ICML), 2024 Jing Xu Jingzhao Zhang 210 11 0 04 May 2024
Creative Problem Solving in Large Language and Vision Models -- What Would it Take?Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024 Lakshmi Nair Evana Gizzi Jivko Sinapov MLLM 306 4 0 02 May 2024
Espresso: Robust Concept Filtering in Text-to-Image Models Anudeep Das Vasisht Duddu Rui Zhang Nadarajah Asokan EGVM 446 11 0 30 Apr 2024
HFT: Half Fine-Tuning for Large Language Models Tingfeng Hui Ying Tai Shuohuan Wang Weiran Xu Yu Sun Hua Wu CLL 279 13 0 29 Apr 2024
Model Extrapolation Expedites Alignment Chujie Zheng Ziqi Wang Mengyue Yang Shiyu Huang Nanyun Peng MoMe 361 32 0 25 Apr 2024
No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement Mateusz Klimaszewski Piotr Andruszkiewicz Alexandra Birch MoMe 253 5 0 24 Apr 2024
DynaMMo: Dynamic Model Merging for Efficient Class Incremental Learning for Medical Images Mohammad Areeb Qazi Ibrahim Almakky Anees Ur Rehman Hashmi Santosh Sanjeev Mohammad Yaqub MoMe 228 7 0 22 Apr 2024
Decomposing and Editing Predictions by Modeling Model Computation Harshay Shah Andrew Ilyas Aleksander Madry KELM 274 24 0 17 Apr 2024
In-Context Learning State Vector with Inner and Momentum Optimization Dongfang Li Zhenyu Liu Xinshuo Hu Zetian Sun Baotian Hu Min Zhang 248 12 0 17 Apr 2024
MaxFusion: Plug&Play Multi-Modal Generation in Text-to-Image Diffusion Models Nithin Gopalakrishnan Nair Jeya Maria Jose Valanarasu Vishal M. Patel MoMe 269 13 0 15 Apr 2024
Learn Your Reference Model for Real Good Alignment Alexey Gorbatovski Boris Shaposhnikov Alexey Malakhov Nikita Surnachev Gleb Gerasimov Ian Maksimov Nikita Balagansky Daniil Gavrilov OffRL 511 45 0 15 Apr 2024
DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models Nastaran Saadati Minh Pham Nasla Saleem Joshua R. Waite Aditya Balu Zhanhong Jiang Chinmay Hegde Soumik Sarkar MoMe 208 5 0 11 Apr 2024
Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model Merging Tianshuo Cong Delong Ran Zesen Liu Xinlei He Jinyuan Liu Yichen Gong Qi Li Anyu Wang Xiaoyun Wang MoMe 130 22 0 08 Apr 2024
Lossless and Near-Lossless Compression for Foundation Models Moshik Hershcovitch Leshem Choshen Andrew Wood Ilias Enmouri Peter Chin S. Sundararaman Danny Harnik 234 12 0 05 Apr 2024
Digital Forgetting in Large Language Models: A Survey of Unlearning MethodsArtificial Intelligence Review (Artif Intell Rev), 2024 Alberto Blanco-Justicia N. Jebreel Benet Manzanares-Salor David Sánchez Josep Domingo-Ferrer Guillem Collell Kuan Eeik Tan KELM MU 312 40 0 02 Apr 2024
Linear Combination of Saved Checkpoints Makes Consistency and Diffusion Models BetterInternational Conference on Learning Representations (ICLR), 2024 En-hao Liu Junyi Zhu Zinan Lin Xuefei Ning Shuaiqi Wang ... Sergey Yekhanin Guohao Dai Huazhong Yang Yu Wang Yu Wang MoMe 405 5 0 02 Apr 2024
Model Stock: All we need is just a few fine-tuned models Dong-Hwan Jang Sangdoo Yun Dongyoon Han OODD MoMe 324 68 0 28 Mar 2024
A Unified Module for Accelerating STABLE-DIFFUSION: LCM-LORA Ayush Thakur Rashmi Vashisth MoMe 83 5 0 24 Mar 2024
Emergent World Models and Latent Variable Estimation in Chess-Playing Language Models Adam Karvonen 258 38 0 21 Mar 2024
FissionFusion: Fast Geometric Generation and Hierarchical Souping for Medical Image Analysis Santosh Sanjeev Nuren Zhaksylyk Ibrahim Almakky Anees Ur Rehman Hashmi Mohammad Areeb Qazi Mohammad Yaqub 308 4 0 20 Mar 2024
FedFisher: Leveraging Fisher Information for One-Shot Federated LearningInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2024 Divyansh Jhunjhunwala Shiqiang Wang Gauri Joshi FedML 205 20 0 19 Mar 2024
CoCoCo: Improving Text-Guided Video Inpainting for Better Consistency, Controllability and Compatibility Bojia Zi Shihao Zhao Xianbiao Qi Jianan Wang Yukai Shi Qianyu Chen Bin Liang Kam-Fai Wong Lei Zhang DiffM VGen 264 38 0 18 Mar 2024
DAM: Dynamic Adapter Merging for Continual Video QA LearningIEEE Workshop/Winter Conference on Applications of Computer Vision (WACV), 2024 Feng Cheng Ziyang Wang Yi-Lin Sung Yan-Bo Lin Mohit Bansal Gedas Bertasius CLL MoMe 328 18 0 13 Mar 2024
Unleashing the Power of Meta-tuning for Few-shot Generalization Through Sparse Interpolated ExpertsInternational Conference on Machine Learning (ICML), 2024 Shengzhuang Chen Jihoon Tack Yunqiao Yang Yee Whye Teh Jonathan Richard Schwarz Ying Wei MoE 451 4 0 13 Mar 2024
SELMA: Learning and Merging Skill-Specific Text-to-Image Experts with Auto-Generated DataNeural Information Processing Systems (NeurIPS), 2024 Jialu Li Jaemin Cho Yi-Lin Sung Jaehong Yoon Mohit Bansal MoMe DiffM 225 15 0 11 Mar 2024
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning Nathaniel Li Alexander Pan Anjali Gopal Summer Yue Daniel Berrios ... Yan Shoshitaishvili Jimmy Ba K. Esvelt Alexandr Wang Dan Hendrycks ELM 714 294 0 05 Mar 2024
Training-Free Pretrained Model Merging Zhenxing Xu Ke Yuan Huiqiong Wang Yong Wang Weilong Dai Mingli Song MoMe 370 22 0 04 Mar 2024
Dissecting Language Models: Machine Unlearning via Selective Pruning Nicholas Pochinkov Nandi Schoots MILM MU 174 31 0 02 Mar 2024
Eight Methods to Evaluate Robust Unlearning in LLMs Aengus Lynch Phillip Guo Aidan Ewart Stephen Casper Dylan Hadfield-Menell ELM MU 310 115 0 26 Feb 2024
Training Neural Networks from Scratch with Parallel Low-Rank Adapters Minyoung Huh Brian Cheung Jeremy Bernstein Phillip Isola Pulkit Agrawal 250 15 0 26 Feb 2024
InstructEdit: Instruction-based Knowledge Editing for Large Language Models Ningyu Zhang Bo Tian Siyuan Cheng Xiaozhuan Liang Yi Hu Kouying Xue Yanjie Gou Xi Chen Huajun Chen KELM 178 10 0 25 Feb 2024
Knowledge Fusion of Chat LLMs: A Preliminary Technical Report Fanqi Wan Ziyi Yang Longguang Zhong Xiaojun Quan Xinting Huang Wei Bi MoMe 438 2 0 25 Feb 2024
Does Combining Parameter-efficient Modules Improve Few-shot Transfer Accuracy? Nader Asadi Mahdi Beitollahi Yasser H. Khalil Yinchuan Li Guojun Zhang Xi Chen MoMe 236 12 0 23 Feb 2024
Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking Nikhil Prakash Tamar Rott Shaham Tal Haklay Yonatan Belinkov David Bau 298 94 0 22 Feb 2024
Q-Probe: A Lightweight Approach to Reward Maximization for Language Models Kenneth Li Samy Jelassi Hugh Zhang Sham Kakade Martin Wattenberg David Brandfonbrener 271 15 0 22 Feb 2024
Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization James Oldfield Markos Georgopoulos Grigorios G. Chrysos Christos Tzelepis Yannis Panagakis M. Nicolaou Jiankang Deng Ioannis Patras MoE 257 14 0 19 Feb 2024
Rethinking Machine Unlearning for Large Language Models Sijia Liu Yuanshun Yao Jinghan Jia Stephen Casper Nathalie Baracaldo ... Hang Li Kush R. Varshney Mohit Bansal Sanmi Koyejo Yang Liu AILaw MU 398 196 0 13 Feb 2024
Learning to Route Among Specialized Experts for Zero-Shot Generalization Mohammed Muqeeth Haokun Liu Yufan Liu Colin Raffel MoMe 248 52 0 08 Feb 2024
On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning ParadigmInternational Conference on Machine Learning (ICML), 2024 Zhanpeng Zhou Zijun Chen Yilan Chen Bo Zhang Junchi Yan MoMe 357 22 0 06 Feb 2024
Representation Surgery for Multi-Task Model Merging Enneng Yang Li Shen Zhenyi Wang Guibing Guo Xiaojun Chen Xingwei Wang Dacheng Tao MoMe 279 77 0 05 Feb 2024
MixedNUTS: Training-Free Accuracy-Robustness Balance via Nonlinearly Mixed Classifiers Yatong Bai Mo Zhou Vishal M. Patel Somayeh Sojoudi AAML 359 16 0 03 Feb 2024
Merging Multi-Task Models via Weight-Ensembling Mixture of Experts Anke Tang Li Shen Yong Luo Nan Yin Lefei Zhang Dacheng Tao MoMe 272 80 0 01 Feb 2024
How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation? Rheeya Uppaal Yixuan Li Junjie Hu 440 6 0 31 Jan 2024
WARM: On the Benefits of Weight Averaged Reward ModelsInternational Conference on Machine Learning (ICML), 2024 Alexandre Ramé Nino Vieillard Léonard Hussenot Robert Dadashi Geoffrey Cideron Olivier Bachem Johan Ferret 342 130 0 22 Jan 2024