Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch

6 November 2023

Papers citing "Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch"

50 / 56 papers shown

Title
SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning Jinpeng Chen Runmin Cong Yuzhi Zhao Hongzheng Yang Guangneng Hu H. Ip Sam Kwong CLL KELM 59 0 0 05 May 2025
Dynamic Fisher-weighted Model Merging via Bayesian Optimization Sanwoo Lee Jiahao Liu Qifan Wang J. Wang Xunliang Cai Yunfang Wu MoMe 47 0 0 26 Apr 2025
Param $Δ$ for Direct Weight Mixing: Post-Train Large Language Model at Zero Cost Sheng Cao Mingrui Wu Karthik Prasad Yuandong Tian Zechun Liu MoMe 74 0 0 23 Apr 2025
When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers Hongkang Li Yihua Zhang Shuai Zhang M. Wang Sijia Liu Pin-Yu Chen MoMe 43 2 0 15 Apr 2025
SEA-LION: Southeast Asian Languages in One Network Raymond Ng Thanh Ngan Nguyen Yuli Huang Ngee Chia Tai Wai Yi Leong ... David Ong Tat-Wee B. Liu William-Chandra Tjhi Erik Cambria Leslie Teo 24 11 0 08 Apr 2025
ZJUKLAB at SemEval-2025 Task 4: Unlearning via Model Merging Haoming Xu Shuxun Wang Yanqiu Zhao Yi Zhong Ziyan Jiang Ningyuan Zhao Shumin Deng H. Chen N. Zhang MoMe MU 61 0 0 27 Mar 2025
Enhanced Continual Learning of Vision-Language Models with Model Fusion Haoyuan Gao Zicong Zhang Yuqi Wei Linglan Zhao Guilin Li Y. Li Linghe Kong Weiran Huang CLL VLM 65 0 0 12 Mar 2025
Liger: Linearizing Large Language Models to Gated Recurrent Structures Disen Lan Weigao Sun Jiaxi Hu Jusen Du Yu-Xi Cheng 64 0 0 03 Mar 2025
CAMEx: Curvature-aware Merging of Experts Dung V. Nguyen Minh H. Nguyen Luc Q. Nguyen R. Teo T. Nguyen Linh Duy Tran MoMe 56 2 0 26 Feb 2025
Parameter Efficient Merging for Multimodal Large Language Models with Complementary Parameter Adaptation Fanhu Zeng Haiyang Guo Fei Zhu Li Shen Hao Tang MoMe 43 1 0 24 Feb 2025
Low-Rank and Sparse Model Merging for Multi-Lingual Speech Recognition and Translation Qiuming Zhao Guangzhi Sun Chao Zhang Mingxing Xu Thomas Fang Zheng MoMe VLM 47 0 0 24 Feb 2025
LED-Merging: Mitigating Safety-Utility Conflicts in Model Merging with Location-Election-Disjoint Qianli Ma Dongrui Liu Qian Chen Linfeng Zhang Jing Shao MoMe 47 0 0 24 Feb 2025
Mixup Model Merge: Enhancing Model Merging Performance through Randomized Linear Interpolation Yue Zhou Yi-Ju Chang Yuan Wu MoMe 57 2 0 24 Feb 2025
Sens-Merging: Sensitivity-Guided Parameter Balancing for Merging Large Language Models Shuqi Liu Han Wu Bowei He Xiongwei Han M. Yuan Linqi Song MoMe 35 1 0 20 Feb 2025
SuperMerge: An Approach For Gradient-Based Model Merging Haoyu Yang Zheng Zhang Saket Sathe MoMe 118 0 0 17 Feb 2025
Exploring Translation Mechanism of Large Language Models Hongbin Zhang Kehai Chen Xuefeng Bai Xiucheng Li Yang Xiang Min Zhang 47 1 0 17 Feb 2025
Optimal Brain Iterative Merging: Mitigating Interference in LLM Merging Zhixiang Wang Zhenyu Mao Yixuan Qiao Yunfang Wu Biye Li MoMe 73 0 0 17 Feb 2025
Sparse High Rank Adapters K. Bhardwaj N. Pandey Sweta Priyadarshi Viswanath Ganapathy Rafael Esteves ... P. Whatmough Risheek Garrepalli M. V. Baalen Harris Teague Markus Nagel MQ 31 4 0 28 Jan 2025
Evolutionary Optimization of Model Merging Recipes Takuya Akiba Makoto Shing Yujin Tang Qi Sun David Ha MoMe 98 96 0 28 Jan 2025
Multi-Task Model Merging via Adaptive Weight Disentanglement Feng Xiong Runxi Cheng Wang Chen Zhanqiu Zhang Yiwen Guo Chun Yuan Ruifeng Xu MoMe 80 4 0 10 Jan 2025
Task Singular Vectors: Reducing Task Interference in Model Merging Antonio Andrea Gargiulo Donato Crisostomi Maria Sofia Bucarelli Simone Scardapane Fabrizio Silvestri Emanuele Rodolà MoMe 83 8 0 26 Nov 2024
Collective Model Intelligence Requires Compatible Specialization Jyothish Pari Samy Jelassi Pulkit Agrawal MoMe 17 1 0 04 Nov 2024
Can Large Language Models Invent Algorithms to Improve Themselves? Yoichi Ishibashi Taro Yano Masafumi Oyamada AIFin LRM 25 1 0 21 Oct 2024
A Unified View of Delta Parameter Editing in Post-Trained Large-Scale Models Qiaoyu Tang Le Yu Bowen Yu Hongyu Lin K. Lu Y. Lu Xianpei Han Le Sun MoMe 24 1 0 17 Oct 2024
Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware Subspace Jinluan Yang A. Tang Didi Zhu Zhengyu Chen Li Shen Fei Wu MoMe AAML 48 2 0 17 Oct 2024
On the Role of Attention Heads in Large Language Model Safety Z. Zhou Haiyang Yu Xinghua Zhang Rongwu Xu Fei Huang Kun Wang Yang Liu Junfeng Fang Yongbin Li 38 5 0 17 Oct 2024
Self-Data Distillation for Recovering Quality in Pruned Large Language Models Vithursan Thangarasa Ganesh Venkatesh Mike Lasby Nish Sinnadurai Sean Lie SyDa 30 0 0 13 Oct 2024
ELICIT: LLM Augmentation via External In-Context Capability Futing Wang Jianhao Yan Yue Zhang Tao Lin 33 0 0 12 Oct 2024
DARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned Models Wenlong Deng Yize Zhao V. Vakilian Minghui Chen Xiaoxiao Li Christos Thrampoulidis 33 3 0 12 Oct 2024
SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture Jiayi Han Liang Du Hongwei Du Xiangguo Zhou Yiwen Wu Weibo Zheng Donghong Han CLL MoMe MoE 29 2 0 10 Oct 2024
Extracting and Transferring Abilities For Building Multi-lingual Ability-enhanced Large Language Models Zhipeng Chen Liang Song K. Zhou Wayne Xin Zhao B. Wang Weipeng Chen Ji-Rong Wen 57 0 0 10 Oct 2024
Determine-Then-Ensemble: Necessity of Top-k Union for Large Language Model Ensembling Yuxuan Yao Han Wu Mingyang Liu Sichun Luo Xiongwei Han Jie Liu Zhijiang Guo Linqi Song 37 4 0 03 Oct 2024
Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models Lucas Bandarkar Benjamin Muller Pritish Yuvraj Rui Hou Nayan Singhal Hongjiang Lv Bing-Quan Liu KELM LRM MoMe 16 2 0 02 Oct 2024
Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging Yiming Ju Ziyi Ni Xingrun Xing Zhixiong Zeng hanyu Zhao Siqi Fan Zheng Zhang MoMe 21 2 0 01 Oct 2024
Recent Advances in Attack and Defense Approaches of Large Language Models Jing Cui Yishi Xu Zhewei Huang Shuchang Zhou Jianbin Jiao Junge Zhang PILM AAML 45 1 0 05 Sep 2024
From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuning Wei Chen Zhen Huang Liang Xie Binbin Lin Houqiang Li ... Deng Cai Yonggang Zhang Wenxiao Wang Xu Shen Jieping Ye 33 6 0 03 Sep 2024
Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement Le Yu Bowen Yu Haiyang Yu Fei Huang Yongbin Li MoMe 27 5 0 06 Aug 2024
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning Yupeng Chen Senmiao Wang Zhihang Lin Zhihang Lin Yushun Zhang Tian Ding Ruoyu Sun Ruoyu Sun CLL 60 1 0 30 Jul 2024
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models Jinliang Lu Ziliang Pang Min Xiao Yaochen Zhu Rui Xia Jiajun Zhang MoMe 19 17 0 08 Jul 2024
PLeaS -- Merging Models with Permutations and Least Squares Anshul Nasery J. Hayase Pang Wei Koh Sewoong Oh MoMe 26 3 0 02 Jul 2024
It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimization Bingdong Li Zixiang Di Yanting Yang Hong Qian Peng Yang Hao Hao Ke Tang Aimin Zhou MoMe 19 5 0 29 Jun 2024
WARP: On the Benefits of Weight Averaged Rewarded Policies Alexandre Ramé Johan Ferret Nino Vieillard Robert Dadashi Léonard Hussenot Pierre-Louis Cedoz Pier Giuseppe Sessa Sertan Girgin Arthur Douillard Olivier Bachem 39 13 0 24 Jun 2024
A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluations Jinqiang Wang Huansheng Ning Yi Peng Qikai Wei Daniel Tesfai Wenwei Mao Tao Zhu Runhe Huang LM&MA AI4MH ELM 28 4 0 14 Jun 2024
Merging Improves Self-Critique Against Jailbreak Attacks Victor Gallego AAML MoMe 23 3 0 11 Jun 2024
MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation Lu Li T. Zhang Zhiqi Bu Suyuchen Wang Huan He Jie Fu Yonghui Wu Jiang Bian Yong Chen Yoshua Bengio FedML MoMe 89 3 0 11 Jun 2024
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning Yibo Yang Xiaojie Li Zhongzhu Zhou S. Song Jianlong Wu Liqiang Nie Bernard Ghanem 39 6 0 07 Jun 2024
HFT: Half Fine-Tuning for Large Language Models Tingfeng Hui Zhenyu Zhang Shuohuan Wang Weiran Xu Yu Sun Hua-Hong Wu CLL 29 4 0 29 Apr 2024
A Survey on Self-Evolution of Large Language Models Zhengwei Tao Ting-En Lin Xiancai Chen Hangyu Li Yuchuan Wu Yongbin Li Zhi Jin Fei Huang Dacheng Tao Jingren Zhou LRM LM&Ro 38 21 0 22 Apr 2024
MedExpQA: Multilingual Benchmarking of Large Language Models for Medical Question Answering Inigo Alonso Maite Oronoz Rodrigo Agerri AI4MH LM&MA ELM 34 14 1 08 Apr 2024
Linear Combination of Saved Checkpoints Makes Consistency and Diffusion Models Better En-hao Liu Junyi Zhu Zinan Lin Xuefei Ning Shuaiqi Wang ... Sergey Yekhanin Guohao Dai Huazhong Yang Yu-Xiang Wang Yu Wang MoMe 53 4 0 02 Apr 2024