SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From
Pre-Trained Foundation Models

SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models

19 August 2024

Li Shen

Han Hu

Lefei Zhang

Dacheng Tao

Papers citing "SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models"

4 / 4 papers shown

Title
Task Singular Vectors: Reducing Task Interference in Model Merging Antonio Andrea Gargiulo Donato Crisostomi Maria Sofia Bucarelli Simone Scardapane Fabrizio Silvestri Emanuele Rodolà MoMe 83 8 0 26 Nov 2024
Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware Subspace Jinluan Yang A. Tang Didi Zhu Zhengyu Chen Li Shen Fei Wu MoMe AAML 45 2 0 17 Oct 2024
Representation Surgery for Multi-Task Model Merging Enneng Yang Li Shen Zhenyi Wang Guibing Guo Xiaojun Chen Xingwei Wang Dacheng Tao MoMe 33 5 0 05 Feb 2024
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 239 313 0 11 Sep 2022