Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization

MoMe
MoE
Re-assign community