Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training

MoE
Re-assign community