ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2405.13997
  4. Cited By
Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture
  of Experts

Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts

22 May 2024
Huy Nguyen
Nhat Ho
Alessandro Rinaldo
ArXivPDFHTML

Papers citing "Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts"

5 / 5 papers shown
Title
Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts
Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts
Minh Le
Chau Nguyen
Huy Nguyen
Quyen Tran
Trung Le
Nhat Ho
25
3
0
03 Oct 2024
On Least Square Estimation in Softmax Gating Mixture of Experts
On Least Square Estimation in Softmax Gating Mixture of Experts
Huy Nguyen
Nhat Ho
Alessandro Rinaldo
28
13
0
05 Feb 2024
FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion
FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion
Xing Han
Huy Nguyen
Carl Harris
Nhat Ho
S. Saria
MoE
39
16
0
05 Feb 2024
From Sparse to Soft Mixtures of Experts
From Sparse to Soft Mixtures of Experts
J. Puigcerver
C. Riquelme
Basil Mustafa
N. Houlsby
MoE
114
114
0
02 Aug 2023
Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT
Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT
James Lee-Thorp
Joshua Ainslie
MoE
22
11
0
24 May 2022
1