Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales

Terms and Conditions

Twitter GitHub LinkedIn Bluesky Youtube

© 2026 ResearchTrend.AI, All rights reserved.

Home
Papers
2505.06708
Cited By

Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free

Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free

10 May 2025

Shangshang Yang

ArXiv (abs)PDF HTML

Papers citing "Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free"

30 / 30 papers shown

Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling

Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling

418

3

0

01 Dec 2025

TGSFormer: Scalable Temporal Gaussian Splatting for Embodied Semantic Scene Completion

TGSFormer: Scalable Temporal Gaussian Splatting for Embodied Semantic Scene Completion

150

0

0

29 Nov 2025

TWEO: Transformers Without Extreme Outliers Enables FP8 Training And Quantization For Dummies

TWEO: Transformers Without Extreme Outliers Enables FP8 Training And Quantization For Dummies

219

1

0

28 Nov 2025

SSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space

SSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space

367

0

0

25 Nov 2025

Training Foundation Models on a Full-Stack AMD Platform: Compute, Networking, and System Design

Training Foundation Models on a Full-Stack AMD Platform: Compute, Networking, and System Design

Quentin G. Anthony

Srivatsan Rajagopal

Praneeth Medepalli

...

292

0

0

21 Nov 2025

CAMS: Towards Compositional Zero-Shot Learning via Gated Cross-Attention and Multi-Space Disentanglement

340

0

0

20 Nov 2025

TNT: Improving Chunkwise Training for Test-Time Memorization

TNT: Improving Chunkwise Training for Test-Time Memorization

Praneeth Kacham

Meisam Razaviyayn

239

2

0

10 Nov 2025

TetraJet-v2: Accurate NVFP4 Training for Large Language Models with Oscillation Suppression and Outlier Control

TetraJet-v2: Accurate NVFP4 Training for Large Language Models with Oscillation Suppression and Outlier Control

157

4

0

31 Oct 2025

Kimi Linear: An Expressive, Efficient Attention Architecture

Kimi Linear: An Expressive, Efficient Attention Architecture

...

157

28

0

30 Oct 2025

Knocking-Heads Attention

Knocking-Heads Attention

122

0

0

27 Oct 2025

Mitigating Attention Sinks and Massive Activations in Audio-Visual Speech Recognition with LLMs

Mitigating Attention Sinks and Massive Activations in Audio-Visual Speech Recognition with LLMs

Umberto Cappellazzo

Stavros Petridis

175

0

0

26 Oct 2025

A Hierarchical Quantized Tokenization Framework for Task-Adaptive Graph Representation Learning

A Hierarchical Quantized Tokenization Framework for Task-Adaptive Graph Representation Learning

148

0

0

14 Oct 2025

Value-State Gated Attention for Mitigating Extreme-Token Phenomena in Transformers

Value-State Gated Attention for Mitigating Extreme-Token Phenomena in Transformers

192

0

0

10 Oct 2025

Which Heads Matter for Reasoning? RL-Guided KV Cache Compression

Which Heads Matter for Reasoning? RL-Guided KV Cache Compression

177

5

0

09 Oct 2025

Revisiting Long-context Modeling from Context Denoising Perspective

Revisiting Long-context Modeling from Context Denoising Perspective

188

1

0

07 Oct 2025

Hybrid Architectures for Language Models: Systematic Analysis and Design Insights

Hybrid Architectures for Language Models: Systematic Analysis and Design Insights

168

5

0

06 Oct 2025

Effective Model Pruning: Measure The Redundancy of Model Components

Effective Model Pruning: Measure The Redundancy of Model Components

Warren E. Dixon

78

0

0

30 Sep 2025

TTT3R: 3D Reconstruction as Test-Time Training

TTT3R: 3D Reconstruction as Test-Time Training

311

22

0

30 Sep 2025

SparseDoctor: Towards Efficient Chat Doctor with Mixture of Experts Enhanced Large Language Models

SparseDoctor: Towards Efficient Chat Doctor with Mixture of Experts Enhanced Large Language Models

Richard Tai-Chiu Hsung

Harris Sik-Ho Tsang

211

0

0

15 Sep 2025

Unveiling Super Experts in Mixture-of-Experts Large Language Models

Unveiling Super Experts in Mixture-of-Experts Large Language Models

296

5

0

31 Jul 2025

On the Expressiveness of Softmax Attention: A Recurrent Neural Network Perspective

On the Expressiveness of Softmax Attention: A Recurrent Neural Network Perspective

Gabriel Mongaras

123

4

0

31 Jul 2025

GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment

GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment

...

282

11

0

18 May 2025

Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing

Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing

Róbert Csordás

Jürgen Schmidhuber

609

8

0

01 May 2025

Softpick: No Attention Sink, No Massive Activations with Rectified Softmax

Softpick: No Attention Sink, No Massive Activations with Rectified Softmax

Zayd Muhammad Kawakibi Zuhri

Erland Hilman Fuadi

Alham Fikri Aji

299

7

0

29 Apr 2025

Numerical Error Analysis of Large Language Models

Stanislav Budzinskiy

Philipp Petersen

240

5

0

13 Mar 2025

LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillation

LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration DistillationAnnual Meeting of the Association for Computational Linguistics (ACL), 2025

844

10

0

11 Feb 2025

Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models

Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert ModelsAnnual Meeting of the Association for Computational Linguistics (ACL), 2025

453

31

0

21 Jan 2025

MiniMax-01: Scaling Foundation Models with Lightning Attention

MiniMax-01: Scaling Foundation Models with Lightning Attention

...

401

112

0

14 Jan 2025

MaskControl: Spatio-Temporal Control for Masked Motion Synthesis

MaskControl: Spatio-Temporal Control for Masked Motion Synthesis

Ekkasit Pinyoanuntapong

Muhammad Usama Saleem

Korrawe Karunratanakul

Sergey Tulyakov

504

101

0

14 Oct 2024

Fibottention: Inceptive Visual Representation Learning with Diverse Attention Across Heads

Fibottention: Inceptive Visual Representation Learning with Diverse Attention Across Heads

Ali Khaleghi Rahimian

Manish Kumar Govind

Dominick Reilly

Christian Kummerle

248

1

0

27 Jun 2024

Page 1 of 1