ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2410.13835
  4. Cited By
Active-Dormant Attention Heads: Mechanistically Demystifying
  Extreme-Token Phenomena in LLMs

Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs

17 October 2024
Tianyu Guo
Druv Pai
Yu Bai
Jiantao Jiao
Michael I. Jordan
Song Mei
ArXivPDFHTML

Papers citing "Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs"

2 / 2 papers shown
Title
Softpick: No Attention Sink, No Massive Activations with Rectified Softmax
Softpick: No Attention Sink, No Massive Activations with Rectified Softmax
Zayd Muhammad Kawakibi Zuhri
Erland Hilman Fuadi
Alham Fikri Aji
24
0
0
29 Apr 2025
Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
Prateek Chhikara
Dev Khant
Saket Aryan
Taranjeet Singh
Deshraj Yadav
LLMAG
RALM
52
0
0
28 Apr 2025
1