FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision

11 July 2024

Papers citing "FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision"

50 / 82 papers shown

Title
BiGSCoder: State Space Model for Code Understanding Shweta Verma Abhinav Anand Mira Mezini Mamba 36 0 0 02 May 2025
GPU Performance Portability needs Autotuning Burkhard Ringlein Thomas Parnell Radu Stoica 43 0 0 30 Apr 2025
RWKV-X: A Linear Complexity Hybrid Language Model Haowen Hou Zhiyi Huang Kaifeng Tan Rongchang Lu Fei Richard Yu VLM 78 0 0 30 Apr 2025
TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate A. Zandieh Majid Daliri Majid Hadian Vahab Mirrokni MQ 74 0 0 28 Apr 2025
Tempo: Application-aware LLM Serving with Mixed SLO Requirements Wei Zhang Zhiyu Wu Yi Mu Banruo Liu Myungjin Lee Fan Lai 51 0 0 24 Apr 2025
TileLang: A Composable Tiled Programming Model for AI Systems Lei Wang Yu Cheng Yining Shi Zhengju Tang Zhiwen Mo ... Lingxiao Ma Yuqing Xia Jilong Xue Fan Yang Z. Yang 54 1 0 24 Apr 2025
Generalized Neighborhood Attention: Multi-dimensional Sparse Attention at the Speed of Light Ali Hassani Fengzhe Zhou Aditya Kane Jiannan Huang Chieh-Yun Chen ... Bing Xu Haicheng Wu Wen-mei W. Hwu Ming-Yu Liu Humphrey Shi 24 0 0 23 Apr 2025
Efficient Pretraining Length Scaling Bohong Wu Shen Yan Sijun Zhang Jianqiao Lu Yutao Zeng Ya Wang Xun Zhou 49 0 0 21 Apr 2025
ModernBERT or DeBERTaV3? Examining Architecture and Data Influence on Transformer Encoder Models Performance Wissam Antoun B. Sagot Djamé Seddah MQ 30 0 0 11 Apr 2025
Accelerating LLM Inference Throughput via Asynchronous KV Cache Prefetching Yanhao Dong Yubo Miao Weinan Li Xiao Zheng Chao Wang Feng Lyu 24 0 0 08 Apr 2025
One-Minute Video Generation with Test-Time Training Karan Dalal Daniel Koceja Gashon Hussein Jiarui Xu Yue Zhao ... Tatsunori Hashimoto Sanmi Koyejo Yejin Choi Yu Sun Xiaolong Wang ViT 91 3 0 07 Apr 2025
HeterMoE: Efficient Training of Mixture-of-Experts Models on Heterogeneous GPUs Yongji Wu Xueshen Liu Shuowei Jin Ceyu Xu Feng Qian Ziming Mao Matthew Lentz Danyang Zhuo Ion Stoica MoMe MoE 59 0 0 04 Apr 2025
ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Grounding Indraneil Paul Haoyi Yang Goran Glavas Kristian Kersting Iryna Gurevych AAML SyDa 34 0 0 27 Mar 2025
BitDecoding: Unlocking Tensor Cores for Long-Context LLMs Decoding with Low-Bit KV Cache Dayou Du Shijie Cao Jianyi Cheng Ting Cao M. Yang MQ 64 0 0 24 Mar 2025
ML-Triton, A Multi-Level Compilation and Language Extension to Triton GPU Programming Dewei Wang Wei Zhu Liyang Ling Ettore Tiotto Quintin Wang Whitney Tsang Julian Opperman Jacky Deng 36 0 0 19 Mar 2025
Tiled Flash Linear Attention: More Efficient Linear RNN and xLSTM Kernels M. Beck Korbinian Poppel Phillip Lippe Sepp Hochreiter 59 1 0 18 Mar 2025
Theoretical Foundation of Flow-Based Time Series Generation: Provable Approximation, Generalization, and Efficiency Jiangxuan Long Zhao-quan Song Chiwun Yang AI4TS 73 0 0 18 Mar 2025
Fake Runs, Real Fixes -- Analyzing xPU Performance Through Simulation Ioannis Zarkadas Amanda Tomlinson Asaf Cidon Baris Kasikci Ofir Weisse 45 0 0 18 Mar 2025
xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference M. Beck Korbinian Poppel Phillip Lippe Richard Kurle P. Blies G. Klambauer Sebastian Böck Sepp Hochreiter LRM 40 0 0 17 Mar 2025
VGGT: Visual Geometry Grounded Transformer Jianyuan Wang Minghao Chen Nikita Karaev Andrea Vedaldi Christian Rupprecht David Novotny ViT 48 6 0 14 Mar 2025
L $^2$ M: Mutual Information Scaling Law for Long-Context Language Modeling Zhuo Chen Oriol Mayné i Comas Zhuotao Jin Di Luo Marin Soljacic 62 0 0 06 Mar 2025
Predicting Team Performance from Communications in Simulated Search-and-Rescue Ali Jalal-Kamali Nikolos Gurney David Pynadath AI4TS 105 8 0 05 Mar 2025
FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference Xunhao Lai Jianqiao Lu Yao Luo Yiyuan Ma Xun Zhou 63 5 0 28 Feb 2025
Training-free and Adaptive Sparse Attention for Efficient Long Video Generation Yifei Xia Suhan Ling Fangcheng Fu Y. Wang Huixia Li Xuefeng Xiao Bin Cui VGen 51 2 0 28 Feb 2025
DReSD: Dense Retrieval for Speculative Decoding Milan Gritta Huiyin Xue Gerasimos Lampouras RALM 93 0 0 24 Feb 2025
AttentionEngine: A Versatile Framework for Efficient Attention Mechanisms on Diverse Hardware Platforms Feiyang Chen Yu Cheng Lei Wang Yuqing Xia Ziming Miao ... Fan Yang J. Xue Zhi Yang M. Yang H. Chen 71 1 0 24 Feb 2025
Simpler Fast Vision Transformers with a Jumbo CLS Token A. Fuller Yousef Yassin Daniel G. Kyrollos Evan Shelhamer James R. Green 67 0 0 24 Feb 2025
Compression Barriers for Autoregressive Transformers Themistoklis Haris Krzysztof Onak 35 1 0 21 Feb 2025
Surface Vision Mamba: Leveraging Bidirectional State Space Model for Efficient Spherical Manifold Representation Rongzhao He Weihao Zheng Leilei Zhao Ying Wang Dalin Zhu Dan Wu Bin Hu Mamba 84 0 0 21 Feb 2025
Slamming: Training a Speech Language Model on One GPU in a Day Gallil Maimon Avishai Elmakies Yossi Adi 38 3 0 19 Feb 2025
HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading Cheng Luo Zefan Cai Hanshi Sun Jinqi Xiao Bo Yuan Wen Xiao Junjie Hu Jiawei Zhao Beidi Chen Anima Anandkumar 59 1 0 18 Feb 2025
APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs Yuxiang Huang Mingye Li Xu Han Chaojun Xiao Weilin Zhao Sun Ao Hao Zhou Jie Zhou Zhiyuan Liu Maosong Sun 42 0 0 17 Feb 2025
KernelBench: Can LLMs Write Efficient GPU Kernels? Anne Ouyang Simon Guo Simran Arora Alex L. Zhang William Hu Christopher Ré Azalia Mirhoseini ALM 38 1 0 14 Feb 2025
QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache Rishabh Tiwari Haocheng Xi Aditya Tomar Coleman Hooper Sehoon Kim Maxwell Horton Mahyar Najibi Michael W. Mahoney K. K. Amir Gholami MQ 43 1 0 05 Feb 2025
Adaptive Self-improvement LLM Agentic System for ML Library Development Genghan Zhang Weixin Liang Olivia Hsu K. Olukotun 63 0 0 04 Feb 2025
Longer Attention Span: Increasing Transformer Context Length with Sparse Graph Processing Techniques Nathaniel Tomczak Sanmukh Kuppannagari 89 0 0 31 Jan 2025
PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantization Mengzhao Chen Yi Liu Jiahao Wang Yi Bin Wenqi Shao Ping Luo MQ 61 2 0 28 Jan 2025
MPCache: MPC-Friendly KV Cache Eviction for Efficient Private Large Language Model Inference Wenxuan Zeng Ye Dong Jinjin Zhou Junming Ma Jin Tan Runsheng Wang Meng Li 47 0 0 12 Jan 2025
FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving Zihao Ye Lequn Chen Ruihang Lai Wuwei Lin Yineng Zhang ... Tianqi Chen Baris Kasikci Vinod Grover Arvind Krishnamurthy Luis Ceze 65 19 0 02 Jan 2025
Tackling the Dynamicity in a Production LLM Serving System with SOTA Optimizations via Hybrid Prefill/Decode/Verify Scheduling on Efficient Meta-kernels Mingcong Song Xinru Tang Fengfan Hou Jing Li Wei Wei ... Hongjie Si D. Jiang Shouyi Yin Yang Hu Guoping Long 36 1 0 24 Dec 2024
Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference Benjamin Warner Antoine Chaffin Benjamin Clavié Orion Weller Oskar Hallström ... Tom Aarsen Nathan Cooper Griffin Adams Jeremy Howard Iacopo Poli 88 72 0 18 Dec 2024
HadaCore: Tensor Core Accelerated Hadamard Transform Kernel Krish Agarwal Rishi Astra Adnan Hoque M. Srivatsa R. Ganti Less Wright Sijia Chen 84 2 0 12 Dec 2024
Flex Attention: A Programming Model for Generating Optimized Attention Kernels Juechu Dong Boyuan Feng Driss Guessous Yanbo Liang Horace He 61 8 0 07 Dec 2024
Marconi: Prefix Caching for the Era of Hybrid LLMs Rui Pan Zhuang Wang Zhen Jia Can Karakus Luca Zancato Tri Dao Ravi Netravali Yida Wang 87 4 0 28 Nov 2024
Reassessing Layer Pruning in LLMs: New Insights and Methods Yao Lu Hao Cheng Yujie Fang Zeyu Wang Jiaheng Wei Dongwei Xu Qi Xuan Xiaoniu Yang Zhaowei Zhu 61 0 0 23 Nov 2024
MAS-Attention: Memory-Aware Stream Processing for Attention Acceleration on Resource-Constrained Edge Devices Mohammadali Shakerdargah Shan Lu Chao Gao Di Niu 70 0 0 20 Nov 2024
SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization Jintao Zhang Haofeng Huang Pengle Zhang Jia wei Jun-Jie Zhu Jianfei Chen VLM MQ 56 2 0 17 Nov 2024
Efficient Adaptive Optimization via Subset-Norm and Subspace-Momentum: Fast, Memory-Reduced Training with Convergence Guarantees T. Nguyen Huy Le Nguyen ODL 28 0 0 11 Nov 2024
Retentive Neural Quantum States: Efficient Ansätze for Ab Initio Quantum Chemistry Oliver Knitter Dan Zhao J. Stokes M. Ganahl Stefan Leichenauer S. Veerapaneni 34 1 0 06 Nov 2024
Context Parallelism for Scalable Million-Token Inference Amy Yang Jingyi Yang Aya Ibrahim Xinfeng Xie Bangsheng Tang Grigory Sizov Jeremy Reizenstein Jongsoo Park Jianyu Huang MoE LRM 57 5 0 04 Nov 2024