FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

27 May 2022

Papers citing "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness"

50 / 1,418 papers shown

Title
When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruning Junwei Luo Yingying Zhang X. J. Yang Kang Wu Qi Zhu Lei Liang Jingdong Chen Yansheng Li 57 0 0 10 Mar 2025
From Reusing to Forecasting: Accelerating Diffusion Models with TaylorSeers Jiacheng Liu Chang Zou Yuanhuiyi Lyu Junjie Chen Linfeng Zhang DiffM 54 0 0 10 Mar 2025
Queueing, Predictions, and LLMs: Challenges and Open Problems Michael Mitzenmacher Rana Shahout AI4TS LRM 33 1 0 10 Mar 2025
Slim attention: cut your context memory in half without loss of accuracy -- K-cache is all you need for MHA Nils Graef Andrew Wasielewski 24 1 0 07 Mar 2025
Shifting Long-Context LLMs Research from Input to Output Yuhao Wu Yushi Bai Zhiqing Hu Shangqing Tu Ming Shan Hee Juanzi Li Roy Ka-Wei Lee 57 0 0 06 Mar 2025
Universality of Layer-Level Entropy-Weighted Quantization Beyond Model Architecture and Size Alireza Behtash Marijan Fofonjka Ethan Baird Tyler Mauer Hossein Moghimifam David Stout Joel Dennison MQ 50 1 0 06 Mar 2025
L $^2$ M: Mutual Information Scaling Law for Long-Context Language Modeling Zhuo Chen Oriol Mayné i Comas Zhuotao Jin Di Luo Marin Soljacic 59 0 0 06 Mar 2025
LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLM S. Mohammed Irfan Kurpath Sahal Shaji Mullappilly Jean Lahoud Fahad A Khan Rao Muhammad Anwer Salman Khan Hisham Cholakkal AuLLM 57 0 0 06 Mar 2025
DIMSUM: Discourse in Mathematical Reasoning as a Supervision Module Krish Sharma Niyar R. Barman Nicholas M. Asher Akshay Chaturvedi LRM AIMat 64 6 0 06 Mar 2025
DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models Saeed Ranjbar Alvar Gursimran Singh Mohammad Akbari Yong Zhang VLM 71 0 0 04 Mar 2025
FlexInfer: Breaking Memory Constraint via Flexible and Efficient Offloading for On-Device LLM Inference Hongchao Du Shangyu Wu Arina Kharlamova Nan Guan Chun Jason Xue 49 1 0 04 Mar 2025
Optimizing open-domain question answering with graph-based retrieval augmented generation Joyce Cahoon Prerna Singh Nick Litombe Jonathan Larson Ha Trinh Yiwen Zhu A. Mueller Fotis Psallidas Carlo Curino 29 0 0 04 Mar 2025
Union of Experts: Adapting Hierarchical Routing to Equivalently Decomposed Transformer Yujiao Yang Jing Lian Linhui Li MoE 70 0 0 04 Mar 2025
Alchemist: Towards the Design of Efficient Online Continual Learning System Yuyang Huang Yuhan Liu Haryadi S. Gunawi Beibin Li Changho Hwang CLL OnRL 98 0 0 03 Mar 2025
Position: Ensuring mutual privacy is necessary for effective external evaluation of proprietary AI systems Ben Bucknall Robert F. Trager Michael A. Osborne 77 0 0 03 Mar 2025
Structural Deep Encoding for Table Question Answering Raphael Mouravieff Benjamin Piwowarski Sylvain Lamprier LMTD 41 0 0 03 Mar 2025
Neural ODE Transformers: Analyzing Internal Dynamics and Adaptive Fine-tuning Anh Tong Thanh Nguyen-Tang Dongeun Lee Duc Nguyen Toan M. Tran David Hall Cheongwoong Kang Jaesik Choi 33 0 0 03 Mar 2025
Attention Condensation via Sparsity Induced Regularized Training Eli Sason Darya Frolova Boris Nazarov Felix Goldberd 72 0 0 03 Mar 2025
Advancing MAPF towards the Real World: A Scalable Multi-Agent Realistic Testbed (SMART) Jingtian Yan Zhifei Li William Kang Yulun Zhang Stephen Smith Jiaoyang Li 33 0 0 03 Mar 2025
Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving Qihui Zhou Peiqi Yin Pengfei Zuo James Cheng CLL 35 1 0 01 Mar 2025
Flow Matching for Medical Image Synthesis: Bridging the Gap Between Speed and Quality M. Yazdani Yasamin Medghalchi Pooria Ashrafian I. Hacihaliloglu Dena Shahriari MedIm 24 0 0 01 Mar 2025
Training-free and Adaptive Sparse Attention for Efficient Long Video Generation Yifei Xia Suhan Ling Fangcheng Fu Y. Wang Huixia Li Xuefeng Xiao Bin Cui VGen 51 2 0 28 Feb 2025
ByteScale: Efficient Scaling of LLM Training with a 2048K Context Length on More Than 12,000 GPUs Hao Ge Junda Feng Qi Huang Fangcheng Fu Xiaonan Nie Lei Zuo Haibin Lin Bin Cui Xin Liu 34 2 0 28 Feb 2025
FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference Xunhao Lai Jianqiao Lu Yao Luo Yiyuan Ma Xun Zhou 63 5 0 28 Feb 2025
FANformer: Improving Large Language Models Through Effective Periodicity Modeling Yihong Dong G. Li Xue Jiang Yongding Tao Kechi Zhang ... Huanyu Liu Jiazheng Ding Jia Li Jinliang Deng Hong Mei AI4TS 36 0 0 28 Feb 2025
Implicit Search via Discrete Diffusion: A Study on Chess Jiacheng Ye Zhenyu Wu Jiahui Gao Zhiyong Wu Xin Jiang Z. Li Lingpeng Kong DiffM 43 2 0 27 Feb 2025
LiteASR: Efficient Automatic Speech Recognition with Low-Rank Approximation Keisuke Kamahori Jungo Kasai Noriyuki Kojima Baris Kasikci 27 0 0 27 Feb 2025
Training LLMs with MXFP4 Albert Tseng Tao Yu Youngsuk Park 29 1 0 27 Feb 2025
HDEE: Heterogeneous Domain Expert Ensemble Oğuzhan Ersoy Jari Kolehmainen Gabriel Passamani Andrade MoE 40 0 0 26 Feb 2025
Self-Adjust Softmax Chuanyang Zheng Yihang Gao Guoxuan Chen Han Shi Jing Xiong Xiaozhe Ren Chao Huang Xin Jiang Z. Li Yu-Hu Li 33 0 0 25 Feb 2025
MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference Zhongwei Wan H. Shen Xin Wang C. Liu Zheda Mai M. Zhang VLM 54 3 0 24 Feb 2025
Simpler Fast Vision Transformers with a Jumbo CLS Token A. Fuller Yousef Yassin Daniel G. Kyrollos Evan Shelhamer James R. Green 67 0 0 24 Feb 2025
MUSE: Mamba is Efficient Multi-scale Learner for Text-video Retrieval Haoran Tang Meng Cao Jinfa Huang Ruyang Liu Peng Jin Ge Li Xiaodan Liang Mamba 92 4 0 24 Feb 2025
AttentionEngine: A Versatile Framework for Efficient Attention Mechanisms on Diverse Hardware Platforms Feiyang Chen Yu Cheng Lei Wang Yuqing Xia Ziming Miao ... Fan Yang J. Xue Zhi Yang M. Yang H. Chen 71 1 0 24 Feb 2025
Unveiling Reasoning Thresholds in Language Models: Scaling, Fine-Tuning, and Interpretability through Attention Maps Yen-Che Hsiao Abhishek Dutta LRM ReLM ELM 54 0 0 24 Feb 2025
Vision-LSTM: xLSTM as Generic Vision Backbone Benedikt Alkin M. Beck Korbinian Poppel Sepp Hochreiter Johannes Brandstetter VLM 50 36 0 24 Feb 2025
Training a Generally Curious Agent Fahim Tajwar Yiding Jiang Abitha Thankaraj Sumaita Sadia Rahman J. Zico Kolter Jeff Schneider Ruslan Salakhutdinov 112 1 0 24 Feb 2025
DeepInteraction++: Multi-Modality Interaction for Autonomous Driving Zeyu Yang Nan Song Wei Li Xiatian Zhu L. Zhang Philip H. S. Torr 63 4 0 24 Feb 2025
GeoAggregator: An Efficient Transformer Model for Geo-Spatial Tabular Data Rui Deng Ziqi Li Mingshu Wang 28 0 0 24 Feb 2025
Selective Prompt Anchoring for Code Generation Yuan Tian Tianyi Zhang 77 3 0 24 Feb 2025
CORAL: Learning Consistent Representations across Multi-step Training with Lighter Speculative Drafter Yepeng Weng Dianwen Mei Huishi Qiu Xujie Chen Li Liu Jiang Tian Zhongchao Shi 42 0 0 24 Feb 2025
Surface Vision Mamba: Leveraging Bidirectional State Space Model for Efficient Spherical Manifold Representation Rongzhao He Weihao Zheng Leilei Zhao Ying Wang Dalin Zhu Dan Wu Bin Hu Mamba 84 0 0 21 Feb 2025
Neural Attention Search Difan Deng Marius Lindauer 85 0 0 21 Feb 2025
CoKV: Optimizing KV Cache Allocation via Cooperative Game Qiheng Sun Hongwei Zhang Haocheng Xia Jiayao Zhang Jinfei Liu Kui Ren VLM 37 0 0 21 Feb 2025
Compression Barriers for Autoregressive Transformers Themistoklis Haris Krzysztof Onak 35 1 0 21 Feb 2025
TurboFuzzLLM: Turbocharging Mutation-based Fuzzing for Effectively Jailbreaking Large Language Models in Practice Aman Goel Xian Carrie Wu Zhe Wang Dmitriy Bespalov Yanjun Qi 44 0 0 21 Feb 2025
CLIPPER: Compression enables long-context synthetic data generation Chau Minh Pham Yapei Chang Mohit Iyyer SyDa 72 1 0 21 Feb 2025
Autellix: An Efficient Serving Engine for LLM Agents as General Programs Michael Luo Xiaoxiang Shi Colin Cai Tianjun Zhang Justin Wong ... Chi Wang Yanping Huang Zhifeng Chen Joseph E. Gonzalez Ion Stoica 47 2 0 20 Feb 2025
Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs Longxu Dou Qian Liu Fan Zhou Changyu Chen Zili Wang ... Tianyu Pang Chao Du Xinyi Wan Wei Lu Min Lin 82 1 0 18 Feb 2025
Learning More Effective Representations for Dense Retrieval through Deliberate Thinking Before Search Yifan Ji Zhipeng Xu Zhenghao Liu Yukun Yan S. Yu Y. Li Zhiyuan Liu Yu Gu Ge Yu Maosong Sun RALM 51 0 0 18 Feb 2025