FNet: Mixing Tokens with Fourier Transforms

9 May 2021

James Lee-Thorp

Joshua Ainslie

Ilya Eckstein

Santiago Ontanon

ArXiv PDF HTML

Papers citing "FNet: Mixing Tokens with Fourier Transforms"

50 / 251 papers shown

Title
Non-Stationary Time Series Forecasting Based on Fourier Analysis and Cross Attention Mechanism Yuqi Xiong Yang Wen AI4TS 31 0 0 11 May 2025
BiGSCoder: State Space Model for Code Understanding Shweta Verma Abhinav Anand Mira Mezini Mamba 46 0 0 02 May 2025
FreqKV: Frequency Domain Key-Value Compression for Efficient Context Window Extension Jushi Kai Boyi Zeng Yuhui Wang Haoli Bai Bo Jiang Bo Jiang Zhouhan Lin 42 0 0 01 May 2025
FIM: Frequency-Aware Multi-View Interest Modeling for Local-Life Service Recommendation Guoquan Wang Qiang Luo Weisong Hu Pengfei Yao Wencong Zeng Guorui Zhou Kun Gai AI4TS 56 0 0 23 Apr 2025
zkVC: Fast Zero-Knowledge Proof for Private and Verifiable Computing Yancheng Zhang Mengxin Zheng Xun Chen Jingtong Hu W. Shi Lei Ju Yan Solihin Qian Lou 51 0 0 16 Apr 2025
Learnable Multi-Scale Wavelet Transformer: A Novel Alternative to Self-Attention Andrew Kiruluta Priscilla Burity Samantha Williams 27 3 0 08 Apr 2025
Spectral-Adaptive Modulation Networks for Visual Perception Guhnoo Yun J. Yoo Kijung Kim Jeongho Lee Paul Hongsuck Seo Dong Hwan Kim 42 0 0 31 Mar 2025
NdLinear Is All You Need for Representation Learning Alex Reneau Jerry Yao-Chieh Hu Zhongfang Zhuang Ting-Chun Liu HAI 44 0 0 21 Mar 2025
Changing Base Without Losing Pace: A GPU-Efficient Alternative to MatMul in DNNs Nir Ailon Akhiad Bercovich Omri Weinstein 57 0 0 15 Mar 2025
Brain Effective Connectivity Estimation via Fourier Spatiotemporal Attention Wen Xiong Jinduo Liu Junzhong Ji Fenglong Ma 57 0 0 14 Mar 2025
FourierSR: A Fourier Token-based Plugin for Efficient Image Super-Resolution Wenjie Li Heng Guo Yuefeng Hou Zhanyu Ma 36 0 0 13 Mar 2025
Detection Avoidance Techniques for Large Language Models Sinclair Schneider Florian Steuber João A. G. Schneider Gabi Dreo Rodosek DeLMO 83 0 0 10 Mar 2025
FourierNAT: A Fourier-Mixing-Based Non-Autoregressive Transformer for Parallel Sequence Generation Andrew Kiruluta Eric Lundy Andreas Lemos AI4TS 47 0 0 04 Mar 2025
FANformer: Improving Large Language Models Through Effective Periodicity Modeling Yihong Dong Bernard Ghanem Xue Jiang Yongding Tao Kechi Zhang ... Huanyu Liu Jiazheng Ding Jia Li Jinliang Deng Hong Mei AI4TS 41 0 0 28 Feb 2025
FwNet-ECA: A Classification Model Enhancing Window Attention with Global Receptive Fields via Fourier Filtering Operations Shengtian Mian Ya Wang Nannan Gu Yuping Wang Xiaoqing Li 71 0 0 25 Feb 2025
Vim-F: Visual State Space Model Benefiting from Learning in the Frequency Domain Juntao Zhang Kun Bian Peng Cheng You Zhou Jianning Liu Wenbo An Jun Zhou Kun Shao Mamba 55 2 0 08 Jan 2025
Temporal Action Localization with Cross Layer Task Decoupling and Refinement Qiang Li Di Liu Jun Kong Sen Li Hui Xu Jianzhong Wang 85 0 0 12 Dec 2024
Enhancing Parameter-Efficient Fine-Tuning of Vision Transformers through Frequency-Based Adaptation S. Ly Hien Nguyen 79 1 0 28 Nov 2024
Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction Xiang Xu Hao Wang Wei Guo L. Zhang Wanshan Yang R. Yu Y. Liu Defu Lian Enhong Chen 40 5 0 22 Nov 2024
Visual Fourier Prompt Tuning Runjia Zeng Cheng Han Qifan Wang Chunshu Wu Tong Geng Lifu Huang Ying Nian Wu Dongfang Liu VPVLM VLM 58 6 0 02 Nov 2024
NIMBA: Towards Robust and Principled Processing of Point Clouds With SSMs Nursena Köprücü Destiny Okpekpe Antonio Orvieto Mamba 44 1 0 31 Oct 2024
Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift Yanru Sun Zongxia Xie Emadeldeen Eldele Dongyue Chen Q. Hu Min-man Wu AI4TS 28 1 0 13 Oct 2024
Neural Fourier Modelling: A Highly Compact Approach to Time-Series Analysis Minjung Kim Yusuke Hioka Michael Witbrock AI4TS 32 0 0 07 Oct 2024
FAN: Fourier Analysis Networks Yihong Dong Ge Li Yongding Tao Xue Jiang Kechi Zhang Jia Li Jing Su Jing Su Jun Zhang Jingjing Xu AI4TS 21 4 0 03 Oct 2024
MMFNet: Multi-Scale Frequency Masking Neural Network for Multivariate Time Series Forecasting Aitian Ma Dongsheng Luo Mo Sha AI4TS 28 0 0 02 Oct 2024
Expanding Expressivity in Transformer Models with MöbiusAttention Anna-Maria Halacheva M. Nayyeri Steffen Staab 27 1 0 08 Sep 2024
Wavelet GPT: Wavelet Inspired Large Language Models Prateek Verma AI4TS 23 0 0 04 Sep 2024
Shifted Window Fourier Transform And Retention For Image Captioning J. Hu Roberto Cavicchioli Alessandro Capotondi VLM 42 0 0 25 Aug 2024
Hypergraph Learning based Recommender System for Anomaly Detection, Control and Optimization Sakhinana Sagar Srinivas Rajat Kumar Sarkar Venkataramana Runkana AI4TS 29 2 0 21 Aug 2024
Nonlocal Attention Operator: Materializing Hidden Knowledge Towards Interpretable Physics Discovery Yue Yu Ning Liu Fei Lu Tian Gao S. Jafarzadeh Stewart Silling AI4CE 51 7 0 14 Aug 2024
AFIDAF: Alternating Fourier and Image Domain Adaptive Filters as an Efficient Alternative to Attention in ViTs Yunling Zheng Zeyi Xu Fanghui Xue Biao Yang Jiancheng Lyu Shuai Zhang Y. Qi Jack Xin 53 0 0 16 Jul 2024
Repurformer: Transformers for Repurposing-Aware Molecule Generation Changhun Lee Gyumin Lee 52 0 0 16 Jul 2024
Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers Sukjun Hwang Aakash Lahoti Tri Dao Albert Gu Mamba 62 12 0 13 Jul 2024
On the Anatomy of Attention Nikhil Khatri Tuomas Laakkonen Jonathon Liu Vincent Wang-Ma'scianica 3DV 48 1 0 02 Jul 2024
Multi-State-Action Tokenisation in Decision Transformers for Multi-Discrete Action Spaces Perusha Moodley Pramod S. Kaushik Dhillu Thambi Mark Trovinger Praveen Paruchuri Xia Hong Benjamin Rosman 54 0 0 01 Jul 2024
Exploring the Capability of Mamba in Speech Applications Koichi Miyazaki Yoshiki Masuyama Masato Murata Mamba 40 12 0 24 Jun 2024
GIM: A Million-scale Benchmark for Generative Image Manipulation Detection and Localization Y. Chen X. Huang Quan Zhang Wei Li Mingjian Zhu ... Hanting Chen Hailin Hu J. Yang Wei Liu Jie Hu EGVM 61 1 0 24 Jun 2024
Building on Efficient Foundations: Effectively Training LLMs with Structured Feedforward Layers Xiuying Wei Skander Moalla Razvan Pascanu Çağlar Gülçehre 32 0 0 24 Jun 2024
Towards Signal Processing In Large Language Models Prateek Verma Mert Pilanci 42 3 0 10 Jun 2024
MLIP: Efficient Multi-Perspective Language-Image Pretraining with Exhaustive Data Utilization Yu Zhang Qi Zhang Zixuan Gong Yiwei Shi Yepeng Liu ... Ke Liu Kun Yi Wei Fan Liang Hu Changwei Wang CLIP VLM 61 3 0 03 Jun 2024
LOLAMEME: Logic, Language, Memory, Mechanistic Framework Jay Desai Xiaobo Guo Srinivasan H. Sengamedu 21 0 0 31 May 2024
Fourier Controller Networks for Real-Time Decision-Making in Embodied Learning Hengkai Tan Songming Liu Kai Ma Chengyang Ying Xingxing Zhang Hang Su Jun Zhu 42 2 0 30 May 2024
MGCP: A Multi-Grained Correlation based Prediction Network for Multivariate Time Series Zhicheng Chen Xi Xiao Ke-Li Xu Zhong Zhang Yu Rong Qing Li Guojun Gan Zhiqiang Xu Peilin Zhao AI4TS 46 0 0 30 May 2024
SFANet: Spatial-Frequency Attention Network for Weather Forecasting Jiaze Wang Hao Chen Hongcan Xu Jinpeng Li Bo-Lan Wang Kun Shao Furui Liu Huaxi Chen Guangyong Chen Pheng-Ann Heng 64 0 0 29 May 2024
SelfCP: Compressing Over-Limit Prompt via the Frozen Large Language Model Itself Jun Gao Ziqiang Cao Wenjie Li 27 4 0 27 May 2024
SWAT: Scalable and Efficient Window Attention-based Transformers Acceleration on FPGAs Zhenyu Bai Pranav Dangi Huize Li Tulika Mitra 29 5 0 27 May 2024
Activator: GLU Activation Function as the Core Component of a Vision Transformer Abdullah Nazhat Abdullah Tarkan Aydin ViT 43 0 0 24 May 2024
TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting Shiyu Wang Haixu Wu X. Shi Tengge Hu Huakun Luo Lintao Ma James Y. Zhang Jun-ping Zhou AI4TS 76 102 0 23 May 2024
GestFormer: Multiscale Wavelet Pooling Transformer Network for Dynamic Hand Gesture Recognition Mallika Garg Debashis Ghosh P. M. Pradhan SLR ViT 44 2 0 18 May 2024
Enhancing Maritime Trajectory Forecasting via H3 Index and Causal Language Modelling (CLM) Nicolas Drapier Aladine Chetouani A. Chateigner 35 2 0 15 May 2024