Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models

4 April 2024

Jerry Yao-Chieh Hu

Papers citing "Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models"

28 / 28 papers shown

Title
Fast and Low-Cost Genomic Foundation Models via Outlier Removal Haozheng Luo Chenghao Qiu Maojiang Su Zhihan Zhou Zoe Mehta Guo Ye Jerry Yao-Chieh Hu Han Liu AAML 53 0 0 01 May 2025
Theoretical Foundation of Flow-Based Time Series Generation: Provable Approximation, Generalization, and Efficiency Jiangxuan Long Zhao-quan Song Chiwun Yang AI4TS 65 0 0 18 Mar 2025
Theoretical Guarantees for High Order Trajectory Refinement in Generative Flows Chengyue Gong Xiaoyu Li Yingyu Liang Jiangxuan Long Zhenmei Shi Zhao-quan Song Yu Tian 45 3 0 12 Mar 2025
Video Latent Flow Matching: Optimal Polynomial Projections for Video Interpolation and Extrapolation Yang Cao Zhao-quan Song Chiwun Yang VGen 41 2 0 01 Feb 2025
Fast Gradient Computation for RoPE Attention in Almost Linear Time Yifang Chen Jiayan Huo Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song 48 11 0 03 Jan 2025
$Theoretical Constraints on the Expressive Power of $\mathsf{RoPE}$-based Tensor Attention Transformers$ Theoretical Constraints on the Expressive Power of $\mathsf{RoPE}$ -based Tensor Attention Transformers Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song Mingda Wan 43 8 0 23 Dec 2024
Dense Associative Memory Through the Lens of Random Features Benjamin Hoover Duen Horng Chau Hendrik Strobelt Parikshit Ram Dmitry Krotov BDL 33 5 0 31 Oct 2024
Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descent Bo Chen Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song 67 18 0 15 Oct 2024
HSR-Enhanced Sparse Attention Acceleration Bo Chen Yingyu Liang Zhizhou Sha Zhenmei Shi Zhao-quan Song 73 17 0 14 Oct 2024
Fine-grained Attention I/O Complexity: Comprehensive Analysis for Backward Passes Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song Yufa Zhou 42 15 0 12 Oct 2024
Modern Hopfield Networks meet Encoded Neural Representations -- Addressing Practical Considerations Satyananda Kashyap N. S. D'Souza Luyao Shi Ken C. L. Wong Hongzhi Wang T. Syeda-Mahmood 24 0 0 24 Sep 2024
Differentially Private Kernel Density Estimation Erzhi Liu Jerry Yao-Chieh Hu Alex Reneau Zhao Song Han Liu 47 3 0 03 Sep 2024
A Tighter Complexity Analysis of SparseGPT Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song 58 20 0 22 Aug 2024
Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers Yibo Jiang Goutham Rajendran Pradeep Ravikumar Bryon Aragam CLL KELM 21 6 0 26 Jun 2024
Decoupled Alignment for Robust Plug-and-Play Adaptation Haozheng Luo Jiahao Yu Wenxin Zhang Jialong Li Jerry Yao-Chieh Hu Xingyu Xing Han Liu 31 10 0 03 Jun 2024
Conv-CoA: Improving Open-domain Question Answering in Large Language Models via Conversational Chain-of-Action Zhenyu Pan Haozheng Luo Manling Li Han Liu LRM 30 10 0 28 May 2024
Beyond Scaling Laws: Understanding Transformer Performance with Associative Memory Xueyan Niu Bo Bai Lei Deng Wei Han 20 6 0 14 May 2024
BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model Chenwei Xu Yu-Chao Huang Jerry Yao-Chieh Hu Weijian Li Ammar Gilani H. Goan Han Liu 32 19 0 04 Apr 2024
Outlier-Efficient Hopfield Layers for Large Transformer-Based Models Jerry Yao-Chieh Hu Pei-Hsuan Chang Haozheng Luo Hong-Yu Chen Weijian Li Wei-Po Wang Han Liu 28 25 0 04 Apr 2024
Learning Associative Memories with Gradient Descent Vivien A. Cabannes Berfin Simsek A. Bietti 27 3 0 28 Feb 2024
Massive Activations in Large Language Models Mingjie Sun Xinlei Chen J. Zico Kolter Zhuang Liu 57 64 0 27 Feb 2024
Bridging Associative Memory and Probabilistic Modeling Rylan Schaeffer Nika Zahedi Mikail Khona Dhruv Pai Sang T. Truong ... Sarthak Chandra Andres Carranza Ila Rani Fiete Andrey Gromov Oluwasanmi Koyejo DiffM 35 2 0 15 Feb 2024
Differentially Private Attention Computation Yeqi Gao Zhao-quan Song Xin Yang 28 19 0 08 May 2023
The Closeness of In-Context Learning and Weight Shifting for Softmax Regression Shuai Li Zhao-quan Song Yu Xia Tong Yu Tianyi Zhou 15 32 0 26 Apr 2023
BayesPCN: A Continually Learnable Predictive Coding Associative Memory Jason Yoo F. Wood KELM 87 9 0 20 May 2022
CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP Andreas Fürst Elisabeth Rumetshofer Johannes Lehner Viet-Hung Tran Fei Tang ... David P. Kreil Michael K Kopp G. Klambauer Angela Bitto-Nemling Sepp Hochreiter VLM CLIP 185 101 0 21 Oct 2021
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting Haoyi Zhou Shanghang Zhang J. Peng Shuai Zhang Jianxin Li Hui Xiong Wan Zhang AI4TS 161 3,799 0 14 Dec 2020
Modern Hopfield Networks and Attention for Immune Repertoire Classification Michael Widrich Bernhard Schafl Hubert Ramsauer Milena Pavlović Lukas Gruber ... Johannes Brandstetter G. K. Sandve Victor Greiff Sepp Hochreiter G. Klambauer 171 117 0 16 Jul 2020