Attention Approximates Sparse Distributed Memory

10 November 2021

Papers citing "Attention Approximates Sparse Distributed Memory"

25 / 25 papers shown

Title
Hyperspherical Energy Transformer with Recurrent Depth Yunzhe Hu Difan Zou Dong Xu 41 0 0 17 Feb 2025
On Design Choices in Similarity-Preserving Sparse Randomized Embeddings Denis Kleyko D. Rachkovskij 45 0 0 28 Jan 2025
An In-depth Investigation of Sparse Rate Reduction in Transformer-like Models Yunzhe Hu Difan Zou Dong Xu 71 1 0 26 Nov 2024
Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers Yibo Jiang Goutham Rajendran Pradeep Ravikumar Bryon Aragam CLL KELM 29 6 0 26 Jun 2024
Manifold Learning via Memory and Context Xin Li 18 0 0 17 May 2024
Associative Memories in the Feature Space Tommaso Salvatori Beren Millidge Yuhang Song Rafal Bogacz Thomas Lukasiewicz 27 1 0 16 Feb 2024
Bridging Associative Memory and Probabilistic Modeling Rylan Schaeffer Nika Zahedi Mikail Khona Dhruv Pai Sang T. Truong ... Sarthak Chandra Andres Carranza Ila Rani Fiete Andrey Gromov Oluwasanmi Koyejo DiffM 43 4 0 15 Feb 2024
Assessing Knowledge Editing in Language Models via Relation Perspective Yifan Wei Xiaoyan Yu Huanhuan Ma Fangyu Lei Yixuan Weng Ran Song Kang Liu KELM 28 15 0 15 Nov 2023
Large Language Models are In-context Teachers for Knowledge Reasoning Jiachen Zhao Zonghai Yao Zhichao Yang Hong-ye Yu ReLM LRM 32 1 0 12 Nov 2023
In-Context Exemplars as Clues to Retrieving from Large Associative Memory Jiachen Zhao 11 7 0 06 Nov 2023
Hopfield-Enhanced Deep Neural Networks for Artifact-Resilient Brain State Decoding Arnau Marin-Llobet Arnau Manasanch M. Sanchez-Vives 15 3 0 06 Nov 2023
Inverse distance weighting attention Calvin McCarter 21 1 0 28 Oct 2023
Birth of a Transformer: A Memory Viewpoint A. Bietti Vivien A. Cabannes Diane Bouchacourt Hervé Jégou Léon Bottou 21 81 0 01 Jun 2023
A Framework for Analyzing Cross-correlators using Price's Theorem and Piecewise-Linear Decomposition Zhili Xiao S. Chakrabartty 9 1 0 18 Apr 2023
On the Relationship Between Variational Inference and Auto-Associative Memory Louis Annabi Alexandre Pitti M. Quoy BDL 22 5 0 14 Oct 2022
Decoupled Context Processing for Context Augmented Language Modeling Zonglin Li Ruiqi Guo Surinder Kumar RALM KELM 14 22 0 11 Oct 2022
Memory-Augmented Graph Neural Networks: A Brain-Inspired Review Guixiang Ma Vy A. Vo Ted Willke Nesreen Ahmed 24 1 0 22 Sep 2022
Kernel Memory Networks: A Unifying Framework for Memory Modeling Georgios Iatropoulos Johanni Brea W. Gerstner 18 9 0 19 Aug 2022
BayesPCN: A Continually Learnable Predictive Coding Associative Memory Jason Yoo F. Wood KELM 89 9 0 20 May 2022
Should attention be all we need? The epistemic and ethical implications of unification in machine learning N. Fishman Leif Hancox-Li 25 10 0 09 May 2022
Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models Beren Millidge Tommaso Salvatori Yuhang Song Thomas Lukasiewicz Rafal Bogacz VLM 14 52 0 09 Feb 2022
O-ViT: Orthogonal Vision Transformer Yanhong Fei Yingjie Liu Xian Wei Mingsong Chen ViT 13 7 0 28 Jan 2022
A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges Denis Kleyko D. Rachkovskij Evgeny Osipov A. Rahim 16 126 0 12 Nov 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 267 1,812 0 14 Dec 2020
Building machines that adapt and compute like brains Brenden Lake J. Tenenbaum AI4CE FedML NAI AILaw 245 890 0 11 Nov 2017