Understanding Transformer Memorization Recall Through Idioms

7 October 2022

Papers citing "Understanding Transformer Memorization Recall Through Idioms"

45 / 45 papers shown

Title
Invisible Strings: Revealing Latent Dancer-to-Dancer Interactions with Graph Neural Networks Luis Zerkowski Zixuan Wang I. Vidrin M. Pettee 35 0 0 04 Mar 2025
CoME: An Unlearning-based Approach to Conflict-free Model Editing Dahyun Jung Jaehyung Seo Jaewook Lee Chanjun Park Heuiseok Lim MU KELM 47 0 0 20 Feb 2025
Fact Recall, Heuristics or Pure Guesswork? Precise Interpretations of Language Models for Fact Completion Denitsa Saynova Lovisa Hagström Moa Johansson Richard Johansson Marco Kuhlmann HILM 32 0 0 18 Oct 2024
AERO: Softmax-Only LLMs for Efficient Private Inference N. Jha Brandon Reagen 20 1 0 16 Oct 2024
Undesirable Memorization in Large Language Models: A Survey Ali Satvaty Suzan Verberne Fatih Turkmen ELM PILM 67 7 0 03 Oct 2024
Geometric Signatures of Compositionality Across a Language Model's Lifetime Jin Hwa Lee Thomas Jiralerspong Lei Yu Yoshua Bengio Emily Cheng CoGe 82 0 0 02 Oct 2024
A Mechanistic Interpretation of Syllogistic Reasoning in Auto-Regressive Language Models Geonhee Kim Marco Valentino André Freitas LRM AI4CE 22 7 0 16 Aug 2024
Generalisation First, Memorisation Second? Memorisation Localisation for Natural Language Classification Tasks Verna Dankers Ivan Titov 29 5 0 09 Aug 2024
Understanding Memorisation in LLMs: Dynamics, Influencing Factors, and Implications Till Speicher Mohammad Aflah Khan Qinyuan Wu Vedant Nanda Soumi Das Bishwamittra Ghosh Krishna P. Gummadi Evimaria Terzi 33 3 0 27 Jul 2024
Demystifying Verbatim Memorization in Large Language Models Jing Huang Diyi Yang Christopher Potts ELM PILM MU 45 19 0 25 Jul 2024
From Loops to Oops: Fallback Behaviors of Language Models Under Uncertainty Maor Ivgi Ori Yoran Jonathan Berant Mor Geva HILM 44 8 0 08 Jul 2024
Improving LLM Abilities in Idiomatic Translation Sundesh Donthi Maximilian Spencer Om Patel Joon Doh Eid Rodan Kevin Zhu Sean O'Brien 23 3 0 03 Jul 2024
Unveiling the Spectrum of Data Contamination in Language Models: A Survey from Detection to Remediation Chunyuan Deng Yilun Zhao Yuzhao Heng Yitong Li Jiannan Cao Xiangru Tang Arman Cohan 27 13 0 20 Jun 2024
Natural Language Processing RELIES on Linguistics Juri Opitz Shira Wein Nathan Schneider AI4CE 42 7 0 09 May 2024
On Large Language Models' Hallucination with Regard to Known Facts Che Jiang Biqing Qi Xiangyu Hong Dayuan Fu Yang Cheng Fandong Meng Mo Yu Bowen Zhou Jie Zhou HILM LRM 31 17 0 29 Mar 2024
Localizing Paragraph Memorization in Language Models Niklas Stoehr Mitchell Gordon Chiyuan Zhang Owen Lewis MU 30 13 0 28 Mar 2024
Not All Similarities Are Created Equal: Leveraging Data-Driven Biases to Inform GenAI Copyright Disputes Uri Y. Hacohen Adi Haviv Shahar Sarfaty Bruria Friedman N. Elkin-Koren Roi Livni Amit H. Bermano AILaw 28 7 0 26 Mar 2024
ROME: Memorization Insights from Text, Logits and Representation Bo Li Qing Xia Zhao Lijie Wen 17 2 0 01 Mar 2024
Robust Guidance for Unsupervised Data Selection: Capturing Perplexing Named Entities for Domain-Specific Machine Translation Seunghyun Ji H. R. Sinulingga Darongsae Kwon 33 0 0 29 Feb 2024
The Hidden Space of Transformer Language Adapters Jesujoba Oluwadara Alabi Marius Mosbach Matan Eyal Dietrich Klakow Mor Geva 48 7 1 20 Feb 2024
Backward Lens: Projecting Language Model Gradients into the Vocabulary Space Shahar Katz Yonatan Belinkov Mor Geva Lior Wolf 41 9 1 20 Feb 2024
Semantics of Multiword Expressions in Transformer-Based Models: A Survey Filip Miletic Sabine Schulte im Walde 34 6 0 27 Jan 2024
A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia Giovanni Monea Maxime Peyrard Martin Josifoski Vishrav Chaudhary Jason Eisner Emre Kiciman Hamid Palangi Barun Patra Robert West KELM 47 12 0 04 Dec 2023
Investigating Data Contamination in Modern Benchmarks for Large Language Models Chunyuan Deng Yilun Zhao Xiangru Tang Mark B. Gerstein Arman Cohan AAML ELM 11 50 0 16 Nov 2023
Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine Translation Verna Dankers Ivan Titov Dieuwke Hupkes 33 5 0 09 Nov 2023
Future Lens: Anticipating Subsequent Tokens from a Single Hidden State Koyena Pal Jiuding Sun Andrew Yuan Byron C. Wallace David Bau 22 50 0 08 Nov 2023
Characterizing Mechanisms for Factual Recall in Language Models Qinan Yu Jack Merullo Ellie Pavlick KELM 35 23 0 24 Oct 2023
Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss Weighting Emmy Liu Aditi Chaudhary Graham Neubig 17 7 0 10 Oct 2023
Are Multilingual LLMs Culturally-Diverse Reasoners? An Investigation into Multicultural Proverbs and Sayings Chen Cecilia Liu Fajri Koto Timothy Baldwin Iryna Gurevych LRM 17 17 0 15 Sep 2023
Eva-KELLM: A New Benchmark for Evaluating Knowledge Editing of LLMs Suhang Wu Minlong Peng Yue Chen Jinsong Su Mingming Sun KELM 24 35 0 19 Aug 2023
Training Data Extraction From Pre-trained Language Models: A Survey Shotaro Ishihara 24 46 0 25 May 2023
VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformers Shahar Katz Yonatan Belinkov 29 26 0 22 May 2023
Editing Large Language Models: Problems, Methods, and Opportunities Yunzhi Yao Peng Wang Bo Tian Shuyang Cheng Zhoubo Li Shumin Deng Huajun Chen Ningyu Zhang KELM 30 275 0 22 May 2023
Dissecting Recall of Factual Associations in Auto-Regressive Language Models Mor Geva Jasmijn Bastings Katja Filippova Amir Globerson KELM 189 260 0 28 Apr 2023
Recognition, recall, and retention of few-shot memories in large language models A. Orhan LRM KELM CLL 19 3 0 30 Mar 2023
Analyzing And Editing Inner Mechanisms Of Backdoored Language Models Max Lamparth Anka Reuel KELM 22 10 0 24 Feb 2023
Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks Vikas Raunak Arul Menezes 22 13 0 24 Oct 2022
Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation Verna Dankers Christopher G. Lucas Ivan Titov 33 36 0 30 May 2022
NOPE: A Corpus of Naturally-Occurring Presuppositions in English Alicia Parrish Sebastian Schuster Alex Warstadt Omar Agha Soo-hwan Lee Zhuoye Zhao Sam Bowman Tal Linzen LRM 28 23 0 14 Sep 2021
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 234 588 0 14 Jul 2021
Memorisation versus Generalisation in Pre-trained Language Models Michael Tänzer Sebastian Ruder Marek Rei 84 50 0 16 Apr 2021
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 255 343 0 01 Feb 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 264 1,798 0 14 Dec 2020
When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning? Gavin Brown Mark Bun Vitaly Feldman Adam D. Smith Kunal Talwar 245 80 0 11 Dec 2020
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 396 2,576 0 03 Sep 2019