Memorisation versus Generalisation in Pre-trained Language Models

16 April 2021

Papers citing "Memorisation versus Generalisation in Pre-trained Language Models"

7 / 7 papers shown

Title
Reasoning Bias of Next Token Prediction Training Pengxiao Lin Zhongwang Zhang Zhi-Qin John Xu LRM 80 1 0 21 Feb 2025
How Do Large Language Models Acquire Factual Knowledge During Pretraining? Hoyeon Chang Jinho Park Seonghyeon Ye Sohee Yang Youngkyung Seo Du-Seong Chang Minjoon Seo KELM 23 30 0 17 Jun 2024
PreCog: Exploring the Relation between Memorization and Performance in Pre-trained Language Models Leonardo Ranaldi Elena Sofia Ruzzetti Fabio Massimo Zanzotto 17 6 0 08 May 2023
Koala: An Index for Quantifying Overlaps with Pre-training Corpora Thuy-Trang Vu Xuanli He Gholamreza Haffari Ehsan Shareghi CLL 8 12 0 26 Mar 2023
Understanding Transformer Memorization Recall Through Idioms Adi Haviv Ido Cohen Jacob Gidron R. Schuster Yoav Goldberg Mor Geva 6 48 0 07 Oct 2022
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 393 2,216 0 03 Sep 2019
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Z. Tu Kaiming He 261 10,106 0 16 Nov 2016