Emergence of a High-Dimensional Abstraction Phase in Language Transformers

Emergence of a High-Dimensional Abstraction Phase in Language Transformers

24 May 2024

Corentin Kervadec

Papers citing "Emergence of a High-Dimensional Abstraction Phase in Language Transformers"

13 / 13 papers shown

Title
Text-Speech Language Models with Improved Cross-Modal Transfer by Aligning Abstraction Levels Santiago Cuervo Adel Moumen Yanis Labrak Sameer Khurana Antoine Laurent Mickael Rouvier R. Marxer 55 1 0 08 Mar 2025
Prediction hubs are context-informed frequent tokens in LLMs Beatrix M. G. Nielsen Iuri Macocco Marco Baroni 93 0 0 17 Feb 2025
The Geometry of Tokens in Internal Representations of Large Language Models Karthik Viswanathan Yuri Gardinazzi Giada Panerai Alberto Cazzaniga Matteo Biagetti AIFin 60 1 0 17 Jan 2025
Emergent effects of scaling on the functional hierarchies within large language models Paul C. Bogdan 55 0 0 13 Jan 2025
Understanding Variational Autoencoders with Intrinsic Dimension and Information Imbalance Charles Camboulin Diego Doimo Aldo Glielmo DRL 43 0 0 04 Nov 2024
Unsupervised detection of semantic correlations in big data Santiago Acevedo Alex Rodriguez A. Laio 41 1 0 04 Nov 2024
Geometric Signatures of Compositionality Across a Language Model's Lifetime Jin Hwa Lee Thomas Jiralerspong Lei Yu Yoshua Bengio Emily Cheng CoGe 58 1 0 02 Oct 2024
Evidence from fMRI Supports a Two-Phase Abstraction Process in Language Models Emily Cheng Richard Antonello 45 1 0 09 Sep 2024
The representation landscape of few-shot learning and fine-tuning in large language models Diego Doimo Alessandro Serra A. Ansuini Alberto Cazzaniga 69 3 0 05 Sep 2024
Dissecting Recall of Factual Associations in Auto-Regressive Language Models Mor Geva Jasmijn Bastings Katja Filippova Amir Globerson KELM 174 152 0 28 Apr 2023
The Intrinsic Dimension of Images and Its Impact on Learning Phillip E. Pope Chen Zhu Ahmed Abdelkader Micah Goldblum Tom Goldstein 159 206 0 18 Apr 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 214 1,508 0 31 Dec 2020
What you can cram into a single vector: Probing sentence embeddings for linguistic properties Alexis Conneau Germán Kruszewski Guillaume Lample Loïc Barrault Marco Baroni 189 824 0 03 May 2018