Bridging Information-Theoretic and Geometric Compression in Language Models

20 October 2023

Papers citing "Bridging Information-Theoretic and Geometric Compression in Language Models"

19 / 19 papers shown

Title
Bridging the Dimensional Chasm: Uncover Layer-wise Dimensional Reduction in Transformers through Token Correlation Zhuo-Yang Song Zeyu Li Qing-Hong Cao Ming-xing Luo Hua Xing Zhu 28 0 0 28 Mar 2025
EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling Theodoros Kouzelis Ioannis Kakogeorgiou Spyros Gidaris N. Komodakis DRL 67 5 0 17 Feb 2025
Explaining Context Length Scaling and Bounds for Language Models Jingzhe Shi Qinwei Ma Hongyi Liu Hang Zhao Jeng-Neng Hwang Serge Belongie Lei Li LRM 73 2 0 03 Feb 2025
The Geometry of Tokens in Internal Representations of Large Language Models Karthik Viswanathan Yuri Gardinazzi Giada Panerai Alberto Cazzaniga Matteo Biagetti AIFin 88 4 0 17 Jan 2025
Unsupervised detection of semantic correlations in big data Santiago Acevedo Alex Rodriguez A. Laio 65 2 0 04 Nov 2024
ResiDual Transformer Alignment with Spectral Decomposition Lorenzo Basile Valentino Maiorca Luca Bortolussi Emanuele Rodolà Francesco Locatello 45 1 0 31 Oct 2024
Persistent Topological Features in Large Language Models Yuri Gardinazzi Giada Panerai Karthik Viswanathan A. Ansuini Alberto Cazzaniga Matteo Biagetti 39 2 0 14 Oct 2024
Geometric Signatures of Compositionality Across a Language Model's Lifetime Jin Hwa Lee Thomas Jiralerspong Lei Yu Yoshua Bengio Emily Cheng CoGe 82 0 0 02 Oct 2024
The representation landscape of few-shot learning and fine-tuning in large language models Diego Doimo Alessandro Serra A. Ansuini Alberto Cazzaniga 88 4 0 05 Sep 2024
Intrinsic Dimension Correlation: uncovering nonlinear connections in multimodal representations Lorenzo Basile Santiago Acevedo Luca Bortolussi Fabio Anselmi Alex Rodriguez 34 4 0 22 Jun 2024
Interpretability of Language Models via Task Spaces Lucas Weber Jaap Jumelet Elia Bruni Dieuwke Hupkes 27 4 0 10 Jun 2024
Emergence of a High-Dimensional Abstraction Phase in Language Transformers Emily Cheng Diego Doimo Corentin Kervadec Iuri Macocco Jade Yu A. Laio Marco Baroni 104 11 0 24 May 2024
The geometry of hidden representations of large transformer models L. Valeriani Diego Doimo F. Cuturello A. Laio A. Ansuini Alberto Cazzaniga MILM 19 48 0 01 Feb 2023
Intrinsic Dimensionality Estimation within Tight Localities: A Theoretical and Experimental Analysis Laurent Amsaleg Oussama Chelly Michael E. Houle Ken-ichi Kawarabayashi Miloš Radovanović Weeris Treeratanajaru 25 50 0 29 Sep 2022
The Intrinsic Dimension of Images and Its Impact on Learning Phillip E. Pope Chen Zhu Ahmed Abdelkader Micah Goldblum Tom Goldstein 180 256 0 18 Apr 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 245 1,977 0 31 Dec 2020
A Survey on Neural Network Interpretability Yu Zhang Peter Tiño A. Leonardis K. Tang FaML XAI 129 654 0 28 Dec 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018
Teaching Machines to Read and Comprehend Karl Moritz Hermann Tomás Kociský Edward Grefenstette L. Espeholt W. Kay Mustafa Suleyman Phil Blunsom 170 3,504 0 10 Jun 2015