Generalization on the Unseen, Logic Reasoning and Degree Curriculum

30 January 2023

Papers citing "Generalization on the Unseen, Logic Reasoning and Degree Curriculum"

41 / 41 papers shown

Title
Geometric Reasoning in the Embedding Space Jan Hůla David Mojžíšek Jiří Janeček David Herel Mikoláš Janota 34 0 0 02 Apr 2025
Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More Arvid Frydenlund LRM 48 0 0 13 Mar 2025
The Role of Sparsity for Length Generalization in Transformers Noah Golowich Samy Jelassi David Brandfonbrener Sham Kakade Eran Malach 37 0 0 24 Feb 2025
Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers Alireza Amiri Xinting Huang Mark Rofin Michael Hahn LRM 83 0 0 04 Feb 2025
Out-of-distribution generalization via composition: a lens through induction heads in Transformers Jiajun Song Zhuoyan Xu Yiqiao Zhong 71 4 0 31 Dec 2024
Selective Attention: Enhancing Transformer through Principled Context Control Xuechen Zhang Xiangyu Chang Mingchen Li A. Roy-Chowdhury J. Chen Samet Oymak 60 2 0 19 Nov 2024
Number Cookbook: Number Understanding of Language Models and How to Improve It Haotong Yang Yi Hu Shijia Kang Zhouchen Lin Muhan Zhang LRM 41 2 0 06 Nov 2024
Provable Length Generalization in Sequence Prediction via Spectral Filtering Annie Marsden Evan Dogariu Naman Agarwal Xinyi Chen Daniel Suo Elad Hazan 32 1 0 01 Nov 2024
Transformation-Invariant Learning and Theoretical Guarantees for OOD Generalization Omar Montasser Han Shao Emmanuel Abbe OOD 27 1 0 30 Oct 2024
Interchangeable Token Embeddings for Extendable Vocabulary and Alpha-Equivalence İlker Işık R. G. Cinbis Ebru Aydin Gol 21 0 0 22 Oct 2024
Low-Dimension-to-High-Dimension Generalization And Its Implications for Length Generalization Yang Chen Yitao Liang Zhouchen Lin 13 0 0 11 Oct 2024
Visual Scratchpads: Enabling Global Reasoning in Vision Aryo Lotfi Enrico Fini Samy Bengio Moin Nabi Emmanuel Abbe LRM 30 0 0 10 Oct 2024
Relating the Seemingly Unrelated: Principled Understanding of Generalization for Generative Models in Arithmetic Reasoning Tasks Xingcheng Xu Zibo Zhao Haipeng Zhang Yanqing Yang LRM 19 0 0 25 Jul 2024
Universal Length Generalization with Turing Programs Kaiying Hou David Brandfonbrener Sham Kakade Samy Jelassi Eran Malach 29 7 0 03 Jul 2024
VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbation Kun Qian Shunji Wan Claudia Tang Youzhi Wang Xuanming Zhang Maximillian Chen Zhou Yu AAML 35 8 0 25 Jun 2024
Transformers meet Neural Algorithmic Reasoners Wilfried Bounsi Borja Ibarz Andrew Dudzik Jessica B. Hamrick Larisa Markeeva Alex Vitvitskyi Razvan Pascanu Petar Veličković NAI AI4CE LRM 28 5 0 13 Jun 2024
On the Minimal Degree Bias in Generalization on the Unseen for non-Boolean Functions Denys Pushkin Raphael Berthier Emmanuel Abbe 21 0 0 10 Jun 2024
Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers Brian K Chen Tianyang Hu Hui Jin Hwee Kuan Lee Kenji Kawaguchi 32 0 0 05 Jun 2024
Feature contamination: Neural networks learn uncorrelated features and fail to generalize Tianren Zhang Chujie Zhao Guanyu Chen Yizhou Jiang Feng Chen OOD MLT OODD 59 2 0 05 Jun 2024
Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Tasks Mahdi Sabbaghi George Pappas Hamed Hassani Surbhi Goel 26 4 0 04 Jun 2024
Language Models Need Inductive Biases to Count Inductively Yingshan Chang Yonatan Bisk LRM 32 5 0 30 May 2024
When does compositional structure yield compositional generalization? A kernel theory Samuel Lippl Kim Stachenfeld NAI CoGe 58 5 0 26 May 2024
Theoretical Analysis of Weak-to-Strong Generalization Hunter Lang David Sontag Aravindan Vijayaraghavan 19 19 0 25 May 2024
Learning Invariant Causal Mechanism from Vision-Language Models Zeen Song Siyu Zhao Xingyu Zhang Jiangmeng Li Changwen Zheng Wenwen Qiang CML BDL VLM 24 0 0 24 May 2024
A Theory for Length Generalization in Learning to Reason Changnan Xiao Bing Liu LRM 23 8 0 31 Mar 2024
Transfer Learning Beyond Bounded Density Ratios Alkis Kalavasis Ilias Zadik Manolis Zampetakis 18 4 0 18 Mar 2024
Neural Redshift: Random Networks are not Random Functions Damien Teney A. Nicolicioiu Valentin Hartmann Ehsan Abbasnejad 86 18 0 04 Mar 2024
Why are Sensitive Functions Hard for Transformers? Michael Hahn Mark Rofin 20 22 0 15 Feb 2024
Transformers Can Achieve Length Generalization But Not Robustly Yongchao Zhou Uri Alon Xinyun Chen Xuezhi Wang Rishabh Agarwal Denny Zhou 27 36 0 14 Feb 2024
On Provable Length and Compositional Generalization Kartik Ahuja Amin Mansouri OODD 20 7 0 07 Feb 2024
Overcoming the Pitfalls of Vision-Language Model Finetuning for OOD Generalization Yuhang Zang Hanlin Goh Josh Susskind Chen Huang VLM 24 12 0 29 Jan 2024
When can transformers reason with abstract symbols? Enric Boix-Adserà Omid Saremi Emmanuel Abbe Samy Bengio Etai Littwin Josh Susskind LRM NAI 11 7 0 15 Oct 2023
Adaptivity and Modularity for Efficient Generalization Over Task Complexity Samira Abnar Omid Saremi Laurent Dinh Shantel Wilson Miguel Angel Bautista ... Vimal Thilak Etai Littwin Jiatao Gu Josh Susskind Samy Bengio 14 5 0 13 Oct 2023
Improving Length-Generalization in Transformers via Task Hinting Pranjal Awasthi Anupam Gupta 16 7 0 01 Oct 2023
Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputs Emmanuel Abbe Elisabetta Cornacchia Aryo Lotfi 8 11 0 29 Jun 2023
A Mathematical Model for Curriculum Learning for Parities Elisabetta Cornacchia Elchanan Mossel 12 10 0 31 Jan 2023
Towards Better Out-of-Distribution Generalization of Neural Algorithmic Reasoning Tasks Sadegh Mahdavi Kevin Swersky Thomas Kipf Milad Hashemi Christos Thrampoulidis Renjie Liao LRM OOD NAI 38 26 0 01 Nov 2022
Revisiting Neural Scaling Laws in Language and Vision Ibrahim M. Alabdulmohsin Behnam Neyshabur Xiaohua Zhai 148 101 0 13 Sep 2022
The CLRS Algorithmic Reasoning Benchmark Petar Velivcković Adria Puigdomenech Badia David Budden Razvan Pascanu Andrea Banino Mikhail Dashevskiy R. Hadsell Charles Blundell 154 86 0 31 May 2022
A Fine-Grained Analysis on Distribution Shift Olivia Wiles Sven Gowal Florian Stimberg Sylvestre-Alvise Rebuffi Ira Ktena Krishnamurthy Dvijotham A. Cemgil OOD 215 196 0 21 Oct 2021
Domain Adaptation: Learning Bounds and Algorithms Yishay Mansour M. Mohri Afshin Rostamizadeh 179 786 0 19 Feb 2009