Length Generalization in Arithmetic Transformers

Length Generalization in Arithmetic Transformers

27 June 2023

Stéphane dÁscoli

Carles Domingo-Enrich

Franccois Charton

Papers citing "Length Generalization in Arithmetic Transformers"

11 / 11 papers shown

Title
Distributional Scaling Laws for Emergent Capabilities Rosie Zhao Tian Qin David Alvarez-Melis Sham Kakade Naomi Saphra LRM 37 0 0 24 Feb 2025
Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges Nayoung Lee Ziyang Cai Avi Schwarzschild Kangwook Lee Dimitris Papailiopoulos ReLM VLM LRM AI4CE 73 4 0 03 Feb 2025
Mathematical Language Models: A Survey W. Liu Hanglei Hu Jie Zhou Yuyang Ding Junsong Li ... Mengliang He Qin Chen Bo Jiang Aimin Zhou Liang He LRM 79 12 0 03 Jan 2025
Mixture of Parrots: Experts improve memorization more than reasoning Samy Jelassi Clara Mohri David Brandfonbrener Alex Gu Nikhil Vyas Nikhil Anand David Alvarez-Melis Yuanzhi Li Sham Kakade Eran Malach MoE 28 4 0 24 Oct 2024
Carrying over algorithm in transformers J. Kruthoff 19 0 0 15 Jan 2024
Adaptivity and Modularity for Efficient Generalization Over Task Complexity Samira Abnar Omid Saremi Laurent Dinh Shantel Wilson Miguel Angel Bautista ... Vimal Thilak Etai Littwin Jiatao Gu Josh Susskind Samy Bengio 27 5 0 13 Oct 2023
GPT Can Solve Mathematical Problems Without a Calculator Z. Yang Ming Ding Qingsong Lv Zhihuan Jiang Zehai He Yuyi Guo Jinfeng Bai Jie Tang RALM LRM 26 52 0 06 Sep 2023
It Ain't That Bad: Understanding the Mysterious Performance Drop in OOD Generalization for Generative Transformer Models Xingcheng Xu Zihao Pan Haipeng Zhang Yanqing Yang LRM 13 2 0 16 Aug 2023
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,448 0 28 Jan 2022
SHAPE: Shifted Absolute Position Embedding for Transformers Shun Kiyono Sosuke Kobayashi Jun Suzuki Kentaro Inui 225 45 0 13 Sep 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 245 695 0 27 Aug 2021