Code Pretraining Improves Entity Tracking Abilities of Language Models

31 May 2024

Papers citing "Code Pretraining Improves Entity Tracking Abilities of Language Models"

12 / 12 papers shown

Title
Rethinking Multilingual Continual Pretraining: Data Mixing for Adapting LLMs Across Languages and Resources Zihao Li Shaoxiong Ji Hengyu Luo Jörg Tiedemann CLL 101 0 0 05 Apr 2025
General Reasoning Requires Learning to Reason from the Get-go Seungwook Han Jyothish Pari Samuel J. Gershman Pulkit Agrawal LRM 123 0 0 26 Feb 2025
MET-Bench: Multimodal Entity Tracking for Evaluating the Limitations of Vision-Language and Reasoning Models Vanya Cohen Raymond J. Mooney 39 0 0 15 Feb 2025
Chain and Causal Attention for Efficient Entity Tracking Erwan Fagnou Paul Caillon Blaise Delattre Alexandre Allauzen 21 2 0 07 Oct 2024
EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models Shaoxiong Ji Zihao Li Indraneil Paul Jaakko Paavola Peiqin Lin ... Dayyán O'Brien Hengyu Luo Hinrich Schütze Jörg Tiedemann Barry Haddow CLL 35 3 0 26 Sep 2024
Representational Analysis of Binding in Language Models Qin Dai Benjamin Heinzerling Kentaro Inui 29 0 0 09 Sep 2024
How Does Code Pretraining Affect Language Model Task Performance? Jackson Petty Sjoerd van Steenkiste Tal Linzen 60 8 0 06 Sep 2024
To Code, or Not To Code? Exploring Impact of Code in Pre-training Viraat Aryabumi Yixuan Su Raymond Ma Adrien Morisot Ivan Zhang Acyr F. Locatelli Marzieh Fadaee A. Ustun Sara Hooker SyDa AI4CE 40 18 0 20 Aug 2024
A Notion of Complexity for Theory of Mind via Discrete World Models X. A. Huang Emanuele La Malfa Samuele Marro Andrea Asperti Anthony Cohn Michael Wooldridge 37 6 0 16 Jun 2024
Entity Tracking in Language Models Najoung Kim Sebastian Schuster 50 16 0 03 May 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 306 11,909 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 317 8,448 0 28 Jan 2022