Title
CodeARC: Benchmarking Reasoning Capabilities of LLM Agents for Inductive Program Synthesis Anjiang Wei Tarun Suresh Jiannan Cao Naveen Kannan Yuheng Wu Kai Yan Thiago S. F. X. Teixeira Ke Wang Alex Aiken ELM LRM 34 0 0 29 Mar 2025
L0-Reasoning Bench: Evaluating Procedural Correctness in Language Models via Simple Program Execution Simeng Sun Cheng-Ping Hsieh Faisal Ladhak Erik Arakelyan Santiago Akle Serano Boris Ginsburg ReLM ELM LRM 47 0 0 28 Mar 2025
A Showdown of ChatGPT vs DeepSeek in Solving Programming Tasks Ronas Shakya Farhad Vadiee Mohammad Khalil ELM LRM 58 0 0 16 Mar 2025
RGD: Multi-LLM Based Agent Debugger via Refinement and Generation Guidance Haolin Jin Zechao Sun Huaming Chen LLMAG 43 2 0 02 Oct 2024
From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future Haolin Jin Linghan Huang Haipeng Cai Jun Yan Bo Li Huaming Chen 53 24 0 05 Aug 2024
Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models Jia Zheng Boxi Cao Zhengzhao Ma Ruotong Pan Hongyu Lin Yaojie Lu Xianpei Han Le Sun ALM 18 0 0 16 Jul 2024
Harnessing Business and Media Insights with Large Language Models Yujia Bao Ankit Parag Shah Neeru Narang Jonathan Rivers Rajeev Maksey ... Gyuhak Kim Dengpan Yin Don Hejna Mo Nomeli Wei Wei AIFin 27 2 0 02 Jun 2024
A Survey on Large Language Models for Code Generation Juyong Jiang Fan Wang Jiasi Shen Sungju Kim Sunghun Kim 35 74 0 01 Jun 2024
Test Code Generation for Telecom Software Systems using Two-Stage Generative Model Mohamad Nabeel Doumitrou Daniil Nimara Tahar Zanouda 24 2 0 14 Apr 2024
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code Naman Jain King Han Alex Gu Wen-Ding Li Fanjia Yan Tianjun Zhang Sida I. Wang Armando Solar-Lezama Koushik Sen Ion Stoica ELM 24 260 0 12 Mar 2024
Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models Martin Riddell Ansong Ni Arman Cohan ELM 21 14 0 06 Mar 2024
Grounding Data Science Code Generation with Input-Output Specifications Yeming Wen Pengcheng Yin Kensen Shi Henryk Michalewski Swarat Chaudhuri A. Polozov SyDa 16 10 0 12 Feb 2024
LLMs for Science: Usage for Code Generation and Data Analysis Mohamed Nejjar Luca Zacharias Fabian Stiehle Ingo Weber 11 12 0 28 Nov 2023
Binding Language Models in Symbolic Languages Zhoujun Cheng Tianbao Xie Peng Shi Chengzu Li Rahul Nadkarni ... Dragomir R. Radev Mari Ostendorf Luke Zettlemoyer Noah A. Smith Tao Yu LMTD 109 195 0 06 Oct 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 297 3,163 0 21 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation Shuai Lu Daya Guo Shuo Ren Junjie Huang Alexey Svyatkovskiy ... Nan Duan Neel Sundaresan Shao Kun Deng Shengyu Fu Shujie Liu ELM 186 853 0 09 Feb 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 236 1,508 0 31 Dec 2020
Task-Oriented Dialogue as Dataflow Synthesis Semantic Machines Jacob Andreas J. Bufe David Burkett Charles C. Chen ... Izabela Witoszko Jason Wolfe A. Wray Yuchen Zhang Alexander Zotov AIFin 180 151 0 24 Sep 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 3,054 0 23 Jan 2020