Title
DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories Jia Li Ge Li Yunfei Zhao Yongming Li Huanyu Liu ... Yihong Dong Zhi Jin Binhua Li Fei Huang Yongbin Li ALM 21 25 0 30 May 2024
Dataflow-Guided Retrieval Augmentation for Repository-Level Code Completion Wei Cheng Yuhan Wu Wei Hu 22 3 0 30 May 2024
RTL-Repo: A Benchmark for Evaluating LLMs on Large-Scale RTL Design Projects Ahmed Allam Mohamed Shalan 27 14 0 27 May 2024
Granite Code Models: A Family of Open Foundation Models for Code Intelligence Mayank Mishra Matt Stallone Gaoyuan Zhang Yikang Shen Aditya Prasad ... Amith Singhee Nirmit Desai David D. Cox Ruchir Puri Rameswar Panda AI4TS 46 51 0 07 May 2024
EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories Jia Li Ge Li Xuanming Zhang Yihong Dong Zhi Jin 32 31 0 31 Mar 2024
Iterative Refinement of Project-Level Code Context for Precise Code Generation with Compiler Feedback Zhangqian Bi Yao Wan Zheng Wang Hongyu Zhang Batu Guan Fangxin Lu Zili Zhang Yulei Sui Hai Jin Xuanhua Shi 29 13 0 25 Mar 2024
CodeS: Natural Language to Code Repository via Multi-Layer Sketch Daoguang Zan Ailun Yu Wei Liu Dong Chen Bo Shen ... Bei Guan Zhiguang Yang Yongji Wang Qianxiang Wang Li-zhen Cui 18 14 0 25 Mar 2024
Reasoning Runtime Behavior of a Program with LLM: How Far Are We? Junkai Chen Zhiyuan Pan Xing Hu Zhenhao Li Ge Li Xin Xia LRM 26 20 0 25 Mar 2024
Exploiting Semantic Reconstruction to Mitigate Hallucinations in Vision-Language Models Minchan Kim Minyeong Kim Junik Bae Suhwan Choi Sungkyung Kim Buru Chang VLM 17 3 0 24 Mar 2024
Repoformer: Selective Retrieval for Repository-Level Code Completion Di Wu W. Ahmad Dejiao Zhang M. K. Ramanathan Xiaofei Ma 25 9 0 15 Mar 2024
CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferences M. Weyssow Aton Kamanda H. Sahraoui ALM 59 30 0 14 Mar 2024
DevBench: A Comprehensive Benchmark for Software Development Bowen Li Wenhan Wu Ziwei Tang Lin Shi John Yang ... He Du Ping Yang Dahua Lin Chao Peng Kai Chen 85 10 0 13 Mar 2024
AutoDev: Automated AI-Driven Development Michele Tufano Anisha Agarwal Jinu Jang Roshanak Zilouchian Moghaddam Neel Sundaresan 28 14 0 13 Mar 2024
RepoHyper: Better Context Retrieval Is All You Need for Repository-Level Code Completion Huy N. Phan Hoang N. Phan Tien N. Nguyen Nghi D. Q. Bui 32 12 0 10 Mar 2024
Test-Driven Development for Code Generation N. Mathews Mei Nagappan 27 7 0 21 Feb 2024
Unsupervised Evaluation of Code LLMs with Round-Trip Correctness Miltiadis Allamanis Sheena Panthaplackel Pengcheng Yin ALM OffRL LRM 43 9 0 13 Feb 2024
DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence Daya Guo Qihao Zhu Dejian Yang Zhenda Xie Kai Dong ... Yu-Huan Wu Y. K. Li Fuli Luo Yingfei Xiong W. Liang ELM 39 624 0 25 Jan 2024
DevEval: Evaluating Code Generation in Practical Software Projects Jia Li Ge Li Yunfei Zhao Yongming Li Zhi Jin ... Xuanming Zhang Yihong Dong Yuqi Zhu Bin Gu Mengfei Yang ALM ELM 30 11 0 12 Jan 2024
Adapting Large Language Models for Education: Foundational Capabilities, Potentials, and Challenges Qingyao Li Lingyue Fu Weiming Zhang Xianyu Chen Jingwei Yu Wei Xia Weinan Zhang Ruiming Tang Yong Yu AI4Ed ELM 25 16 0 27 Dec 2023
Magicoder: Empowering Code Generation with OSS-Instruct Yuxiang Wei Zhe Wang Jiawei Liu Yifeng Ding Lingming Zhang SyDa 37 89 0 04 Dec 2023
Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair Yuxiang Wei Chun Xia Lingming Zhang KELM 22 89 0 01 Sep 2023
CodeGen2: Lessons for Training LLMs on Programming and Natural Languages Erik Nijkamp A. Ghobadzadeh Caiming Xiong Silvio Savarese Yingbo Zhou 141 163 0 03 May 2023
CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Context Yangruibo Ding Zijian Wang Wasi Uddin Ahmad M. K. Ramanathan Ramesh Nallapati Parminder Bhatia Dan Roth Bing Xiang 13 64 0 20 Dec 2022
Multi-lingual Evaluation of Code Generation Models Ben Athiwaratkun Sanjay Krishna Gouda Zijian Wang Xiaopeng Li Yuchen Tian ... Baishakhi Ray Parminder Bhatia Sudipta Sengupta Dan Roth Bing Xiang ELM 109 117 0 26 Oct 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
A Systematic Evaluation of Large Language Models of Code Frank F. Xu Uri Alon Graham Neubig Vincent J. Hellendoorn ELM ALM 196 624 0 26 Feb 2022
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation Yue Wang Weishi Wang Shafiq R. Joty S. Hoi 204 1,451 0 02 Sep 2021
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 237 588 0 14 Jul 2021
Measuring Coding Challenge Competence With APPS Dan Hendrycks Steven Basart Saurav Kadavath Mantas Mazeika Akul Arora ... Collin Burns Samir Puranik Horace He D. Song Jacob Steinhardt ELM AIMat ALM 194 614 0 20 May 2021
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation Shuai Lu Daya Guo Shuo Ren Junjie Huang Alexey Svyatkovskiy ... Nan Duan Neel Sundaresan Shao Kun Deng Shengyu Fu Shujie Liu ELM 190 853 0 09 Feb 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 4,424 0 23 Jan 2020