RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems

5 June 2023

Papers citing "RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems"

50 / 108 papers shown

Title
Self-evolving Agents with reflective and memory-augmented abilities Xuechen Liang Yangfan He Yinghui Xia Xinyuan Song Jianhui Wang ... Keqin Li Jiaqi Chen Jinsong Yang Siyuan Chen Tianyu Shi LLMAG KELM CLL 33 2 0 01 Sep 2024
SWE-bench-java: A GitHub Issue Resolving Benchmark for Java Daoguang Zan Zhirong Huang Ailun Yu Shaoxin Lin Yifan Shi ... Bei Guan Pengjie Huang Tao Xie Yongji Wang Qianxiang Wang 26 7 0 26 Aug 2024
CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding? Yuwei Zhao Ziyang Luo Yuchen Tian Hongzhan Lin Weixiang Yan Annan Li Jing Ma ELM ALM LRM 37 8 0 20 Aug 2024
Retrieval-augmented code completion for local projects using large language models Marko Hostnik Marko Robnik-Sikonja RALM 17 0 0 09 Aug 2024
CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases Xiangyan Liu Bo Lan Zhiyuan Hu Yang Liu Zhicheng Zhang Fei-Yue Wang Michael Shieh Wenmeng Zhou 37 14 0 07 Aug 2024
NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time Yilong Chen Guoxia Wang Junyuan Shang Shiyao Cui Zhenyu Zhang Tingwen Liu Shuohuan Wang Yu Sun Dianhai Yu Hua-Hong Wu 18 14 0 07 Aug 2024
Making Long-Context Language Models Better Multi-Hop Reasoners Yanyang Li Shuo Liang M. Lyu Liwei Wang LLMAG LRM 22 8 0 06 Aug 2024
Palu: Compressing KV-Cache with Low-Rank Projection Chi-Chih Chang Wei-Cheng Lin Chien-Yu Lin Chong-Yan Chen Yu-Fang Hu Pei-Shuo Wang N. Huang Luis Ceze Kai-Chiang Wu 51 0 0 30 Jul 2024
Scaling Granite Code Models to 128K Context Matt Stallone Vaibhav Saxena Leonid Karlinsky Bridget McGinn Tim Bula ... Rogerio Feris Nirmit Desai David D. Cox Ruchir Puri Rameswar Panda 32 4 0 18 Jul 2024
On Mitigating Code LLM Hallucinations with API Documentation Nihal Jain Robert Kwiatkowski Baishakhi Ray M. K. Ramanathan Varun Kumar 33 7 0 13 Jul 2024
CodeUpdateArena: Benchmarking Knowledge Editing on API Updates Zeyu Leo Liu Shrey Pandit Xi Ye Eunsol Choi Greg Durrett KELM ALM 57 4 0 08 Jul 2024
Let the Code LLM Edit Itself When You Edit the Code Zhenyu He Jun Zhang Shengjie Luo Jingjing Xu Z. Zhang Di He KELM 29 0 0 03 Jul 2024
Agentless: Demystifying LLM-based Software Engineering Agents Chunqiu Steven Xia Yinlin Deng Soren Dunn Lingming Zhang LLMAG 32 78 0 01 Jul 2024
Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP Omer Goldman Alon Jacovi Aviv Slobodkin Aviya Maimon Ido Dagan Reut Tsarfaty 53 10 0 29 Jun 2024
Hierarchical Context Pruning: Optimizing Real-World Code Completion with Repository-Level Pretrained Code LLMs Lei Zhang Yunshui Li Jiaming Li Xiaobo Xia Jiaxi Yang Run Luo Minzheng Wang Longze Chen Junhao Liu Min Yang 27 0 0 26 Jun 2024
Qiskit HumanEval: An Evaluation Benchmark For Quantum Code Generative Models Sanjay Vishwakarma Francis Harkins Siddharth Golecha Vishal Sharathchandra Bajpe Nicolas Dupuis Luca Buratti David Kremer Ismael Faro Ruchir Puri Juan Cruz-Benito ELM 28 3 0 20 Jun 2024
DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence DeepSeek-AI Qihao Zhu Daya Guo Zhihong Shao Dejian Yang ... Jiashi Li Chenggang Zhao Chong Ruan Fuli Luo Wenfeng Liang MoE LRM ELM VLM 45 149 0 17 Jun 2024
AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology Minh Huynh Nguyen Thang Phan Chau Phong X. Nguyen Nghi D. Q. Bui 26 11 0 16 Jun 2024
Benchmarking Generative Models on Computational Thinking Tests in Elementary Visual Programming Victor-Alexandru Pădurean Adish Singla ELM 44 3 0 14 Jun 2024
DCA-Bench: A Benchmark for Dataset Curation Agents Benhao Huang Yingzhuo Yu Jin Huang Xingjian Zhang Jiaqi Ma 23 1 0 11 Jun 2024
RepoQA: Evaluating Long Context Code Understanding Jiawei Liu Jia Le Tian Vijay Daita Yuxiang Wei Yifeng Ding Yuhan Katherine Wang Jun Yang Lingming Zhang LLMAG 25 17 0 10 Jun 2024
Enhancing Repository-Level Code Generation with Integrated Contextual Information Zhiyuan Pan Xing Hu Xin Xia Xiaohu Yang 26 3 0 05 Jun 2024
Chain of Agents: Large Language Models Collaborating on Long-Context Tasks Yusen Zhang Ruoxi Sun Yanfei Chen Tomas Pfister Rui Zhang Sercan Ö. Arik RALM AI4CE LLMAG 42 28 0 04 Jun 2024
PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling Zefan Cai Yichi Zhang Bofei Gao Yuliang Liu Tianyu Liu ... Wayne Xiong Yue Dong Baobao Chang Junjie Hu Wen Xiao 55 83 0 04 Jun 2024
A Survey on Large Language Models for Code Generation Juyong Jiang Fan Wang Jiasi Shen Sungju Kim Sunghun Kim 40 74 0 01 Jun 2024
Divide-and-Conquer Meets Consensus: Unleashing the Power of Functions in Code Generation Jingchang Chen Hongxuan Tang Zheng Chu Qianglong Chen Zekun Wang Ming Liu Bing Qin 42 4 0 30 May 2024
DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories Jia Li Ge Li Yunfei Zhao Yongming Li Huanyu Liu ... Yihong Dong Zhi Jin Binhua Li Fei Huang Yongbin Li ALM 21 25 0 30 May 2024
Dataflow-Guided Retrieval Augmentation for Repository-Level Code Completion Wei Cheng Yuhan Wu Wei Hu 25 3 0 30 May 2024
RTL-Repo: A Benchmark for Evaluating LLMs on Large-Scale RTL Design Projects Ahmed Allam Mohamed Shalan 30 14 0 27 May 2024
MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generation Jianbo Dai Jianqiao Lu Yunlong Feng Rongju Ruan Ming Cheng Haochen Tan Zhijiang Guo ELM LRM 36 12 0 19 May 2024
Granite Code Models: A Family of Open Foundation Models for Code Intelligence Mayank Mishra Matt Stallone Gaoyuan Zhang Yikang Shen Aditya Prasad ... Amith Singhee Nirmit Desai David D. Cox Ruchir Puri Rameswar Panda AI4TS 46 51 0 07 May 2024
On the Limitations of Embedding Based Methods for Measuring Functional Correctness for Code Generation Atharva Naik 33 2 0 26 Apr 2024
CORM: Cache Optimization with Recent Message for Large Language Model Inference Jincheng Dai Zhuowei Huang Haiyun Jiang Chen Chen Deng Cai Wei Bi Shuming Shi 19 3 0 24 Apr 2024
Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks Chonghua Wang Haodong Duan Songyang Zhang Dahua Lin Kai-xiang Chen ELM 24 16 0 09 Apr 2024
EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories Jia Li Ge Li Xuanming Zhang Yihong Dong Zhi Jin 32 31 0 31 Mar 2024
Iterative Refinement of Project-Level Code Context for Precise Code Generation with Compiler Feedback Zhangqian Bi Yao Wan Zheng Wang Hongyu Zhang Batu Guan Fangxin Lu Zili Zhang Yulei Sui Hai Jin Xuanhua Shi 29 13 0 25 Mar 2024
CodeS: Natural Language to Code Repository via Multi-Layer Sketch Daoguang Zan Ailun Yu Wei Liu Dong Chen Bo Shen ... Bei Guan Zhiguang Yang Yongji Wang Qianxiang Wang Li-zhen Cui 18 14 0 25 Mar 2024
DevBench: A Comprehensive Benchmark for Software Development Bowen Li Wenhan Wu Ziwei Tang Lin Shi John Yang ... He Du Ping Yang Dahua Lin Chao Peng Kai Chen 85 10 0 13 Mar 2024
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code Naman Jain King Han Alex Gu Wen-Ding Li Fanjia Yan Tianjun Zhang Sida I. Wang Armando Solar-Lezama Koushik Sen Ion Stoica ELM 29 260 0 12 Mar 2024
InfiBench: Evaluating the Question-Answering Capabilities of Code Large Language Models Linyi Li Shijie Geng Zhenwen Li Yibo He Hao Yu Ziyue Hua Guanghan Ning Siwei Wang Tao Xie Hongxia Yang ELM 29 2 0 11 Mar 2024
RepoHyper: Better Context Retrieval Is All You Need for Repository-Level Code Completion Huy N. Phan Hoang N. Phan Tien N. Nguyen Nghi D. Q. Bui 35 12 0 10 Mar 2024
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks Linyuan Gong Sida Wang Mostafa Elhoushi Alvin Cheung 27 15 0 07 Mar 2024
LongWanjuan: Towards Systematic Measurement for Long Text Quality Kai Lv Xiaoran Liu Qipeng Guo Hang Yan Conghui He Xipeng Qiu Dahua Lin 28 4 0 21 Feb 2024
EffiBench: Benchmarking the Efficiency of Automatically Generated Code Dong Huang Yuhao Qing Weiyi Shang Heming Cui Jie M. Zhang 77 30 0 03 Feb 2024
Extending LLMs' Context Window with 100 Samples Yikai Zhang Junlong Li Pengfei Liu 16 11 0 13 Jan 2024
ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code Xiangru Tang Yuliang Liu Zefan Cai Yan Shao Junjie Lu ... Yujia Qin Wangchunshu Zhou Yilun Zhao Arman Cohan Mark B. Gerstein ELM LLMAG 27 17 0 16 Nov 2023
Prompt Cache: Modular Attention Reuse for Low-Latency Inference In Gim Guojun Chen Seung-seob Lee Nikhil Sarda Anurag Khandelwal Lin Zhong 22 71 0 07 Nov 2023
Bias Testing and Mitigation in LLM-based Code Generation Dong Huang Qingwen Bu Jie M. Zhang Xiaofei Xie Junjie Chen Heming Cui 33 20 0 03 Sep 2023
BioCoder: A Benchmark for Bioinformatics Code Generation with Large Language Models Xiangru Tang Bill Qian Rick Gao Jiakang Chen Xinyun Chen Mark B. Gerstein 16 10 0 31 Aug 2023
LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding Yushi Bai Xin Lv Jiajie Zhang Hong Lyu Jiankai Tang ... Aohan Zeng Lei Hou Yuxiao Dong Jie Tang Juanzi Li LLMAG RALM 24 486 0 28 Aug 2023