CoIR: A Comprehensive Benchmark for Code Information Retrieval Models

CoIR: A Comprehensive Benchmark for Code Information Retrieval Models

3 July 2024

Xiangyang Li

Xinyi Dai

Yasheng Wang

Ruiming Tang

Papers citing "CoIR: A Comprehensive Benchmark for Code Information Retrieval Models"

18 / 18 papers shown

Title
FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documents Nandan Thakur Jimmy J. Lin Sam Havens Michael Carbin Omar Khattab Andrew Drozdov 27 2 0 17 Apr 2025
Code-Craft: Hierarchical Graph-Based Code Summarization for Enhanced Context Retrieval David Sounthiraraj Jared Hancock Yassin Kortam Ashok Javvaji Prabhat Singh Shaila Shankar 16 0 0 11 Apr 2025
LoRACode: LoRA Adapters for Code Embeddings Saumya Chaturvedi Aman Chadha Laurent Bindschaedler 47 0 0 07 Mar 2025
Benchmarking AI Models in Software Engineering: A Review, Search Tool, and Enhancement Protocol Roham Koohestani Philippe de Bekker M. Izadi VLM 40 0 0 07 Mar 2025
Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence Mohsen Fayyaz Ali Modarressi Hinrich Schuetze Nanyun Peng 49 0 0 06 Mar 2025
Granite Embedding Models Parul Awasthy Aashka Trivedi Yulong Li Mihaela A. Bornea David D. Cox ... Sukriti Sharma Avirup Sil Kate Soule Arafat Sultan Radu Florian RALM 48 0 0 27 Feb 2025
OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement Tianyu Zheng Ge Zhang Tianhao Shen Xueling Liu Bill Yuchen Lin Jie Fu Wenhu Chen Xiang Yue SyDa 61 102 0 08 Jan 2025
Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference Benjamin Warner Antoine Chaffin Benjamin Clavié Orion Weller Oskar Hallström ... Tom Aarsen Nathan Cooper Griffin Adams Jeremy Howard Iacopo Poli 72 51 0 18 Dec 2024
CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Reranking Tarun Suresh R. Reddy Yifei Xu Zach Nussbaum Andriy Mulyar Brandon Duderstadt Heng Ji 72 3 0 01 Dec 2024
CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrieval Y. Liu Rui Meng Shafiq R. Joty Silvio Savarese Caiming Xiong Yingbo Zhou Semih Yavuz 70 2 0 19 Nov 2024
A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why? QiHong Chen Jiawei Li Jiecheng Deng Jiachen Yu Justin Tian Jin Chen Iftekhar Ahmed 28 0 0 03 Nov 2024
Retrieval-Augmented Generation for AI-Generated Content: A Survey Penghao Zhao Hailin Zhang Qinhan Yu Zhengren Wang Yunteng Geng Fangcheng Fu Ling Yang Wentao Zhang Jie Jiang Bin Cui 3DV 96 182 0 29 Feb 2024
Repetition Improves Language Model Embeddings Jacob Mitchell Springer Suhas Kotha Daniel Fried Graham Neubig Aditi Raghunathan 34 9 0 23 Feb 2024
BGE M3-Embedding: Multi-Lingual, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation Jianlv Chen Shitao Xiao Peitian Zhang Kun Luo Defu Lian Zheng Liu 115 306 0 05 Feb 2024
RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder Shitao Xiao Zheng Liu Yingxia Shao Zhao Cao RALM 112 105 0 24 May 2022
Measuring Coding Challenge Competence With APPS Dan Hendrycks Steven Basart Saurav Kadavath Mantas Mazeika Akul Arora ... Collin Burns Samir Puranik Horace He D. Song Jacob Steinhardt ELM AIMat ALM 181 614 0 20 May 2021
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models Nandan Thakur Nils Reimers Andreas Rucklé Abhishek Srivastava Iryna Gurevych VLM 229 720 0 17 Apr 2021
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation Shuai Lu Daya Guo Shuo Ren Junjie Huang Alexey Svyatkovskiy ... Nan Duan Neel Sundaresan Shao Kun Deng Shengyu Fu Shujie Liu ELM 183 853 0 09 Feb 2021