LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

12 March 2024

Tianjun Zhang

Papers citing "LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code"

50 / 64 papers shown

Title
LLMs Get Lost In Multi-Turn Conversation Philippe Laban Hiroaki Hayashi Yingbo Zhou Jennifer Neville 21 0 0 09 May 2025
WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch Zimu Lu Y. Yang Houxing Ren Haotian Hou Han Xiao Ke Wang Weikang Shi Aojun Zhou Mingjie Zhan H. Li LLMAG 35 0 0 06 May 2025
Exploring the Potential of Offline RL for Reasoning in LLMs: A Preliminary Study Xiaoyu Tian Sitong Zhao Haotian Wang Shuaiting Chen Yiping Peng Yunjie Ji Han Zhao Xiangang Li OffRL LRM 24 0 0 04 May 2025
R-Bench: Graduate-level Multi-disciplinary Benchmarks for LLM & MLLM Complex Reasoning Evaluation Meng-Hao Guo Jiajun Xu Yi Zhang Jiaxi Song Haoyang Peng ... Yongming Rao Houwen Peng Han Hu Gordon Wetzstein Shi-Min Hu ELM LRM 49 0 0 04 May 2025
T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT D. Jiang Ziyu Guo Renrui Zhang Zhuofan Zong Hao Li Le Zhuo Shilin Yan Pheng-Ann Heng H. Li LRM 57 0 0 01 May 2025
ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning Jingyang Yi Jiazheng Wang ReLM OODD LRM 54 0 0 30 Apr 2025
CodeFlowBench: A Multi-turn, Iterative Benchmark for Complex Code Generation Sizhe Wang Z. Wang Dongsheng Ma Yongan Yu Rui Ling Z. Li Feiyu Xiong W. Zhang LRM 47 0 0 30 Apr 2025
SecRepoBench: Benchmarking LLMs for Secure Code Generation in Real-World Repositories Connor Dilgren Purva Chiniya Luke Griffith Yu Ding Yizheng Chen 33 0 0 29 Apr 2025
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks Yixin Cao Shibo Hong X. Li Jiahao Ying Yubo Ma ... Juanzi Li Aixin Sun Xuanjing Huang Tat-Seng Chua Yu Jiang ALM ELM 84 0 0 26 Apr 2025
Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning Minju Seo Jinheon Baek Seongyun Lee S. Hwang AI4CE 35 0 0 24 Apr 2025
DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Training Xiaoyu Tian Sitong Zhao Haotian Wang Shuaiting Chen Yiping Peng Yunjie Ji Han Zhao Xiangang Li LRM 47 1 0 24 Apr 2025
Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning Chris Yichen Wei Yi Peng X. Wang Weijie Qiu ... Jianhao Zhang Y. Hao Xuchen Song Yang Liu Yahui Zhou OffRL AI4TS SyDa LRM VLM 67 0 0 23 Apr 2025
Synergizing RAG and Reasoning: A Systematic Review Yunfan Gao Yun Xiong Yijie Zhong Yuxi Bi Ming Xue H. Wang LRM AI4CE 24 0 0 22 Apr 2025
MARFT: Multi-Agent Reinforcement Fine-Tuning Junwei Liao Muning Wen J. Wang W. Zhang OffRL 21 0 0 21 Apr 2025
Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? Yang Yue Zhiqi Chen Rui Lu Andrew Zhao Zhaokai Wang Yang Yue Shiji Song Gao Huang ReLM LRM 40 5 0 18 Apr 2025
FuseRL: Dense Preference Optimization for Heterogeneous Model Fusion Longguang Zhong Fanqi Wan Ziyi Yang Guosheng Liang Tianyuan Shi Xiaojun Quan MoMe 53 0 0 09 Apr 2025
Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models Ruikang Liu Yuxuan Sun Manyi Zhang Haoli Bai Xianzhi Yu Tiezheng Yu C. Yuan Lu Hou MQ LRM 23 5 0 07 Apr 2025
SEAL: Steerable Reasoning Calibration of Large Language Models for Free Runjin Chen Zhenyu (Allen) Zhang Junyuan Hong Souvik Kundu Zhangyang Wang OffRL LRM 44 2 0 07 Apr 2025
How Accurately Do Large Language Models Understand Code? Sabaat Haroon Ahmad Faraz Khan Ahmad Humayun Waris Gill Abdul Haddi Amjad A. R. Butt Mohammad Taha Khan Muhammad Ali Gulzar ELM LRM 23 0 0 06 Apr 2025
Generative Evaluation of Complex Reasoning in Large Language Models Haowei Lin X. Wang Ruilin Yan Baizhou Huang Haotian Ye Jianhua Zhu Zihao Wang James Y. Zou Jianzhu Ma Yitao Liang ReLM ELM LRM 76 0 0 03 Apr 2025
L0-Reasoning Bench: Evaluating Procedural Correctness in Language Models via Simple Program Execution Simeng Sun Cheng-Ping Hsieh Faisal Ladhak Erik Arakelyan Santiago Akle Serano Boris Ginsburg ReLM ELM LRM 39 0 0 28 Mar 2025
FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation Wei Li Xin Zhang Zhongxin Guo Shaoguang Mao Wen Luo Guangyue Peng Yangyu Huang Houfeng Wang Scarlett Li 53 0 0 09 Mar 2025
Unity RL Playground: A Versatile Reinforcement Learning Framework for Mobile Robots Linqi Ye Rankun Li Xiaowen Hu Jiayi Li Boyang Xing Yan Peng Bin Liang 43 0 0 07 Mar 2025
Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs Ling Team B. Zeng C. Huang Chao Zhang Changxin Tian ... Zhaoxin Huan Zujie Wen Zhenhang Sun Zhuoxuan Du Z. He MoE ALM 94 2 0 07 Mar 2025
IterPref: Focal Preference Learning for Code Generation via Iterative Debugging Jie Wu Haoling Li Xin Zhang Jianwen Luo Yangyu Huang Ruihang Chu Y. Yang Scarlett Li 67 0 0 04 Mar 2025
Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning? Yancheng He Shilong Li J. Liu Weixun Wang Xingyuan Bu ... Zhongyuan Peng Z. Zhang Zhicheng Zheng Wenbo Su Bo Zheng ELM LRM 60 6 0 26 Feb 2025
InductionBench: LLMs Fail in the Simplest Complexity Class Wenyue Hua Tyler Wong Sun Fei Liangming Pan Adam Jardine William Yang Wang LRM 48 2 0 20 Feb 2025
A Unified Approach to Routing and Cascading for LLMs Jasper Dekoninck Maximilian Baader Martin Vechev 60 2 0 17 Feb 2025
SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?$ Samuel Miserendino M. Wang Tejal Patwardhan Johannes Heidecke 41 17 0 17 Feb 2025
Preference Optimization for Reasoning with Pseudo Feedback Fangkai Jiao Geyang Guo Xingxing Zhang Nancy F. Chen Shafiq R. Joty Furu Wei LRM 95 8 0 17 Feb 2025
Quantifying the Capability Boundary of DeepSeek Models: An Application-Driven Performance Analysis Kaikai Zhao Zhaoxiang Liu Xuejiao Lei Ning Wang Zhenhong Long ... W. Liu Kai Wang Wen Liu Kai Wang Shiguo Lian ELM LRM 44 1 0 16 Feb 2025
BoT: Breaking Long Thought Processes of o1-like Large Language Models through Backdoor Attack Zihao Zhu Hongbao Zhang Mingda Zhang Ruotong Wang Guanzong Wu Ke Xu Baoyuan Wu AAML LRM 46 4 0 16 Feb 2025
BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models Xu Huang Wenhao Zhu Hanxu Hu Conghui He Lei Li Shujian Huang Fei Yuan ELM 41 3 0 11 Feb 2025
ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates L. Yang Zhaochen Yu Bin Cui Mengdi Wang ReLM LRM AI4CE 90 10 0 10 Feb 2025
CSR-Bench: Benchmarking LLM Agents in Deployment of Computer Science Research Repositories Yijia Xiao Runhui Wang Luyang Kong Davor Golac Wei Wang LLMAG 54 0 0 10 Feb 2025
Twilight: Adaptive Attention Sparsity with Hierarchical Top- $p$ Pruning C. Lin Jiaming Tang Shuo Yang Hanshuo Wang Tian Tang Boyu Tian Ion Stoica Song Han Mingyu Gao 68 2 0 04 Feb 2025
Learning to Generate Unit Tests for Automated Debugging Archiki Prasad Elias Stengel-Eskin Justin Chih-Yao Chen Zaid Khan Mohit Bansal ELM 71 1 0 03 Feb 2025
Kimi k1.5: Scaling Reinforcement Learning with LLMs Kimi Team Angang Du Bofei Gao Bowei Xing Changjiu Jiang ... Zhilin Yang Zhiqi Huang Zihao Huang Ziyao Xu Z. Yang VLM ALM OffRL AI4TS LRM 85 128 0 22 Jan 2025
AIOpsLab: A Holistic Framework to Evaluate AI Agents for Enabling Autonomous Clouds Yinfang Chen Manish Shetty Gagan Somashekar Minghua Ma Yogesh L. Simmhan Jonathan Mace Chetan Bansal Rujia Wang Saravan Rajmohan 46 0 0 12 Jan 2025
OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement Tianyu Zheng Ge Zhang Tianhao Shen Xueling Liu Bill Yuchen Lin Jie Fu Wenhu Chen Xiang Yue SyDa 67 102 0 08 Jan 2025
Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Survey Junqiao Wang Zeng Zhang Yangfan He Yuyang Song Tianyu Shi ... Hengyuan Xu Kunyu Wu Guangwu Qian Qiuwu Chen Lewei He 23 8 0 03 Jan 2025
CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratings Shanghaoran Quan Jiaxi Yang Bowen Yu Bo Zheng Dayiheng Liu ... Zeyu Cui Yang Fan Y. Zhang Binyuan Hui Junyang Lin ALM ELM LRM 58 13 0 02 Jan 2025
WarriorCoder: Learning from Expert Battles to Augment Code Large Language Models Huawen Feng Pu Zhao Qingfeng Sun Can Xu Fangkai Yang ... Qianli Ma Qingwei Lin Saravan Rajmohan Dongmei Zhang Qi Zhang AAML ALM 60 0 0 23 Dec 2024
Are Large Language Models Memorizing Bug Benchmarks? Daniel Ramos Claudia Mamede Kush Jain Paulo Canelas Catarina Gamboa Claire Le Goues PILM ELM 84 6 0 20 Nov 2024
JudgeBench: A Benchmark for Evaluating LLM-based Judges Sijun Tan Siyuan Zhuang Kyle Montgomery William Y. Tang Alejandro Cuadron Chenguang Wang Raluca A. Popa Ion Stoica ELM ALM 45 35 0 16 Oct 2024
3DArticCyclists: Generating Synthetic Articulated 8D Pose-Controllable Cyclist Data for Computer Vision Applications Eduardo R. Corral-Soto Yang Liu Tongtong Cao Y. Ren Liu Bingbing 42 0 0 14 Oct 2024
CursorCore: Assist Programming through Aligning Anything Hao Jiang Qi Liu Rui Li Shengyu Ye Shijin Wang 34 1 0 09 Oct 2024
DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback Zaid Khan Elias Stengel-Eskin Jaemin Cho Mohit Bansal VGen 26 1 0 08 Oct 2024
ProcBench: Benchmark for Multi-Step Reasoning and Following Procedure Ippei Fujisawa Sensho Nobe Hiroki Seto Rina Onda Yoshiaki Uchida Hiroki Ikoma Pei-Chun Chien Ryota Kanai LRM 34 3 0 04 Oct 2024
What can Large Language Models Capture about Code Functional Equivalence? Nickil Maveli Antonio Vergari Shay B. Cohen 21 2 0 20 Aug 2024