ResearchTrend.AI
Measuring Coding Challenge Competence With APPS
v1v2v3 (latest)

Measuring Coding Challenge Competence With APPS

Dan Hendrycks
Steven Basart
Saurav Kadavath
Mantas Mazeika
Akul Arora
Ethan Guo
Collin Burns
Samir Puranik
Horace He
Basel Alomair
Jacob Steinhardt
ArXiv (abs)PDFHTMLHuggingFace (1 upvotes)

Papers citing "Measuring Coding Challenge Competence With APPS"

50 / 541 papers shown
Title
Where Do LLMs Still Struggle? An In-Depth Analysis of Code Generation Benchmarks
Where Do LLMs Still Struggle? An In-Depth Analysis of Code Generation Benchmarks
Amir Molzam Sharifloo
Maedeh Heydari
Parsa Kazerooni
Daniel Maninger
Mira Mezini
ALM
28
0
0
06 Nov 2025
HarnessLLM: Automatic Testing Harness Generation via Reinforcement Learning
HarnessLLM: Automatic Testing Harness Generation via Reinforcement Learning
Yujian Liu
Jiabao Ji
Yang Zhang
Wenbo Guo
Tommi Jaakkola
Shiyu Chang
0
0
0
02 Nov 2025
A CPU-Centric Perspective on Agentic AI
A CPU-Centric Perspective on Agentic AI
Ritik Raj
Hong Wang
Tushar Krishna
0
0
0
01 Nov 2025
CodeAlignBench: Assessing Code Generation Models on Developer-Preferred Code Adjustments
CodeAlignBench: Assessing Code Generation Models on Developer-Preferred Code Adjustments
Forough Mehralian
Ryan Shar
James Rae
Alireza Hashemi
ALMELM
168
0
0
31 Oct 2025
Beyond Synthetic Benchmarks: Evaluating LLM Performance on Real-World Class-Level Code Generation
Beyond Synthetic Benchmarks: Evaluating LLM Performance on Real-World Class-Level Code Generation
Musfiqur Rahman
SayedHassan Khatoonabadi
Emad Shihab
ELM
144
0
0
30 Oct 2025
QCoder Benchmark: Bridging Language Generation and Quantum Hardware through Simulator-Based Feedback
QCoder Benchmark: Bridging Language Generation and Quantum Hardware through Simulator-Based Feedback
Taku Mikuriya
Tatsuya Ishigaki
Masayuki Kawarada
Shunya Minami
Tadashi Kadowaki
...
Shunya Takata
Takumi Kato
Tamotsu Basseda
Reo Yamada
Hiroya Takamura
ALMELM
52
1
0
30 Oct 2025
Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agents
Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agents
Jiayi Kuang
Yinghui Li
Xin Zhang
Yangning Li
Di Yin
Xing Sun
Ying Shen
Philip S. Yu
4
0
0
29 Oct 2025
Lifecycle-Aware code generation: Leveraging Software Engineering Phases in LLMs
Lifecycle-Aware code generation: Leveraging Software Engineering Phases in LLMs
Xing Xing
Wei Wang
Lipeng Ma
Weidong Yang
Junjie Zheng
5
0
0
28 Oct 2025
Smaller Models, Smarter Rewards: A Two-Sided Approach to Process and Outcome Rewards
Smaller Models, Smarter Rewards: A Two-Sided Approach to Process and Outcome Rewards
Jan Niklas Groeneveld
Xi Qin
Alexander Schaefer
Yaad Oren
ALMLRM
40
0
0
27 Oct 2025
Verifiable Accuracy and Abstention Rewards in Curriculum RL to Alleviate Lost-in-Conversation
Verifiable Accuracy and Abstention Rewards in Curriculum RL to Alleviate Lost-in-Conversation
Ming Li
KELM
48
0
0
21 Oct 2025
Noise-corrected GRPO: From Noisy Rewards to Unbiased Gradients
Noise-corrected GRPO: From Noisy Rewards to Unbiased Gradients
Omar El mansouri
Mohamed El Amine Seddik
Salem Lahlou
NoLa
154
0
0
21 Oct 2025
CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment
CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment
Xue Jiang
Yihong Dong
Mengyang Liu
Hongyi Deng
Tian Wang
...
Zhi Jin
Wenpin Jiao
Fei Huang
Yongbin Li
Ge Li
16
0
0
21 Oct 2025
TREAT: A Code LLMs Trustworthiness / Reliability Evaluation and Testing Framework
TREAT: A Code LLMs Trustworthiness / Reliability Evaluation and Testing Framework
Shuzheng Gao
E. Li
Man Ho Lam
Jingyu Xiao
Yuxuan Wan
Chaozheng Wang
Ng Man Tik
Michael R. Lyu
36
0
0
20 Oct 2025
Leave It to the Experts: Detecting Knowledge Distillation via MoE Expert Signatures
Leave It to the Experts: Detecting Knowledge Distillation via MoE Expert Signatures
Pingzhi Li
Morris Yu-Chao Huang
Zhen Tan
Qingquan Song
Jie Peng
Kai Zou
Yu Cheng
Kaidi Xu
Tianlong Chen
MoEAAML
73
0
0
19 Oct 2025
A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
Zhi Zhou
Yuhao Tan
Zenan Li
Yuan Yao
Lan-Zhe Guo
Yu-Feng Li
Xiaoxing Ma
LRM
26
0
0
17 Oct 2025
An Experimental Study of Real-Life LLM-Proposed Performance Improvements
An Experimental Study of Real-Life LLM-Proposed Performance Improvements
Lirong Yi
Gregory Gay
Philipp Leitner
32
0
0
17 Oct 2025
Midtraining Bridges Pretraining and Posttraining Distributions
Midtraining Bridges Pretraining and Posttraining Distributions
Emmy Liu
Graham Neubig
Chenyan Xiong
CLL
64
0
0
16 Oct 2025
Stronger Together: On-Policy Reinforcement Learning for Collaborative LLMs
Stronger Together: On-Policy Reinforcement Learning for Collaborative LLMs
Yujie Zhao
Lanxiang Hu
Y. Wang
Minmin Hou
Hao Zhang
Ke Ding
Jishen Zhao
36
1
0
13 Oct 2025
LiveOIBench: Can Large Language Models Outperform Human Contestants in Informatics Olympiads?
LiveOIBench: Can Large Language Models Outperform Human Contestants in Informatics Olympiads?
Kaijian Zou
Aaron Xiong
Yunxiang Zhang
Frederick Zhang
Yueqi Ren
Jirong Yang
Ayoung Lee
Shitanshu Bhushan
Lu Wang
ReLMALMELMLRM
45
0
0
10 Oct 2025
MOSAIC: Multi-agent Orchestration for Task-Intelligent Scientific Coding
MOSAIC: Multi-agent Orchestration for Task-Intelligent Scientific Coding
Siddeshwar Raghavan
Tanwi Mallick
AI4CE
24
0
0
09 Oct 2025
U-Bench: A Comprehensive Understanding of U-Net through 100-Variant Benchmarking
U-Bench: A Comprehensive Understanding of U-Net through 100-Variant Benchmarking
Fenghe Tang
Chengqi Dong
Wenxin Ma
Zikang Xu
Heqin Zhu
Zihang Jiang
Rongsheng Wang
Yuhao Wang
Chenxu Wu
S. Kevin Zhou
ELMVLM
44
0
0
08 Oct 2025
Vibe Checker: Aligning Code Evaluation with Human Preference
Vibe Checker: Aligning Code Evaluation with Human Preference
Ming Zhong
Xiang Zhou
T. Chang
Q. Wang
Nan Xu
...
Shyam Upadhyay
Jeremiah Zhe Liu
Jiawei Han
Benoit Schillings
Jiao Sun
44
0
0
08 Oct 2025
Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Context
Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Context
Yoav Gur-Arieh
Mor Geva
Atticus Geiger
KELM
72
2
0
07 Oct 2025
Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning
Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning
Jonas Hübotter
Leander Diaz-Bone
Ido Hakimi
Andreas Krause
Moritz Hardt
58
0
0
06 Oct 2025
Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time
Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time
Daniel Tan
Anders Woodruff
Niels Warncke
Arun Jose
Maxime Riché
David Demitri Africa
Mia Taylor
106
1
0
05 Oct 2025
Investigating The Smells of LLM Generated Code
Investigating The Smells of LLM Generated Code
Debalina Ghosh Paul
Hong Zhu
Ian Bayley
20
0
0
03 Oct 2025
Addressing Pitfalls in the Evaluation of Uncertainty Estimation Methods for Natural Language Generation
Addressing Pitfalls in the Evaluation of Uncertainty Estimation Methods for Natural Language Generation
Mykyta Ielanskyi
Kajetan Schweighofer
L. Aichberger
Sepp Hochreiter
HILM
52
0
0
02 Oct 2025
Is It Thinking or Cheating? Detecting Implicit Reward Hacking by Measuring Reasoning Effort
Is It Thinking or Cheating? Detecting Implicit Reward Hacking by Measuring Reasoning Effort
Xinpeng Wang
Nitish Joshi
Barbara Plank
Rico Angell
He He
LRM
75
0
1
01 Oct 2025
BroRL: Scaling Reinforcement Learning via Broadened Exploration
BroRL: Scaling Reinforcement Learning via Broadened Exploration
Jian Hu
Mingjie Liu
Ximing Lu
Fang Wu
Zaïd Harchaoui
...
Yejin Choi
Pavlo Molchanov
J. Yang
Jan Kautz
Yi Dong
LRM
18
0
0
01 Oct 2025
LSPO: Length-aware Dynamic Sampling for Policy Optimization in LLM Reasoning
LSPO: Length-aware Dynamic Sampling for Policy Optimization in LLM Reasoning
Weizhe Chen
Sven Koenig
B. Dilkina
32
0
0
01 Oct 2025
Regression Language Models for Code
Regression Language Models for Code
Yash Akhauri
Xingyou Song
Arissa Wongpanich
Bryan Lewandowski
Mohamed S. Abdelfattah
80
0
0
30 Sep 2025
Evaluating SAP Joule for Code Generation
Evaluating SAP Joule for Code Generation
Joshua Heisler
Johannes Reisinger
Andreas Fischer
ELM
28
0
0
29 Sep 2025
ByteSized32Refactored: Towards an Extensible Interactive Text Games Corpus for LLM World Modeling and Evaluation
ByteSized32Refactored: Towards an Extensible Interactive Text Games Corpus for LLM World Modeling and Evaluation
Haonan Wang
Junfeng Sun
Xingdi Yuan
Ruoyao Wang
Ziang Xiao
20
0
0
28 Sep 2025
A benchmark for vericoding: formally verified program synthesis
A benchmark for vericoding: formally verified program synthesis
Sergiu Bursuc
Theodore Ehrenborg
Shaowei Lin
Lacramioara Astefanoaei
Ionel Emilian Chiosa
...
Adem Bizid
Quinn Dougherty
Miranda Zhao
Max Tan
Max Tegmark
8
0
0
26 Sep 2025
Variational Reasoning for Language Models
Variational Reasoning for Language Models
Xiangxin Zhou
Zichen Liu
Haonan Wang
Chao Du
Min Lin
Chongxuan Li
Liang Wang
Tianyu Pang
OffRLLRM
62
0
0
26 Sep 2025
RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs?
RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs?
Yiyou Sun
Yuhan Cao
Pohao Huang
Haoyue Bai
Hannaneh Hajishirzi
Nouha Dziri
Dawn Song
OffRLLRM
45
0
0
25 Sep 2025
Verification Limits Code LLM Training
Verification Limits Code LLM Training
Srishti Gureja
Elena Tommasone
Jingyi He
Sara Hooker
Matthias Gallé
Marzieh Fadaee
ALMOffRL
56
1
0
25 Sep 2025
MapCoder-Lite: Squeezing Multi-Agent Coding into a Single Small LLM
MapCoder-Lite: Squeezing Multi-Agent Coding into a Single Small LLM
W. Lee
Junhee Cho
Jungwook Choi
LLMAGALM
41
0
0
22 Sep 2025
SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
Xiang Deng
Jeff Da
Edwin Pan
Yannis Yiming He
Charles Ide
...
Zifan Wang
Chen Bo Calvin Zhang
Noah Jacobson
Bing Liu
Brad Kenstler
24
2
0
21 Sep 2025
Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
Keliang Liu
Dingkang Yang
Ziyun Qian
Weijie Yin
Y. Wang
Hongsheng Li
Jun Liu
Peng Zhai
Y. Liu
Lihua Zhang
OffRLLRM
62
2
0
20 Sep 2025
Prompt Stability in Code LLMs: Measuring Sensitivity across Emotion- and Personality-Driven Variations
Prompt Stability in Code LLMs: Measuring Sensitivity across Emotion- and Personality-Driven Variations
Wei Ma
Y. Yang
Jingquan Ge
Xiaofei Xie
Lingxiao Jiang
44
0
0
17 Sep 2025
Do Code Semantics Help? A Comprehensive Study on Execution Trace-Based Information for Code Large Language Models
Do Code Semantics Help? A Comprehensive Study on Execution Trace-Based Information for Code Large Language Models
Jian-Xun Wang
Xiaofei Xie
Q. Hu
Shangqing Liu
Yi Li
LRM
52
2
0
15 Sep 2025
SciML Agents: Write the Solver, Not the Solution
SciML Agents: Write the Solver, Not the Solution
Saarth Gaonkar
Xiang Zheng
Haocheng Xi
Rishabh Tiwari
Kurt Keutzer
Dmitriy Morozov
Michael W. Mahoney
Amir Gholami
LLMAG
84
1
0
12 Sep 2025
Astra: A Multi-Agent System for GPU Kernel Performance Optimization
Astra: A Multi-Agent System for GPU Kernel Performance Optimization
Anjiang Wei
Tianran Sun
Yogesh Seenichamy
Hang Song
Anne Ouyang
Azalia Mirhoseini
Ke Wang
Alex Aiken
43
4
0
09 Sep 2025
ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation
ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation
Kai Ye
Liangcai Su
Chenxiong Qian
SILM
44
0
0
09 Sep 2025
How Small Transformation Expose the Weakness of Semantic Similarity Measures
How Small Transformation Expose the Weakness of Semantic Similarity Measures
Serge Lionel Nikiema
Albérick Euraste Djiré
Abdoul Aziz Bonkoungou
Micheline Bénédicte Moumoula
Jordan Samhi
Abdoul Kader Kaboré
Jacques Klein
Tegawende F. Bissyande
32
0
0
08 Sep 2025
Benchmarking Information Retrieval Models on Complex Retrieval Tasks
Benchmarking Information Retrieval Models on Complex Retrieval Tasks
Julian Killingback
Hamed Zamani
44
2
0
08 Sep 2025
ReCode: Improving LLM-based Code Repair with Fine-Grained Retrieval-Augmented Generation
ReCode: Improving LLM-based Code Repair with Fine-Grained Retrieval-Augmented Generation
Yicong Zhao
Shisong Chen
Jiacheng Zhang
Zhixu Li
44
1
0
02 Sep 2025
Beyond the Surface: A Solution-Aware Retrieval Model for Competition-level Code Generation
Beyond the Surface: A Solution-Aware Retrieval Model for Competition-level Code Generation
Shiwen Zhang
LingXiang Wang
Hainan Zhang
Ziwei Wang
Sijia Wen
Zhiming Zheng
RALMLRM
44
0
0
01 Sep 2025
Can Multi-turn Self-refined Single Agent LMs with Retrieval Solve Hard Coding Problems?
Can Multi-turn Self-refined Single Agent LMs with Retrieval Solve Hard Coding Problems?
Md Tanzib Hosain
Md Kishor Morol
ReLMLRM
42
2
0
30 Aug 2025

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.