ResearchTrend.AI
  • Communities
  • Connect sessions
  • AI calendar
  • Organizations
  • Join Slack
  • Contact Sales
Papers
Communities
Social Events
Terms and Conditions
Pricing
Contact Sales
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2105.09938
  4. Cited By
Measuring Coding Challenge Competence With APPS
v1v2v3 (latest)

Measuring Coding Challenge Competence With APPS

20 May 2021
Dan Hendrycks
Steven Basart
Saurav Kadavath
Mantas Mazeika
Akul Arora
Ethan Guo
Collin Burns
Samir Puranik
Horace He
Basel Alomair
Jacob Steinhardt
    ELMAIMatALM
ArXiv (abs)PDFHTMLHuggingFace (1 upvotes)

Papers citing "Measuring Coding Challenge Competence With APPS"

50 / 541 papers shown
Title
Do Large Language Models Pay Similar Attention Like Human Programmers
  When Generating Code?
Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?
Bonan Kou
Shengmai Chen
Zhijie Wang
Lei Ma
Tianyi Zhang
ALM
140
19
0
02 Jun 2023
Better Context Makes Better Code Language Models: A Case Study on
  Function Call Argument Completion
Better Context Makes Better Code Language Models: A Case Study on Function Call Argument CompletionAAAI Conference on Artificial Intelligence (AAAI), 2023
Hengzhi Pei
Jinman Zhao
Leonard Lausen
Sheng Zha
George Karypis
ELMLRM
108
26
0
01 Jun 2023
CodeTF: One-stop Transformer Library for State-of-the-art Code LLM
CodeTF: One-stop Transformer Library for State-of-the-art Code LLM
Nghi D. Q. Bui
Hung Le
Yue Wang
Junnan Li
Akhilesh Deepak Gotmare
Steven C. H. Hoi
169
26
0
31 May 2023
SheetCopilot: Bringing Software Productivity to the Next Level through
  Large Language Models
SheetCopilot: Bringing Software Productivity to the Next Level through Large Language ModelsNeural Information Processing Systems (NeurIPS), 2023
Hongxin Li
Jingran Su
Yuntao Chen
Qing Li
Zhaoxiang Zhang
LMTD
158
47
0
30 May 2023
ANPL: Towards Natural Programming with Interactive Decomposition
ANPL: Towards Natural Programming with Interactive DecompositionNeural Information Processing Systems (NeurIPS), 2023
Di Huang
Ziyuan Nan
Xingui Hu
Pengwei Jin
Shaohui Peng
...
Rui Zhang
Zidong Du
Qi Guo
Yewen Pu
Yunji Chen
197
13
0
29 May 2023
Demo2Code: From Summarizing Demonstrations to Synthesizing Code via
  Extended Chain-of-Thought
Demo2Code: From Summarizing Demonstrations to Synthesizing Code via Extended Chain-of-ThoughtNeural Information Processing Systems (NeurIPS), 2023
Huaxiaoyue Wang
Gonzalo Gonzalez-Pumariega
Yash Sharma
Sanjiban Choudhury
LM&Ro
213
44
0
26 May 2023
From Words to Wires: Generating Functioning Electronic Devices from
  Natural Language Descriptions
From Words to Wires: Generating Functioning Electronic Devices from Natural Language DescriptionsConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Peter Alexander Jansen
172
3
0
24 May 2023
A New Era in Software Security: Towards Self-Healing Software via Large
  Language Models and Formal Verification
A New Era in Software Security: Towards Self-Healing Software via Large Language Models and Formal VerificationInternational Conference/Workshop on Automation of Software Test (AST), 2023
Norbert Tihanyi
Ridhi Jain
Yiannis Charalambous
M. Ferrag
Youcheng Sun
Lucas C. Cordeiro
155
70
0
24 May 2023
ALGO: Synthesizing Algorithmic Programs with LLM-Generated Oracle
  Verifiers
ALGO: Synthesizing Algorithmic Programs with LLM-Generated Oracle VerifiersNeural Information Processing Systems (NeurIPS), 2023
Kexun Zhang
Danqing Wang
Jingtao Xia
William Yang Wang
Lei Li
155
53
0
24 May 2023
Neural Machine Translation for Code Generation
Neural Machine Translation for Code Generation
K. Dharma
Clayton T. Morrison
160
6
0
22 May 2023
The "code'' of Ethics:A Holistic Audit of AI Code Generators
The "code'' of Ethics:A Holistic Audit of AI Code GeneratorsIEEE Transactions on Dependable and Secure Computing (IEEE TDSC), 2023
Wanlun Ma
Yiliao Song
Minhui Xue
Sheng Wen
Yang Xiang
89
14
0
22 May 2023
Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning
  and Coding with LLMs
Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMsConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Pranjal Aggarwal
Aman Madaan
Yiming Yang
Mausam
LRM
213
71
0
19 May 2023
Think Outside the Code: Brainstorming Boosts Large Language Models in
  Code Generation
Think Outside the Code: Brainstorming Boosts Large Language Models in Code Generation
Xinyu Li
Jiang-Tian Xue
Zheng Xie
Ming Li
LRM
121
35
0
18 May 2023
CodeT5+: Open Code Large Language Models for Code Understanding and
  Generation
CodeT5+: Open Code Large Language Models for Code Understanding and GenerationConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Yue Wang
Hung Le
Akhilesh Deepak Gotmare
Nghi D. Q. Bui
Junnan Li
Steven C. H. Hoi
ALM
234
572
0
13 May 2023
The Vault: A Comprehensive Multilingual Dataset for Advancing Code
  Understanding and Generation
The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation
Dũng Nguyễn Mạnh
Nam Le Hai
An Dau
A. Nguyen
Khanh N. Nghiem
Jingnan Guo
Nghi D. Q. Bui
189
21
0
09 May 2023
Self-Edit: Fault-Aware Code Editor for Code Generation
Self-Edit: Fault-Aware Code Editor for Code GenerationAnnual Meeting of the Association for Computational Linguistics (ACL), 2023
Kechi Zhang
Zhuo Li
Jia Li
Ge Li
Zhi Jin
306
136
0
06 May 2023
Outline, Then Details: Syntactically Guided Coarse-To-Fine Code
  Generation
Outline, Then Details: Syntactically Guided Coarse-To-Fine Code GenerationInternational Conference on Machine Learning (ICML), 2023
Wenqing Zheng
S. Sharan
Ajay Jaiswal
Kevin Wang
Yihan Xi
Dejia Xu
Zhangyang Wang
227
31
0
28 Apr 2023
Stochastic Code Generation
Stochastic Code Generation
Swapnil Sharma
Nikita Anand
V. KranthiKiranG.
SyDa
79
1
0
14 Apr 2023
"What It Wants Me To Say": Bridging the Abstraction Gap Between End-User
  Programmers and Code-Generating Large Language Models
"What It Wants Me To Say": Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language ModelsInternational Conference on Human Factors in Computing Systems (CHI), 2023
Michael Xieyang Liu
Advait Sarkar
Carina Negreanu
B. Zorn
Jack Williams
N. Toronto
Andrew D. Gordon
180
127
0
13 Apr 2023
Teaching Large Language Models to Self-Debug
Teaching Large Language Models to Self-DebugInternational Conference on Learning Representations (ICLR), 2023
Xinyun Chen
Maxwell Lin
Nathanael Scharli
Denny Zhou
LRM
365
851
0
11 Apr 2023
CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual
  Benchmarking on HumanEval-X
CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-XKnowledge Discovery and Data Mining (KDD), 2023
Qinkai Zheng
Xiao Xia
Xu Zou
Yuxiao Dong
Shanshan Wang
...
Andi Wang
Yang Li
Teng Su
Zhilin Yang
Jie Tang
ELMALMSyDa
278
429
0
30 Mar 2023
RepoCoder: Repository-Level Code Completion Through Iterative Retrieval
  and Generation
RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and GenerationConference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Fengji Zhang
B. Chen
Yue Zhang
Jacky Keung
Jin Liu
Daoguang Zan
Yi Mao
Jian-Guang Lou
Weizhu Chen
168
316
0
22 Mar 2023
PanGu-Σ: Towards Trillion Parameter Language Model with Sparse
  Heterogeneous Computing
PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing
Xiaozhe Ren
Pingyi Zhou
Xinfan Meng
Xinjing Huang
Yadao Wang
...
Jiansheng Wei
Xin Jiang
Teng Su
Qun Liu
Jun Yao
ALMMoE
144
75
0
20 Mar 2023
Meet in the Middle: A New Pre-training Paradigm
Meet in the Middle: A New Pre-training ParadigmNeural Information Processing Systems (NeurIPS), 2023
A. Nguyen
Nikos Karampatziakis
Weizhu Chen
82
24
0
13 Mar 2023
Planning with Large Language Models for Code Generation
Planning with Large Language Models for Code GenerationInternational Conference on Learning Representations (ICLR), 2023
Shun Zhang
Zhenfang Chen
Songlin Yang
Mingyu Ding
J. Tenenbaum
Chuang Gan
157
202
0
09 Mar 2023
Cost-Effective Hyperparameter Optimization for Large Language Model
  Generation Inference
Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference
Chi Wang
Susan Liu
Ahmed Hassan Awadallah
142
52
0
08 Mar 2023
xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code
  Understanding, Generation, Translation and Retrieval
xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and RetrievalAnnual Meeting of the Association for Computational Linguistics (ACL), 2023
Mohammad Abdullah Matin Khan
M Saiful Bari
Xuan Long Do
Weishi Wang
Md. Rizwan Parvez
Shafiq Joty
ALMELM
298
45
0
06 Mar 2023
Bounding the Capabilities of Large Language Models in Open Text
  Generation with Prompt Constraints
Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt ConstraintsFindings (Findings), 2023
Albert Lu
Hongxin Zhang
Yanzhe Zhang
Xuezhi Wang
Diyi Yang
LRM
105
37
0
17 Feb 2023
PAC Prediction Sets for Large Language Models of Code
PAC Prediction Sets for Large Language Models of CodeInternational Conference on Machine Learning (ICML), 2023
Adam Khakhar
Stephen Mell
Osbert Bastani
188
7
0
17 Feb 2023
Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard
  Security Attacks
Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks
Daniel Kang
Xuechen Li
Ion Stoica
Carlos Guestrin
Matei A. Zaharia
Tatsunori Hashimoto
AAML
166
301
0
11 Feb 2023
Measuring The Impact Of Programming Language Distribution
Measuring The Impact Of Programming Language DistributionInternational Conference on Machine Learning (ICML), 2023
Gabriel Orlanski
Kefan Xiao
Xavier Garcia
Jeffrey Hui
Joshua Howland
J. Malmaud
Jacob Austin
Rishah Singh
Michele Catasta
285
41
0
03 Feb 2023
Execution-based Code Generation using Deep Reinforcement Learning
Execution-based Code Generation using Deep Reinforcement Learning
Parshin Shojaee
Aneesh Jain
Sindhu Tipirneni
Chandan K. Reddy
272
79
0
31 Jan 2023
SantaCoder: don't reach for the stars!
SantaCoder: don't reach for the stars!
Loubna Ben Allal
Raymond Li
Denis Kocetkov
Chenghao Mou
Christopher Akiki
...
Sean M. Hughes
Daniel Fried
Arjun Guha
H. D. Vries
Leandro von Werra
284
220
0
09 Jan 2023
Parsel: Algorithmic Reasoning with Language Models by Composing
  Decompositions
Parsel: Algorithmic Reasoning with Language Models by Composing DecompositionsNeural Information Processing Systems (NeurIPS), 2022
E. Zelikman
Qian Huang
Gabriel Poesia
Noah D. Goodman
Nick Haber
ReLMLRM
170
67
0
20 Dec 2022
Execution-Based Evaluation for Open-Domain Code Generation
Execution-Based Evaluation for Open-Domain Code GenerationConference on Empirical Methods in Natural Language Processing (EMNLP), 2022
Zhiruo Wang
Shuyan Zhou
Daniel Fried
Graham Neubig
ELM
188
96
0
20 Dec 2022
A Survey on Pretrained Language Models for Neural Code Intelligence
A Survey on Pretrained Language Models for Neural Code Intelligence
Yichen Xu
Yanqiao Zhu
72
18
0
20 Dec 2022
Large Language Models Meet NL2Code: A Survey
Large Language Models Meet NL2Code: A SurveyAnnual Meeting of the Association for Computational Linguistics (ACL), 2022
Daoguang Zan
B. Chen
Fengji Zhang
Di Lu
Bingchao Wu
Bei Guan
Yongji Wang
Jian-Guang Lou
ELMALM
145
222
0
19 Dec 2022
Natural Language to Code Generation in Interactive Data Science
  Notebooks
Natural Language to Code Generation in Interactive Data Science NotebooksAnnual Meeting of the Association for Computational Linguistics (ACL), 2022
Pengcheng Yin
Wen-Ding Li
Kefan Xiao
Abhishek Rao
Yeming Wen
...
Paige Bailey
Michele Catasta
Henryk Michalewski
Oleksandr Polozov
Charles Sutton
158
85
0
19 Dec 2022
Plansformer: Generating Symbolic Plans using Transformers
Plansformer: Generating Symbolic Plans using Transformers
Vishal Pallagani
Bharath Muppasani
K. Murugesan
F. Rossi
L. Horesh
Biplav Srivastava
F. Fabiano
Andrea Loreggia
LM&RoLLMAGOffRL
130
44
0
16 Dec 2022
A Survey on Natural Language Processing for Programming
A Survey on Natural Language Processing for ProgrammingInternational Conference on Language Resources and Evaluation (LREC), 2022
Qingfu Zhu
Xianzhen Luo
Fang Liu
Cuiyun Gao
Wanxiang Che
134
5
0
12 Dec 2022
Coder Reviewer Reranking for Code Generation
Coder Reviewer Reranking for Code GenerationInternational Conference on Machine Learning (ICML), 2022
Tianyi Zhang
Tao Yu
Tatsunori B. Hashimoto
M. Lewis
Anuj Kumar
Daniel Fried
Sida I. Wang
184
107
0
29 Nov 2022
The Stack: 3 TB of permissively licensed source code
The Stack: 3 TB of permissively licensed source code
Denis Kocetkov
Raymond Li
Loubna Ben Allal
Jia Li
Chenghao Mou
...
Sean M. Hughes
Thomas Wolf
Dzmitry Bahdanau
Leandro von Werra
H. D. Vries
154
382
0
20 Nov 2022
DS-1000: A Natural and Reliable Benchmark for Data Science Code
  Generation
DS-1000: A Natural and Reliable Benchmark for Data Science Code GenerationInternational Conference on Machine Learning (ICML), 2022
Yuhang Lai
Chengxi Li
Yiming Wang
Tianyi Zhang
Ruiqi Zhong
Luke Zettlemoyer
Scott Yih
Daniel Fried
Si-yi Wang
Tao Yu
ELMALM
183
411
0
18 Nov 2022
Execution-based Evaluation for Data Science Code Generation Models
Execution-based Evaluation for Data Science Code Generation Models
Junjie Huang
Chenglong Wang
Jipeng Zhang
Cong Yan
Haotian Cui
J. Inala
Colin B. Clement
Nan Duan
Jianfeng Gao
ELM
166
39
0
17 Nov 2022
A Simple, Yet Effective Approach to Finding Biases in Code Generation
A Simple, Yet Effective Approach to Finding Biases in Code GenerationAnnual Meeting of the Association for Computational Linguistics (ACL), 2022
Spyridon Mouselinos
Mateusz Malinowski
Henryk Michalewski
207
10
0
31 Oct 2022
When Language Model Meets Private Library
When Language Model Meets Private LibraryConference on Empirical Methods in Natural Language Processing (EMNLP), 2022
Daoguang Zan
Bei Chen
Zeqi Lin
Bei Guan
Yongji Wang
Jian-Guang Lou
ALM
185
84
0
31 Oct 2022
Aligning Offline Metrics and Human Judgments of Value for Code
  Generation Models
Aligning Offline Metrics and Human Judgments of Value for Code Generation ModelsAnnual Meeting of the Association for Computational Linguistics (ACL), 2022
Victor C. Dibia
Adam Fourney
Gagan Bansal
Forough Poursabzi-Sangdeh
Han Liu
Saleema Amershi
ALMOffRL
146
17
0
29 Oct 2022
Multi-lingual Evaluation of Code Generation Models
Multi-lingual Evaluation of Code Generation ModelsInternational Conference on Learning Representations (ICLR), 2022
Ben Athiwaratkun
Sanjay Krishna Gouda
Zijian Wang
Xiaopeng Li
Yuchen Tian
...
Baishakhi Ray
Parminder Bhatia
Sudipta Sengupta
Dan Roth
Bing Xiang
ELM
307
205
0
26 Oct 2022
Piloting Copilot, Codex, and StarCoder2: Hot Temperature, Cold Prompts, or Black Magic?
Piloting Copilot, Codex, and StarCoder2: Hot Temperature, Cold Prompts, or Black Magic?Journal of Systems and Software (JSS), 2022
Jean-Baptiste Döderlein
Nguessan Hermann Kouadio
M. Acher
D. Khelladi
B. Combemale
153
35
0
26 Oct 2022
Reading Between the Lines: Modeling User Behavior and Costs in
  AI-Assisted Programming
Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming
Hussein Mozannar
Gagan Bansal
Adam Fourney
Eric Horvitz
278
137
0
25 Oct 2022
Previous
123...10119
Next