Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2105.09938
Cited By
v1
v2
v3 (latest)
Measuring Coding Challenge Competence With APPS
20 May 2021
Dan Hendrycks
Steven Basart
Saurav Kadavath
Mantas Mazeika
Akul Arora
Ethan Guo
Collin Burns
Samir Puranik
Horace He
Basel Alomair
Jacob Steinhardt
ELM
AIMat
ALM
Re-assign community
ArXiv (abs)
PDF
HTML
HuggingFace (1 upvotes)
Papers citing
"Measuring Coding Challenge Competence With APPS"
50 / 541 papers shown
Title
Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?
Bonan Kou
Shengmai Chen
Zhijie Wang
Lei Ma
Tianyi Zhang
ALM
140
19
0
02 Jun 2023
Better Context Makes Better Code Language Models: A Case Study on Function Call Argument Completion
AAAI Conference on Artificial Intelligence (AAAI), 2023
Hengzhi Pei
Jinman Zhao
Leonard Lausen
Sheng Zha
George Karypis
ELM
LRM
108
26
0
01 Jun 2023
CodeTF: One-stop Transformer Library for State-of-the-art Code LLM
Nghi D. Q. Bui
Hung Le
Yue Wang
Junnan Li
Akhilesh Deepak Gotmare
Steven C. H. Hoi
169
26
0
31 May 2023
SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Models
Neural Information Processing Systems (NeurIPS), 2023
Hongxin Li
Jingran Su
Yuntao Chen
Qing Li
Zhaoxiang Zhang
LMTD
158
47
0
30 May 2023
ANPL: Towards Natural Programming with Interactive Decomposition
Neural Information Processing Systems (NeurIPS), 2023
Di Huang
Ziyuan Nan
Xingui Hu
Pengwei Jin
Shaohui Peng
...
Rui Zhang
Zidong Du
Qi Guo
Yewen Pu
Yunji Chen
197
13
0
29 May 2023
Demo2Code: From Summarizing Demonstrations to Synthesizing Code via Extended Chain-of-Thought
Neural Information Processing Systems (NeurIPS), 2023
Huaxiaoyue Wang
Gonzalo Gonzalez-Pumariega
Yash Sharma
Sanjiban Choudhury
LM&Ro
213
44
0
26 May 2023
From Words to Wires: Generating Functioning Electronic Devices from Natural Language Descriptions
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Peter Alexander Jansen
172
3
0
24 May 2023
A New Era in Software Security: Towards Self-Healing Software via Large Language Models and Formal Verification
International Conference/Workshop on Automation of Software Test (AST), 2023
Norbert Tihanyi
Ridhi Jain
Yiannis Charalambous
M. Ferrag
Youcheng Sun
Lucas C. Cordeiro
155
70
0
24 May 2023
ALGO: Synthesizing Algorithmic Programs with LLM-Generated Oracle Verifiers
Neural Information Processing Systems (NeurIPS), 2023
Kexun Zhang
Danqing Wang
Jingtao Xia
William Yang Wang
Lei Li
155
53
0
24 May 2023
Neural Machine Translation for Code Generation
K. Dharma
Clayton T. Morrison
160
6
0
22 May 2023
The "code'' of Ethics:A Holistic Audit of AI Code Generators
IEEE Transactions on Dependable and Secure Computing (IEEE TDSC), 2023
Wanlun Ma
Yiliao Song
Minhui Xue
Sheng Wen
Yang Xiang
89
14
0
22 May 2023
Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Pranjal Aggarwal
Aman Madaan
Yiming Yang
Mausam
LRM
213
71
0
19 May 2023
Think Outside the Code: Brainstorming Boosts Large Language Models in Code Generation
Xinyu Li
Jiang-Tian Xue
Zheng Xie
Ming Li
LRM
121
35
0
18 May 2023
CodeT5+: Open Code Large Language Models for Code Understanding and Generation
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Yue Wang
Hung Le
Akhilesh Deepak Gotmare
Nghi D. Q. Bui
Junnan Li
Steven C. H. Hoi
ALM
234
572
0
13 May 2023
The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation
Dũng Nguyễn Mạnh
Nam Le Hai
An Dau
A. Nguyen
Khanh N. Nghiem
Jingnan Guo
Nghi D. Q. Bui
189
21
0
09 May 2023
Self-Edit: Fault-Aware Code Editor for Code Generation
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Kechi Zhang
Zhuo Li
Jia Li
Ge Li
Zhi Jin
306
136
0
06 May 2023
Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generation
International Conference on Machine Learning (ICML), 2023
Wenqing Zheng
S. Sharan
Ajay Jaiswal
Kevin Wang
Yihan Xi
Dejia Xu
Zhangyang Wang
227
31
0
28 Apr 2023
Stochastic Code Generation
Swapnil Sharma
Nikita Anand
V. KranthiKiranG.
SyDa
79
1
0
14 Apr 2023
"What It Wants Me To Say": Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Models
International Conference on Human Factors in Computing Systems (CHI), 2023
Michael Xieyang Liu
Advait Sarkar
Carina Negreanu
B. Zorn
Jack Williams
N. Toronto
Andrew D. Gordon
180
127
0
13 Apr 2023
Teaching Large Language Models to Self-Debug
International Conference on Learning Representations (ICLR), 2023
Xinyun Chen
Maxwell Lin
Nathanael Scharli
Denny Zhou
LRM
365
851
0
11 Apr 2023
CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X
Knowledge Discovery and Data Mining (KDD), 2023
Qinkai Zheng
Xiao Xia
Xu Zou
Yuxiao Dong
Shanshan Wang
...
Andi Wang
Yang Li
Teng Su
Zhilin Yang
Jie Tang
ELM
ALM
SyDa
278
429
0
30 Mar 2023
RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Fengji Zhang
B. Chen
Yue Zhang
Jacky Keung
Jin Liu
Daoguang Zan
Yi Mao
Jian-Guang Lou
Weizhu Chen
168
316
0
22 Mar 2023
PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing
Xiaozhe Ren
Pingyi Zhou
Xinfan Meng
Xinjing Huang
Yadao Wang
...
Jiansheng Wei
Xin Jiang
Teng Su
Qun Liu
Jun Yao
ALM
MoE
144
75
0
20 Mar 2023
Meet in the Middle: A New Pre-training Paradigm
Neural Information Processing Systems (NeurIPS), 2023
A. Nguyen
Nikos Karampatziakis
Weizhu Chen
82
24
0
13 Mar 2023
Planning with Large Language Models for Code Generation
International Conference on Learning Representations (ICLR), 2023
Shun Zhang
Zhenfang Chen
Songlin Yang
Mingyu Ding
J. Tenenbaum
Chuang Gan
157
202
0
09 Mar 2023
Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference
Chi Wang
Susan Liu
Ahmed Hassan Awadallah
142
52
0
08 Mar 2023
xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Mohammad Abdullah Matin Khan
M Saiful Bari
Xuan Long Do
Weishi Wang
Md. Rizwan Parvez
Shafiq Joty
ALM
ELM
298
45
0
06 Mar 2023
Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt Constraints
Findings (Findings), 2023
Albert Lu
Hongxin Zhang
Yanzhe Zhang
Xuezhi Wang
Diyi Yang
LRM
105
37
0
17 Feb 2023
PAC Prediction Sets for Large Language Models of Code
International Conference on Machine Learning (ICML), 2023
Adam Khakhar
Stephen Mell
Osbert Bastani
188
7
0
17 Feb 2023
Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks
Daniel Kang
Xuechen Li
Ion Stoica
Carlos Guestrin
Matei A. Zaharia
Tatsunori Hashimoto
AAML
166
301
0
11 Feb 2023
Measuring The Impact Of Programming Language Distribution
International Conference on Machine Learning (ICML), 2023
Gabriel Orlanski
Kefan Xiao
Xavier Garcia
Jeffrey Hui
Joshua Howland
J. Malmaud
Jacob Austin
Rishah Singh
Michele Catasta
285
41
0
03 Feb 2023
Execution-based Code Generation using Deep Reinforcement Learning
Parshin Shojaee
Aneesh Jain
Sindhu Tipirneni
Chandan K. Reddy
272
79
0
31 Jan 2023
SantaCoder: don't reach for the stars!
Loubna Ben Allal
Raymond Li
Denis Kocetkov
Chenghao Mou
Christopher Akiki
...
Sean M. Hughes
Daniel Fried
Arjun Guha
H. D. Vries
Leandro von Werra
284
220
0
09 Jan 2023
Parsel: Algorithmic Reasoning with Language Models by Composing Decompositions
Neural Information Processing Systems (NeurIPS), 2022
E. Zelikman
Qian Huang
Gabriel Poesia
Noah D. Goodman
Nick Haber
ReLM
LRM
170
67
0
20 Dec 2022
Execution-Based Evaluation for Open-Domain Code Generation
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022
Zhiruo Wang
Shuyan Zhou
Daniel Fried
Graham Neubig
ELM
188
96
0
20 Dec 2022
A Survey on Pretrained Language Models for Neural Code Intelligence
Yichen Xu
Yanqiao Zhu
72
18
0
20 Dec 2022
Large Language Models Meet NL2Code: A Survey
Annual Meeting of the Association for Computational Linguistics (ACL), 2022
Daoguang Zan
B. Chen
Fengji Zhang
Di Lu
Bingchao Wu
Bei Guan
Yongji Wang
Jian-Guang Lou
ELM
ALM
145
222
0
19 Dec 2022
Natural Language to Code Generation in Interactive Data Science Notebooks
Annual Meeting of the Association for Computational Linguistics (ACL), 2022
Pengcheng Yin
Wen-Ding Li
Kefan Xiao
Abhishek Rao
Yeming Wen
...
Paige Bailey
Michele Catasta
Henryk Michalewski
Oleksandr Polozov
Charles Sutton
158
85
0
19 Dec 2022
Plansformer: Generating Symbolic Plans using Transformers
Vishal Pallagani
Bharath Muppasani
K. Murugesan
F. Rossi
L. Horesh
Biplav Srivastava
F. Fabiano
Andrea Loreggia
LM&Ro
LLMAG
OffRL
130
44
0
16 Dec 2022
A Survey on Natural Language Processing for Programming
International Conference on Language Resources and Evaluation (LREC), 2022
Qingfu Zhu
Xianzhen Luo
Fang Liu
Cuiyun Gao
Wanxiang Che
134
5
0
12 Dec 2022
Coder Reviewer Reranking for Code Generation
International Conference on Machine Learning (ICML), 2022
Tianyi Zhang
Tao Yu
Tatsunori B. Hashimoto
M. Lewis
Anuj Kumar
Daniel Fried
Sida I. Wang
184
107
0
29 Nov 2022
The Stack: 3 TB of permissively licensed source code
Denis Kocetkov
Raymond Li
Loubna Ben Allal
Jia Li
Chenghao Mou
...
Sean M. Hughes
Thomas Wolf
Dzmitry Bahdanau
Leandro von Werra
H. D. Vries
154
382
0
20 Nov 2022
DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation
International Conference on Machine Learning (ICML), 2022
Yuhang Lai
Chengxi Li
Yiming Wang
Tianyi Zhang
Ruiqi Zhong
Luke Zettlemoyer
Scott Yih
Daniel Fried
Si-yi Wang
Tao Yu
ELM
ALM
183
411
0
18 Nov 2022
Execution-based Evaluation for Data Science Code Generation Models
Junjie Huang
Chenglong Wang
Jipeng Zhang
Cong Yan
Haotian Cui
J. Inala
Colin B. Clement
Nan Duan
Jianfeng Gao
ELM
166
39
0
17 Nov 2022
A Simple, Yet Effective Approach to Finding Biases in Code Generation
Annual Meeting of the Association for Computational Linguistics (ACL), 2022
Spyridon Mouselinos
Mateusz Malinowski
Henryk Michalewski
207
10
0
31 Oct 2022
When Language Model Meets Private Library
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022
Daoguang Zan
Bei Chen
Zeqi Lin
Bei Guan
Yongji Wang
Jian-Guang Lou
ALM
185
84
0
31 Oct 2022
Aligning Offline Metrics and Human Judgments of Value for Code Generation Models
Annual Meeting of the Association for Computational Linguistics (ACL), 2022
Victor C. Dibia
Adam Fourney
Gagan Bansal
Forough Poursabzi-Sangdeh
Han Liu
Saleema Amershi
ALM
OffRL
146
17
0
29 Oct 2022
Multi-lingual Evaluation of Code Generation Models
International Conference on Learning Representations (ICLR), 2022
Ben Athiwaratkun
Sanjay Krishna Gouda
Zijian Wang
Xiaopeng Li
Yuchen Tian
...
Baishakhi Ray
Parminder Bhatia
Sudipta Sengupta
Dan Roth
Bing Xiang
ELM
307
205
0
26 Oct 2022
Piloting Copilot, Codex, and StarCoder2: Hot Temperature, Cold Prompts, or Black Magic?
Journal of Systems and Software (JSS), 2022
Jean-Baptiste Döderlein
Nguessan Hermann Kouadio
M. Acher
D. Khelladi
B. Combemale
153
35
0
26 Oct 2022
Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming
Hussein Mozannar
Gagan Bansal
Adam Fourney
Eric Horvitz
278
137
0
25 Oct 2022
Previous
1
2
3
...
10
11
9
Next