Exploring the Robustness of Large Language Models for Solving
Programming Problems

Exploring the Robustness of Large Language Models for Solving Programming Problems

26 June 2023

Atsushi Shirafuji

Yutaka Watanobe

Makoto Morishita

Papers citing "Exploring the Robustness of Large Language Models for Solving Programming Problems"

13 / 13 papers shown

Title
Evaluating the Effectiveness of Black-Box Prompt Optimization as the Scale of LLMs Continues to Grow Ziyu Zhou Yihang Wu J. Yang Zhan Xiao Rongjun Li LRM 14 0 0 13 May 2025
RobuNFR: Evaluating the Robustness of Large Language Models on Non-Functional Requirements Aware Code Generation Feng Lin Dong Jae Kim Z. Li Jinqiu Yang Tse-Husn Chen AAML 38 0 0 28 Mar 2025
Turbulence: Systematically and Automatically Testing Instruction-Tuned Large Language Models for Code Shahin Honarvar Mark van der Wilk Alastair Donaldson 74 6 0 28 Jan 2025
CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning Hung Le Yue Wang Akhilesh Deepak Gotmare Silvio Savarese S. Hoi SyDa ALM 121 232 0 05 Jul 2022
Automatic Generation of Programming Exercises and Code Explanations using Large Language Models Sami Sarsa Paul Denny Arto Hellas Juho Leinonen ELM 85 338 0 03 Jun 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
A Systematic Evaluation of Large Language Models of Code Frank F. Xu Uri Alon Graham Neubig Vincent J. Hellendoorn ELM ALM 196 624 0 26 Feb 2022
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation Yue Wang Weishi Wang Shafiq R. Joty S. Hoi 204 1,451 0 02 Sep 2021
Measuring Coding Challenge Competence With APPS Dan Hendrycks Steven Basart Saurav Kadavath Mantas Mazeika Akul Arora ... Collin Burns Samir Puranik Horace He D. Song Jacob Steinhardt ELM AIMat ALM 194 614 0 20 May 2021
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation Shuai Lu Daya Guo Shuo Ren Junjie Huang Alexey Svyatkovskiy ... Nan Duan Neel Sundaresan Shao Kun Deng Shengyu Fu Shujie Liu ELM 190 853 0 09 Feb 2021
Memorization vs. Generalization: Quantifying Data Leakage in NLP Performance Evaluation Aparna Elangovan Jiayuan He Karin Verspoor TDI FedML 156 89 0 03 Feb 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 245 1,977 0 31 Dec 2020
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 267 1,798 0 14 Dec 2020