Benchmarking Language Model Creativity: A Case Study on Code Generation

Benchmarking Language Model Creativity: A Case Study on Code Generation

12 July 2024

Daniel Khashabi

Meng Jiang

Daniel Khashabi

Papers citing "Benchmarking Language Model Creativity: A Case Study on Code Generation"

13 / 13 papers shown

Title
SIMPLEMIX: Frustratingly Simple Mixing of Off- and On-policy Data in Language Model Preference Learning Tianjian Li Daniel Khashabi 48 0 0 05 May 2025
Cooking Up Creativity: A Cognitively-Inspired Approach for Enhancing LLM Creativity through Structured Representations Moran Mizrahi Chen Shani Gabriel Stanovsky Dan Jurafsky Dafna Shahaf 22 0 0 29 Apr 2025
Beyond Memorization: Mapping the Originality-Quality Frontier of Language Models Vishakh Padmakumar Chen Yueh-Han Jane Pan Valerie Chen He He 30 0 0 13 Apr 2025
Has the Creativity of Large-Language Models peaked? An analysis of inter- and intra-LLM variability Jennifer Haase P. Hanel Sebastian Pokutta ALM LRM 50 0 0 10 Apr 2025
How Diversely Can Language Models Solve Problems? Exploring the Algorithmic Diversity of Model-Generated Code Seonghyeon Lee Heejae Chon Joonwon Jang Dongha Lee Hwanjo Yu ALM 39 0 0 02 Mar 2025
LiveIdeaBench: Evaluating LLMs' Divergent Thinking for Scientific Idea Generation with Minimal Context Kai Ruan Xuan Wang Jixiang Hong Hao Sun Yang Liu Hao Sun LRM ELM 32 0 0 23 Dec 2024
CS4: Measuring the Creativity of Large Language Models Automatically by Controlling the Number of Story-Writing Constraints Anirudh Atmakuru Jatin Nainani Rohith Siddhartha Reddy Bheemreddy Anirudh Lakkaraju Zonghai Yao Hamed Zamani Haw-Shiuan Chang 32 2 0 05 Oct 2024
Initial Development and Evaluation of the Creative Artificial Intelligence through Recurring Developments and Determinations (CAIRDD) System Jeremy Straub Zach Johnson 23 0 0 03 Sep 2024
Exploring Precision and Recall to assess the quality and diversity of LLMs Florian Le Bronnec Alexandre Verine Benjamin Négrevergne Y. Chevaleyre Alexandre Allauzen 27 3 0 16 Feb 2024
Understanding the Effects of RLHF on LLM Generalisation and Diversity Robert Kirk Ishita Mediratta Christoforos Nalmpantis Jelena Luketina Eric Hambro Edward Grefenstette Roberta Raileanu AI4CE ALM 95 63 0 10 Oct 2023
Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writers Tuhin Chakrabarty Vishakh Padmakumar Faeze Brahman Smaranda Muresan 41 31 0 22 Sep 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 3,054 0 23 Jan 2020