Benchmarks and Metrics for Evaluations of Code Generation: A Critical Review

18 June 2024

Papers citing "Benchmarks and Metrics for Evaluations of Code Generation: A Critical Review"

5 / 5 papers shown

Title
LLMs Love Python: A Study of LLMs' Bias for Programming Languages and Libraries Lukas Twist Jie M. Zhang Mark Harman Don Syme Joost Noppen Detlef Nauck 39 0 0 21 Mar 2025
Large Language Models for Code Generation: A Comprehensive Survey of Challenges, Techniques, Evaluation, and Applications Nam Huynh Beiyu Lin LM&MA 69 18 0 03 Mar 2025
Programming with AI: Evaluating ChatGPT, Gemini, AlphaCode, and GitHub Copilot for Programmers Md Kamrul Siam Huanying Gu Jerry Q. Cheng ELM 28 5 0 14 Nov 2024
Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation Jiawei Liu Chun Xia Yuyao Wang Lingming Zhang ELM ALM 163 388 0 02 May 2023
Measuring Coding Challenge Competence With APPS Dan Hendrycks Steven Basart Saurav Kadavath Mantas Mazeika Akul Arora ... Collin Burns Samir Puranik Horace He D. Song Jacob Steinhardt ELM AIMat ALM 189 614 0 20 May 2021