CodeBLEU: a Method for Automatic Evaluation of Code Synthesis

22 September 2020

Papers citing "CodeBLEU: a Method for Automatic Evaluation of Code Synthesis"

50 / 56 papers shown

Title
Prompting LLMs for Code Editing: Struggles and Remedies Daye Nam Ahmed Omran Ambar Murillo Saksham Thakur Abner Araujo Marcel Blistein Alexander Frömmgen Vincent J. Hellendoorn Satish Chandra KELM 66 0 0 28 Apr 2025
Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Tasks Kang Yang Xinjun Mao Shangwen Wang Y. Wang Tanghaoran Zhang Bo Lin Yihao Qin Zhang Zhang Yao Lu Kamal Al-Sabahi ALM 117 1 0 28 Apr 2025
Evaluate-and-Purify: Fortifying Code Language Models Against Adversarial Attacks Using LLM-as-a-Judge Wenhan Mu Ling Xu Shuren Pei Le Mi Huichi Zhou AAML ELM 53 0 0 28 Apr 2025
CodeBC: A More Secure Large Language Model for Smart Contract Code Generation in Blockchain LingXiang Wang Hainan Zhang Qinnan Zhang Ziwei Wang Hongwei Zheng Jin Dong Zhiming Zheng 51 0 0 28 Apr 2025
Enhancing Code LLM Training with Programmer Attention Y. Zhang Chen Huang Z. Karas Dung T. Nguyen Kevin Leach Yu Huang 70 0 0 19 Mar 2025
Unified Modeling Language Code Generation from Diagram Images Using Multimodal Large Language Models Averi Bates Ryan Vavricka Shane Carleton Ruosi Shao Chongle Pan 59 0 0 15 Mar 2025
ResBench: Benchmarking LLM-Generated FPGA Designs with Resource Awareness Ce Guo Tong Zhao 61 1 0 11 Mar 2025
Fully Autonomous Programming using Iterative Multi-Agent Debugging with Large Language Models Anastasiia Grishina Vadim Liventsev Aki Härmä Leon Moonen ELM 77 0 0 10 Mar 2025
How Diversely Can Language Models Solve Problems? Exploring the Algorithmic Diversity of Model-Generated Code Seonghyeon Lee Heejae Chon Joonwon Jang Dongha Lee Hwanjo Yu ALM 39 0 0 02 Mar 2025
Learning Code-Edit Embedding to Model Student Debugging Behavior Hasnain Heickal Andrew Lan 51 0 0 26 Feb 2025
UniGenCoder: Merging Seq2Seq and Seq2Tree Paradigms for Unified Code Generation Liangying Shao Yanfu Yan Denys Poshyvanyk Jinsong Su 36 1 0 18 Feb 2025
Bridging LLM-Generated Code and Requirements: Reverse Generation technique and SBC Metric for Developer Insights Ahilan Ayyachamy Nadar Ponnusamy 58 0 0 11 Feb 2025
Towards Making Flowchart Images Machine Interpretable S. Kamath S Prajwal Gatti Yogesh Kumar Vikash Yadav Anand Mishra 53 5 0 29 Jan 2025
Skeleton-Guided-Translation: A Benchmarking Framework for Code Repository Translation with Fine-Grained Quality Evaluation Xing Zhang Jiaheng Wen Fangkai Yang Pu Zhao Yu Kang ... Qingwei Lin Yingnong Dang Saravan Rajmohan Dongmei Zhang Qi Zhang 53 2 0 28 Jan 2025
RAPGen: An Approach for Fixing Code Inefficiencies in Zero-Shot Spandan Garg Roshanak Zilouchian Moghaddam Neel Sundaresan 59 9 0 10 Jan 2025
Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Survey Junqiao Wang Zeng Zhang Yangfan He Yuyang Song Tianyu Shi ... Hengyuan Xu Kunyu Wu Guangwu Qian Qiuwu Chen Lewei He 38 9 0 03 Jan 2025
CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code Batu Guan Yao Wan Zhangqian Bi Zheng Wang Hongyu Zhang Yulei Sui Pan Zhou 32 8 0 31 Dec 2024
Autonomous Evaluation of LLMs for Truth Maintenance and Reasoning Tasks Rushang Karia Daniel Bramblett D. Dobhal Siddharth Srivastava ELM LRM 30 0 0 11 Oct 2024
Text2Chart31: Instruction Tuning for Chart Generation with Automatic Feedback Fatemeh Pesaran Zadeh Juyeon Kim Jin-Hwa Kim Gunhee Kim ALM 44 1 0 05 Oct 2024
CRScore: Grounding Automated Evaluation of Code Review Comments in Code Claims and Smells Atharva Naik Marcus Alenius Daniel Fried Carolyn Rose 21 0 0 29 Sep 2024
What can Large Language Models Capture about Code Functional Equivalence? Nickil Maveli Antonio Vergari Shay B. Cohen 27 2 0 20 Aug 2024
Applying RLAIF for Code Generation with API-usage in Lightweight LLMs Sujan Dutta Sayantan Mahinder R. Anantha Bortik Bandyopadhyay ALM 34 4 0 28 Jun 2024
Automatically Generating UI Code from Screenshot: A Divide-and-Conquer-Based Approach Yuxuan Wan Chaozheng Wang Yi Dong Wenxuan Wang Shuqing Li Yintong Huo M. Lyu 3DV 69 10 0 24 Jun 2024
From Text to CQL: Bridging Natural Language and Corpus Search Engine Luming Lu Jiyuan An Yujie Wang Liner Yang Cunliang Kong ... Shuo Wang Haozhe Lin Mingwei Fang Yaping Huang Erhong Yang 43 1 0 21 Feb 2024
Text-to-Code Generation with Modality-relative Pre-training Fenia Christopoulou Guchun Zhang Gerasimos Lampouras AI4TS 13 1 0 08 Feb 2024
A Comprehensive Evaluation of Parameter-Efficient Fine-Tuning on Software Engineering Tasks Wentao Zou Qi Li Jidong Ge Chuanyi Li Xiaoyu Shen LiGuo Huang Bin Luo 24 5 0 25 Dec 2023
Xpert: Empowering Incident Management with Query Recommendations via Large Language Models Yuxuan Jiang Chaoyun Zhang Shilin He Zhihao Yang Ming-Jie Ma ... Yu Kang Yingnong Dang Saravan Rajmohan Qingwei Lin Dongmei Zhang 37 17 0 19 Dec 2023
Towards Trustworthy AI Software Development Assistance Daniel Maninger Krishna Narasimhan Mira Mezini 22 3 0 14 Dec 2023
Bias Testing and Mitigation in LLM-based Code Generation Dong Huang Qingwen Bu Jie M. Zhang Xiaofei Xie Junjie Chen Heming Cui 36 20 0 03 Sep 2023
CodeBPE: Investigating Subtokenization Options for Large Language Model Pretraining on Source Code Nadezhda Chirkova Sergey Troshin 21 8 0 01 Aug 2023
Multilingual Code Co-Evolution Using Large Language Models Jiyang Zhang Pengyu Nie Junyi Jessy Li Miloš Gligorić 21 20 0 27 Jul 2023
Neural Machine Translation for Code Generation K. Dharma Clayton T. Morrison 32 4 0 22 May 2023
Stochastic Code Generation Swapnil Sharma Nikita Anand V. KranthiKiranG. SyDa 19 0 0 14 Apr 2023
Towards Generating Functionally Correct Code Edits from Natural Language Issue Descriptions Sarah Fakhoury Saikat Chakraborty Madan Musuvathi Shuvendu K. Lahiri 25 21 0 07 Apr 2023
CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X Qinkai Zheng Xiao Xia Xu Zou Yuxiao Dong Shanshan Wang ... Andi Wang Yang Li Teng Su Zhilin Yang Jie Tang ELM ALM SyDa 50 314 0 30 Mar 2023
JaCoText: A Pretrained Model for Java Code-Text Generation Jessica Nayeli López Espejel Mahaman Sanoussi Yahaya Alassan Walid Dahhane E. Ettifouri 27 3 0 22 Mar 2023
ADELT: Transpilation Between Deep Learning Frameworks Linyuan Gong Jiayi Wang Alvin Cheung 30 3 0 07 Mar 2023
CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code Shuyan Zhou Uri Alon Sumit Agarwal Graham Neubig ELM ALM 22 98 0 10 Feb 2023
Exploring Data Augmentation for Code Generation Tasks Pinzhen Chen Gerasimos Lampouras 29 9 0 05 Feb 2023
ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages Yekun Chai Shuohuan Wang Chao Pang Yu Sun Hao Tian Hua-Hong Wu 24 35 0 13 Dec 2022
A Survey on Natural Language Processing for Programming Qingfu Zhu Xianzhen Luo Fang Liu Cuiyun Gao Wanxiang Che 23 1 0 12 Dec 2022
Execution-based Evaluation for Data Science Code Generation Models Junjie Huang Chenglong Wang Jipeng Zhang Cong Yan Haotian Cui J. Inala Colin B. Clement Nan Duan Jianfeng Gao ELM 25 34 0 17 Nov 2022
CodePAD: Sequence-based Code Generation with Pushdown Automaton Yihong Dong Xue Jiang Yuchen Liu Ge Li Zhi Jin 17 6 0 02 Nov 2022
Exploring Code Style Transfer with Neural Networks Karl Munson Anish Savla Chih-Kai Ting Serenity Wade Kiran Kate Kavitha Srinivas CLIP 8 0 0 13 Sep 2022
MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generation Federico Cassano John Gouwar Daniel Nguyen S. Nguyen Luna Phipps-Costin ... Carolyn Jane Anderson Molly Q. Feldman Arjun Guha Michael Greenberg Abhinav Jangda ELM 22 81 0 17 Aug 2022
PanGu-Coder: Program Synthesis with Function-Level Language Modeling Fenia Christopoulou Gerasimos Lampouras Milan Gritta Guchun Zhang Yinpeng Guo ... Guangtai Liang Jia Wei Xin Jiang Qianxiang Wang Qun Liu ELM SyDa ALM 27 74 0 22 Jul 2022
CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning Hung Le Yue Wang Akhilesh Deepak Gotmare Silvio Savarese S. Hoi SyDa ALM 124 237 0 05 Jul 2022
GitHub Copilot AI pair programmer: Asset or Liability? Arghavan Moradi Dakhel Vahid Majdinasab Amin Nikanjam Foutse Khomh Michel C. Desmarais Zhen Ming Z. Jiang 26 331 0 30 Jun 2022
StructCoder: Structure-Aware Transformer for Code Generation Sindhu Tipirneni Ming Zhu Chandan K. Reddy 28 55 0 10 Jun 2022
CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Models Akshita Jha Chandan K. Reddy SILM ELM AAML 25 58 0 31 May 2022