Contrastive Decoding: Open-ended Text Generation as Optimization

27 October 2022

Xiang Lisa Li

Ari Holtzman

Daniel Fried

Percy Liang

Jason Eisner

Tatsunori Hashimoto

Luke Zettlemoyer

M. Lewis

ArXiv PDF HTML

Papers citing "Contrastive Decoding: Open-ended Text Generation as Optimization"

50 / 262 papers shown

Title
Decoding with Limited Teacher Supervision Requires Understanding When to Trust the Teacher Hyunjong Ok Jegwang Ryu Jaeho Lee 37 0 0 26 Jun 2024
Multi-property Steering of Large Language Models with Dynamic Activation Composition Daniel Scalena Gabriele Sarti Malvina Nissim KELM LLMSV AI4CE 19 13 0 25 Jun 2024
Entropy-Based Decoding for Retrieval-Augmented Large Language Models Zexuan Qiu Zijing Ou Bin Wu Jingjing Li Aiwei Liu Irwin King KELM RALM 33 5 0 25 Jun 2024
From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models Sean Welleck Amanda Bertsch Matthew Finlayson Hailey Schoelkopf Alex Xie Graham Neubig Ilia Kulikov Zaid Harchaoui 33 47 0 24 Jun 2024
Adversarial Contrastive Decoding: Boosting Safety Alignment of Large Language Models via Opposite Prompt Optimization Zhengyue Zhao Xiaoyun Zhang Kaidi Xu Xing Hu Rui Zhang Zidong Du Qi Guo Yunji Chen 22 5 0 24 Jun 2024
On the Transformations across Reward Model, Parameter Update, and In-Context Prompt Deng Cai Huayang Li Tingchen Fu Siheng Li Weiwen Xu ... Leyang Cui Yan Wang Lemao Liu Taro Watanabe Shuming Shi KELM 26 2 0 24 Jun 2024
FastMem: Fast Memorization of Prompt Improves Context Awareness of Large Language Models Junyi Zhu Shuochen Liu Yu Yu Bo Tang Yibo Yan Zhiyu Li Feiyu Xiong Tong Xu Matthew B. Blaschko 42 3 0 23 Jun 2024
BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions Terry Yue Zhuo Minh Chien Vu Jenny Chim Han Hu Wenhao Yu ... David Lo Daniel Fried Xiaoning Du H. D. Vries Leandro von Werra 65 128 0 22 Jun 2024
Adaptive Token Biaser: Knowledge Editing via Biasing Key Entities Baolong Bi Shenghua Liu Yiwei Wang Lingrui Mei Hongcheng Gao Yilong Xu Xueqi Cheng KELM 38 11 0 18 Jun 2024
Fast and Slow Generating: An Empirical Study on Large and Small Language Models Collaborative Decoding Kaiyan Zhang Jianyu Wang Ning Ding Biqing Qi Ermo Hua Xingtai Lv Bowen Zhou 39 8 0 18 Jun 2024
Mitigating Large Language Model Hallucination with Faithful Finetuning Minda Hu Bowei He Yufei Wang Liangyou Li Chen-li Ma Irwin King HILM 33 6 0 17 Jun 2024
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion Chenghao Fan Zhenyi Lu Wei Wei Jie Tian Xiaoye Qu Dangyang Chen Yu Cheng MoMe 44 5 0 17 Jun 2024
Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference Jiabao Ji Yujian Liu Yang Zhang Gaowen Liu Ramana Rao Kompella Sijia Liu Shiyu Chang KELM MU 29 21 0 12 Jun 2024
REAL Sampling: Boosting Factuality and Diversity of Open-Ended Generation via Asymptotic Entropy Haw-Shiuan Chang Nanyun Peng Mohit Bansal Anil Ramakrishna Tagyoung Chung HILM 33 2 0 11 Jun 2024
Semantically Diverse Language Generation for Uncertainty Estimation in Language Models L. Aichberger Kajetan Schweighofer Mykyta Ielanskyi Sepp Hochreiter HILM 23 10 0 06 Jun 2024
Outdated Issue Aware Decoding for Reasoning Questions on Edited Knowledge Zengkui Sun Yijin Liu Jiaan Wang Fandong Meng Jinan Xu Yufeng Chen Jie Zhou KELM 33 0 0 05 Jun 2024
Guiding a Diffusion Model with a Bad Version of Itself Tero Karras M. Aittala Tuomas Kynkaanniemi J. Lehtinen Timo Aila S. Laine 28 57 0 04 Jun 2024
Diver: Large Language Model Decoding with Span-Level Mutual Information Verification Jinliang Lu Chen Wang Jiajun Zhang 48 3 0 04 Jun 2024
CODE: Contrasting Self-generated Description to Combat Hallucination in Large Multi-modal Models Junho Kim Hyunjun Kim Yeonju Kim Yong Man Ro MLLM 34 10 0 04 Jun 2024
Latent Logic Tree Extraction for Event Sequence Explanation from LLMs Zitao Song Chao Yang Chaojie Wang Bo An Shuang Li 40 4 0 03 Jun 2024
FOCUS: Forging Originality through Contrastive Use in Self-Plagiarism for Language Models Kaixin Lan Tao Fang Derek F. Wong Yabo Xu Lidia S. Chao Cecilia G. Zhao 53 4 0 02 Jun 2024
Expert-Guided Extinction of Toxic Tokens for Debiased Generation Xueyao Sun Kaize Shi Haoran Tang Guandong Xu Qing Li MU 38 1 0 29 May 2024
Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignment Xin Xiao Bohong Wu Jiacong Wang Chunyuan Li Xun Zhou Haoyuan Guo VLM 34 7 0 28 May 2024
RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in LVLMs Sangmin Woo Jaehyuk Jang Donguk Kim Yubin Choi Changick Kim 34 0 0 28 May 2024
SED: Self-Evaluation Decoding Enhances Large Language Models for Better Generation Ziqin Luo Haixia Han Haokun Zhao Guochao Jiang Chengyu Du Tingyun Li Jiaqing Liang Deqing Yang Yanghua Xiao 46 3 0 26 May 2024
Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges Jonas Becker Jan Philip Wahle Bela Gipp Terry Ruas 23 9 0 24 May 2024
Alleviating Hallucinations in Large Vision-Language Models through Hallucination-Induced Optimization Beitao Chen Xinyu Lyu Lianli Gao Jingkuan Song Hengtao Shen MLLM 54 10 0 24 May 2024
Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrast Chufan Shi Cheng Yang Xinyu Zhu Jiahao Wang Taiqiang Wu Siheng Li Deng Cai Yujiu Yang Yu Meng MoE 45 9 0 23 May 2024
JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Models Kun Zhou Beichen Zhang Jiapeng Wang Zhipeng Chen Wayne Xin Zhao Jing Sha Zhichao Sheng Shijin Wang Ji-Rong Wen SyDa LRM 33 29 0 23 May 2024
Decoding by Contrasting Knowledge: Enhancing LLMs' Confidence on Edited Facts Baolong Bi Shenghua Liu Lingrui Mei Yiwei Wang Pengliang Ji Xueqi Cheng KELM 38 27 0 19 May 2024
Spectral Editing of Activations for Large Language Model Alignment Yifu Qiu Zheng Zhao Yftah Ziser Anna Korhonen E. Ponti Shay B. Cohen KELM LLMSV 23 15 0 15 May 2024
RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors Liam Dugan Alyssa Hwang Filip Trhlik Josh Magnus Ludan Andrew Zhu Hainiu Xu Daphne Ippolito Christopher Callison-Burch DeLMO AAML 33 40 0 13 May 2024
Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding Zheng Zhao Emilio Monti Jens Lehmann H. Assem 42 22 0 04 May 2024
Hallucination of Multimodal Large Language Models: A Survey Zechen Bai Pichao Wang Tianjun Xiao Tong He Zongbo Han Zheng Zhang Mike Zheng Shou VLM LRM 80 139 0 29 Apr 2024
Beyond the Speculative Game: A Survey of Speculative Execution in Large Language Models Chen Zhang Zhuorui Liu Dawei Song LRM 33 3 0 23 Apr 2024
Offset Unlearning for Large Language Models James Y. Huang Wenxuan Zhou Fei Wang Fred Morstatter Sheng Zhang Hoifung Poon Muhao Chen MU 22 13 0 17 Apr 2024
DESTEIN: Navigating Detoxification of Language Models via Universal Steering Pairs and Head-wise Activation Fusion Yu Li Zhihua Wei Han Jiang Chuanyang Gong LLMSV 23 2 0 16 Apr 2024
Entropy Guided Extrapolative Decoding to Improve Factuality in Large Language Models Souvik Das Lifeng Jin Linfeng Song Haitao Mi Baolin Peng Dong Yu HILM 40 2 0 14 Apr 2024
Exploring and Improving Drafts in Blockwise Parallel Decoding Taehyeon Kim A. Suresh Kishore Papineni Michael Riley Sanjiv Kumar Adrian Benton AI4TS 47 2 0 14 Apr 2024
Rho-1: Not All Tokens Are What You Need Zheng-Wen Lin Zhibin Gou Yeyun Gong Xiao Liu Yelong Shen ... Chen Lin Yujiu Yang Jian Jiao Nan Duan Weizhu Chen CLL 46 55 0 11 Apr 2024
CodecLM: Aligning Language Models with Tailored Synthetic Data Zifeng Wang Chun-Liang Li Vincent Perot Long T. Le Jin Miao Zizhao Zhang Chen-Yu Lee Tomas Pfister SyDa ALM 23 17 0 08 Apr 2024
Is Factuality Decoding a Free Lunch for LLMs? Evaluation on Knowledge Editing Benchmark Baolong Bi Shenghua Liu Yiwei Wang Lingrui Mei Xueqi Cheng KELM 41 9 0 30 Mar 2024
Learning From Correctness Without Prompting Makes LLM Efficient Reasoner Yuxuan Yao Han Wu Zhijiang Guo Biyan Zhou Jiahui Gao Sichun Luo Hanxu Hou Xiaojin Fu Linqi Song LLMAG LRM 40 9 0 28 Mar 2024
Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding Xintong Wang Jingheng Pan Liang Ding Christian Biemann MLLM 32 52 0 27 Mar 2024
Monotonic Paraphrasing Improves Generalization of Language Model Prompting Qin Liu Fei Wang Nan Xu Tianyi Yan Tao Meng Muhao Chen LRM 38 7 0 24 Mar 2024
Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination Dingchen Yang Bowen Cao Guang Chen Changjun Jiang 46 7 0 21 Mar 2024
Multi-Modal Hallucination Control by Visual Information Grounding Alessandro Favero L. Zancato Matthew Trager Siddharth Choudhary Pramuditha Perera Alessandro Achille Ashwin Swaminathan Stefano Soatto MLLM 87 62 0 20 Mar 2024
Knowledge Conflicts for LLMs: A Survey Rongwu Xu Zehan Qi Zhijiang Guo Cunxiang Wang Hongru Wang Yue Zhang Wei Xu 198 91 0 13 Mar 2024
Debiasing Multimodal Large Language Models Yi-Fan Zhang Weichen Yu Qingsong Wen Xue Wang Zhang Zhang Liang Wang Rong Jin Tien-Ping Tan 39 4 0 08 Mar 2024
Learning to Decode Collaboratively with Multiple Language Models Zejiang Shen Hunter Lang Bailin Wang Yoon Kim David Sontag 41 28 0 06 Mar 2024