Contrastive Decoding: Open-ended Text Generation as Optimization

27 October 2022

Xiang Lisa Li

Ari Holtzman

Daniel Fried

Percy Liang

Jason Eisner

Tatsunori Hashimoto

Luke Zettlemoyer

M. Lewis

ArXiv PDF HTML

Papers citing "Contrastive Decoding: Open-ended Text Generation as Optimization"

50 / 262 papers shown

Title
CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following Kaiyan Zhang Jianyu Wang Ermo Hua Biqing Qi Ning Ding Bowen Zhou SyDa 30 20 0 05 Mar 2024
Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Training David Wan Jaemin Cho Elias Stengel-Eskin Mohit Bansal VLM ObjD 51 29 0 04 Mar 2024
In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation Shiqi Chen Miao Xiong Junteng Liu Zhengxuan Wu Teng Xiao Siyang Gao Junxian He HILM 51 21 0 03 Mar 2024
HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding Zhaorun Chen Zhuokai Zhao Hongyin Luo Huaxiu Yao Bo Li Jiawei Zhou MLLM 46 57 0 01 Mar 2024
Standardizing the Measurement of Text Diversity: A Tool and a Comparative Analysis of Scores Chantal Shaib Joe Barrow Jiuding Sun Alexa F. Siu Byron C. Wallace A. Nenkova 66 31 0 01 Mar 2024
IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding Lanyun Zhu Deyi Ji Tianrun Chen Peng Xu Jieping Ye Jun Liu MLLM 39 41 0 28 Feb 2024
Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models Zhuoran Jin Pengfei Cao Hongbang Yuan Yubo Chen Jiexin Xu Huaijun Li Xiaojian Jiang Kang Liu Jun Zhao 178 34 0 28 Feb 2024
Collaborative decoding of critical tokens for boosting factuality of large language models Lifeng Jin Baolin Peng Linfeng Song Haitao Mi Ye Tian Dong Yu HILM 16 6 0 28 Feb 2024
TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space Shaolei Zhang Tian Yu Yang Feng HILM KELM 29 39 0 27 Feb 2024
Tug-of-War Between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Models Zhuoran Jin Pengfei Cao Yubo Chen Kang Liu Xiaojian Jiang Jiexin Xu Qiuxia Li Jun Zhao 193 43 0 22 Feb 2024
Qsnail: A Questionnaire Dataset for Sequential Question Generation Yan Lei Liang Pang Yuanzhuo Wang Huawei Shen Xueqi Cheng 27 0 0 22 Feb 2024
Distillation Contrastive Decoding: Improving LLMs Reasoning with Contrastive Decoding and Distillation Phuc Phan Hieu Tran Long Phan 21 6 0 21 Feb 2024
Contrastive Prompts Improve Disentanglement in Text-to-Image Diffusion Models C. Wu Fernando De la Torre DiffM 24 2 0 21 Feb 2024
Emulated Disalignment: Safety Alignment for Large Language Models May Backfire! Zhanhui Zhou Jie Liu Zhichen Dong Jiaheng Liu Chao Yang Wanli Ouyang Yu Qiao 15 16 0 19 Feb 2024
Discerning and Resolving Knowledge Conflicts through Adaptive Decoding with Contextual Information-Entropy Constraint Xiaowei Yuan Zhao Yang Yequan Wang Shengping Liu Jun Zhao Kang Liu 21 7 0 19 Feb 2024
ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao LM&MA 37 22 0 19 Feb 2024
Chain-of-Thought Reasoning Without Prompting Xuezhi Wang Denny Zhou ReLM LRM 144 99 0 15 Feb 2024
ICDPO: Effectively Borrowing Alignment Capability of Others via In-context Direct Preference Optimization Feifan Song Yuxuan Fan Xin Zhang Peiyi Wang Houfeng Wang 30 8 0 14 Feb 2024
Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation Xiaoying Zhang Baolin Peng Ye Tian Jingyan Zhou Lifeng Jin Linfeng Song Haitao Mi Helen Meng HILM 28 43 0 14 Feb 2024
SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding Zhangchen Xu Fengqing Jiang Luyao Niu Jinyuan Jia Bill Yuchen Lin Radha Poovendran AAML 129 85 0 14 Feb 2024
Permute-and-Flip: An optimally stable and watermarkable decoder for LLMs Xuandong Zhao Lei Li Yu-Xiang Wang 55 12 0 08 Feb 2024
Decoding-time Realignment of Language Models Tianlin Liu Shangmin Guo Leonardo Bianco Daniele Calandriello Quentin Berthet Felipe Llinares-López Jessica Hoffmann Lucas Dixon Michal Valko Mathieu Blondel AI4CE 54 35 0 05 Feb 2024
Improving Dialog Safety using Socially Aware Contrastive Learning Souvik Das R. Srihari 11 1 0 01 Feb 2024
Navigating the OverKill in Large Language Models Chenyu Shi Xiao Wang Qiming Ge Songyang Gao Xianjun Yang Tao Gui Qi Zhang Xuanjing Huang Xun Zhao Dahua Lin 11 11 0 31 Jan 2024
Weak-to-Strong Jailbreaking on Large Language Models Xuandong Zhao Xianjun Yang Tianyu Pang Chao Du Lei Li Yu-Xiang Wang William Yang Wang 26 52 0 30 Jan 2024
ARGS: Alignment as Reward-Guided Search Maxim Khanov Jirayu Burapacheep Yixuan Li 26 44 0 23 Jan 2024
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text Abhimanyu Hans Avi Schwarzschild Valeriia Cherepanova Hamid Kazemi Aniruddha Saha Micah Goldblum Jonas Geiping Tom Goldstein DeLMO 26 81 0 22 Jan 2024
Linear Alignment: A Closed-form Solution for Aligning Human Preferences without Tuning and Feedback Songyang Gao Qiming Ge Wei Shen Shihan Dou Junjie Ye ... Yicheng Zou Zhi Chen Hang Yan Qi Zhang Dahua Lin 47 10 0 21 Jan 2024
Tuning Language Models by Proxy Alisa Liu Xiaochuang Han Yizhong Wang Yulia Tsvetkov Yejin Choi Noah A. Smith ALM 36 44 0 16 Jan 2024
Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding Heming Xia Zhe Yang Qingxiu Dong Peiyi Wang Yongqi Li Tao Ge Tianyu Liu Wenjie Li Zhifang Sui LRM 22 97 0 15 Jan 2024
Parameter-Efficient Detoxification with Contrastive Decoding Tong Niu Caiming Xiong Semih Yavuz Yingbo Zhou 12 12 0 13 Jan 2024
WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledge Wenbin Wang Liang Ding Li Shen Yong Luo Han Hu Dacheng Tao 35 11 0 12 Jan 2024
SH2: Self-Highlighted Hesitation Helps You Decode More Truthfully Jushi Kai Hai Hu Zhouhan Lin HILM 20 7 0 11 Jan 2024
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems Tianyu Cui Yanling Wang Chuanpu Fu Yong Xiao Sijia Li ... Junwu Xiong Xinyu Kong Zujie Wen Ke Xu Qi Li 55 56 0 11 Jan 2024
Supervised Knowledge Makes Large Language Models Better In-context Learners Linyi Yang Shuibai Zhang Zhuohao Yu Guangsheng Bao Yidong Wang ... Ruochen Xu Weirong Ye Xing Xie Weizhu Chen Yue Zhang 16 14 0 26 Dec 2023
Alleviating Hallucinations of Large Language Models through Induced Hallucinations Yue Zhang Leyang Cui Wei Bi Shuming Shi HILM 34 49 0 25 Dec 2023
Reducing LLM Hallucinations using Epistemic Neural Networks Shreyas Verma Kien Tran Yusuf Ali Guangyu Min 35 7 0 25 Dec 2023
Context-aware Decoding Reduces Hallucination in Query-focused Summarization Zhichao Xu HILM 11 10 0 21 Dec 2023
SymNoise: Advancing Language Model Fine-tuning with Symmetric Noise A. Yadav Arjun Singh 41 2 0 03 Dec 2023
Applying Large Language Models and Chain-of-Thought for Automatic Scoring Gyeong-Geon Lee Ehsan Latif Xuansheng Wu Ninghao Liu Xiaoming Zhai 31 86 0 30 Nov 2023
I Know You Did Not Write That! A Sampling Based Watermarking Method for Identifying Machine Generated Text Kaan Efe Keles Ömer Kaan Gürbüz Mucahid Kutlu WaLM 13 1 0 29 Nov 2023
Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding Sicong Leng Hang Zhang Guanzheng Chen Xin Li Shijian Lu Chunyan Miao Li Bing VLM MLLM 85 197 0 28 Nov 2023
IG Captioner: Information Gain Captioners are Strong Zero-shot Classifiers Chenglin Yang Siyuan Qiao Yuan Cao Yu Zhang Tao Zhu Alan L. Yuille Jiahui Yu VLM 11 3 0 27 Nov 2023
Deficiency of Large Language Models in Finance: An Empirical Examination of Hallucination Haoqiang Kang Xiao-Yang Liu RALM 31 26 0 27 Nov 2023
The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text Yanzhu Guo Guokan Shang Michalis Vazirgiannis Chloé Clavel 21 47 0 16 Nov 2023
Ever: Mitigating Hallucination in Large Language Models through Real-Time Verification and Rectification Haoqiang Kang Juntong Ni Huaxiu Yao HILM LRM 19 33 0 15 Nov 2023
Speculative Contrastive Decoding Hongyi Yuan Keming Lu Fei Huang Zheng Yuan Chang Zhou 33 5 0 15 Nov 2023
Anti-LM Decoding for Zero-shot In-context Machine Translation Suzanna Sia Alexandra DeLucia Kevin Duh 24 2 0 14 Nov 2023
Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learning Yue Yu Jiaming Shen Tianqi Liu Zhen Qin Jing Nathan Yan Jialu Liu Chao Zhang Michael Bendersky 44 6 0 13 Nov 2023
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions Lei Huang Weijiang Yu Weitao Ma Weihong Zhong Zhangyin Feng ... Qianglong Chen Weihua Peng Xiaocheng Feng Bing Qin Ting Liu LRM HILM 31 714 0 09 Nov 2023