Quark: Controllable Text Generation with Reinforced Unlearning

26 May 2022

Prithviraj Ammanabrolu

Yejin Choi

ArXiv PDF HTML

Papers citing "Quark: Controllable Text Generation with Reinforced Unlearning"

50 / 175 papers shown

Title
SuperHF: Supervised Iterative Learning from Human Feedback Gabriel Mukobi Peter Chatain Su Fong Robert Windesheim Gitta Kutyniok Kush S. Bhatia Silas Alberti ALM 9 6 0 25 Oct 2023
Teaching Language Models to Self-Improve through Interactive Demonstrations Xiao Yu Baolin Peng Michel Galley Jianfeng Gao Zhou Yu LRM ReLM 26 19 0 20 Oct 2023
Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection Akari Asai Zeqiu Wu Yizhong Wang Avirup Sil Hannaneh Hajishirzi RALM 138 600 0 17 Oct 2023
Privacy in Large Language Models: Attacks, Defenses and Future Directions Haoran Li Yulin Chen Jinglong Luo Yan Kang Xiaojin Zhang Qi Hu Chunkit Chan Yangqiu Song PILM 38 39 0 16 Oct 2023
Self-Detoxifying Language Models via Toxification Reversal Chak Tou Leong Yi Cheng Jiashuo Wang Jian Wang Wenjie Li MU 6 27 0 14 Oct 2023
Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model H. Deng Colin Raffel 14 18 0 14 Oct 2023
Large Language Model Unlearning Yuanshun Yao Xiaojun Xu Yang Liu MU 8 108 0 14 Oct 2023
Dont Add, dont Miss: Effective Content Preserving Generation from Pre-Selected Text Spans Aviv Slobodkin Avi Caciularu Eran Hirsch Ido Dagan 18 3 0 13 Oct 2023
Parameter Efficient Multi-task Model Fusion with Partial Linearization A. Tang Li Shen Yong Luo Yibing Zhan Han Hu Bo Du Yixin Chen Dacheng Tao MoMe 11 30 0 07 Oct 2023
Reward Dropout Improves Control: Bi-objective Perspective on Reinforced LM Changhun Lee Chiehyeon Lim 13 0 0 06 Oct 2023
Improving Emotional Expression and Cohesion in Image-Based Playlist Description and Music Topics: A Continuous Parameterization Approach Yuelyu Ji Yuheng Song Wei Wang Ruoyi Xu Zhongqian Xie Huiyun Liu DiffM 12 1 0 02 Oct 2023
Enabling Language Models to Implicitly Learn Self-Improvement Ziqi Wang Le Hou Tianjian Lu Yuexin Wu Yunxuan Li Hongkun Yu Heng Ji ReLM LRM 4 5 0 02 Oct 2023
Parameter-Efficient Tuning Helps Language Model Alignment Tianci Xue Ziqi Wang Heng Ji ALM 23 6 0 01 Oct 2023
Controllable Text Generation with Residual Memory Transformer Hanqing Zhang Sun Si Haiming Wu Dawei Song 14 1 0 28 Sep 2023
The Trickle-down Impact of Reward (In-)consistency on RLHF Lingfeng Shen Sihao Chen Linfeng Song Lifeng Jin Baolin Peng Haitao Mi Daniel Khashabi Dong Yu 13 21 0 28 Sep 2023
Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding Jiacheng Liu Andrew Cohen Ramakanth Pasunuru Yejin Choi Hannaneh Hajishirzi Asli Celikyilmaz 6 22 0 26 Sep 2023
SCREWS: A Modular Framework for Reasoning with Revisions K. Shridhar Harsh Jhamtani Hao Fang Benjamin Van Durme Jason Eisner Patrick Xia KELM LRM 14 14 0 20 Sep 2023
Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models Yangyi Chen Karan Sikka Michael Cogswell Heng Ji Ajay Divakaran LRM 21 24 0 08 Sep 2023
Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties Taylor Sorensen Liwei Jiang Jena D. Hwang Sydney Levine Valentina Pyatkin ... Kavel Rao Chandra Bhagavatula Maarten Sap J. Tasioulas Yejin Choi SLR 8 49 0 02 Sep 2023
Bias and Fairness in Large Language Models: A Survey Isabel O. Gallegos Ryan A. Rossi Joe Barrow Md Mehrab Tanjim Sungchul Kim Franck Dernoncourt Tong Yu Ruiyi Zhang Nesreen Ahmed AILaw 16 478 0 02 Sep 2023
A Survey on Fairness in Large Language Models Yingji Li Mengnan Du Rui Song Xin Wang Ying Wang ALM 35 59 0 20 Aug 2023
Separate the Wheat from the Chaff: Model Deficiency Unlearning via Parameter-Efficient Module Operation Xinshuo Hu Dongfang Li Baotian Hu Zihao Zheng Zhenyu Liu M. Zhang KELM MU 18 26 0 16 Aug 2023
Machine Unlearning: Solutions and Challenges Jie Xu Zihan Wu Cong Wang Xiaohua Jia MU 23 45 0 14 Aug 2023
Proximal Policy Optimization Actual Combat: Manipulating Output Tokenizer Length Miao Fan Chen Hu Shuchang Zhou AAML 14 0 0 10 Aug 2023
Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies Liangming Pan Michael Stephen Saxon Wenda Xu Deepak Nathani Xinyi Wang William Yang Wang KELM LRM 18 200 0 06 Aug 2023
Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges Giorgio Franceschelli Mirco Musolesi AI4CE 16 19 0 31 Jul 2023
On the Effectiveness of Offline RL for Dialogue Response Generation Paloma Sodhi Felix Wu Ethan R. Elenberg Kilian Q. Weinberger Ryan T. McDonald OffRL 8 5 0 23 Jul 2023
FigCaps-HF: A Figure-to-Caption Generative Framework and Benchmark with Human Feedback Ashish Singh Prateek R. Agarwal Zixuan Huang Arpita Singh Tong Yu Sungchul Kim Victor S. Bursztyn N. Vlassis Ryan A. Rossi 20 6 0 20 Jul 2023
The Inner Sentiments of a Thought Christian Gagné Peter Dayan 25 2 0 04 Jul 2023
Learning Profitable NFT Image Diffusions via Multiple Visual-Policy Guided Reinforcement Learning Huiguo He Tianfu Wang Huan Yang Jianlong Fu N. Yuan Jian Yin Hongyang Chao Qi Zhang EGVM 17 9 0 20 Jun 2023
Improving Open Language Models by Learning from Organic Interactions Jing Xu Da Ju Joshua Lane M. Komeili Eric Michael Smith ... Rashel Moritz Sainbayar Sukhbaatar Y-Lan Boureau Jason Weston Kurt Shuster 17 8 0 07 Jun 2023
Click: Controllable Text Generation with Sequence Likelihood Contrastive Learning Chujie Zheng Pei Ke Zheng Zhang Minlie Huang BDL 10 30 0 06 Jun 2023
Fine-Grained Human Feedback Gives Better Rewards for Language Model Training Zeqiu Wu Yushi Hu Weijia Shi Nouha Dziri Alane Suhr Prithviraj Ammanabrolu Noah A. Smith Mari Ostendorf Hannaneh Hajishirzi ALM 14 301 0 02 Jun 2023
Preference-grounded Token-level Guidance for Language Model Fine-tuning Shentao Yang Shujian Zhang Congying Xia Yihao Feng Caiming Xiong Mi Zhou 16 22 0 01 Jun 2023
Less Likely Brainstorming: Using Language Models to Generate Alternative Hypotheses Liyan Tang Yifan Peng Yanshan Wang Ying Ding Greg Durrett Justin F. Rousseau 17 9 0 30 May 2023
Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuning Ximing Lu Faeze Brahman Peter West Jaehun Jang Khyathi Raghavi Chandu ... Bill Yuchen Lin Skyler Hallinan Xiang Ren Sean Welleck Yejin Choi 10 26 0 24 May 2023
Leftover Lunch: Advantage-based Offline Reinforcement Learning for Language Models Ashutosh Baheti Ximing Lu Faeze Brahman Ronan Le Bras Maarten Sap Mark O. Riedl 15 9 0 24 May 2023
Learning from Mistakes via Cooperative Study Assistant for Large Language Models Danqing Wang Lei Li 15 6 0 23 May 2023
AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback Yann Dubois Xuechen Li Rohan Taori Tianyi Zhang Ishaan Gulrajani Jimmy Ba Carlos Guestrin Percy Liang Tatsunori B. Hashimoto ALM 14 531 0 22 May 2023
Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models Guillermo Ortiz-Jiménez Alessandro Favero P. Frossard MoMe 14 103 0 22 May 2023
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing Zhibin Gou Zhihong Shao Yeyun Gong Yelong Shen Yujiu Yang Nan Duan Weizhu Chen KELM LRM 25 350 0 19 May 2023
Language Models Meet World Models: Embodied Experiences Enhance Language Models Jiannan Xiang Tianhua Tao Yi Gu Tianmin Shu Zirui Wang Zichao Yang Zhiting Hu ALM LLMAG LM&Ro CLL 20 93 0 18 May 2023
LeTI: Learning to Generate from Textual Interactions Xingyao Wang Hao Peng Reyhaneh Jabbarvand Heng Ji 14 30 0 17 May 2023
Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-Text Rationales Brihi Joshi Ziyi Liu Sahana Ramnath Aaron Chan Zhewei Tong Shaoliang Nie Qifan Wang Yejin Choi Xiang Ren HAI LRM 16 28 0 11 May 2023
Self-Refine: Iterative Refinement with Self-Feedback Aman Madaan Niket Tandon Prakhar Gupta Skyler Hallinan Luyu Gao ... Bodhisattwa Prasad Majumder Katherine Hermann Sean Welleck Amir Yazdanbakhsh Peter Clark ReLM LRM DiffM 32 1,389 0 30 Mar 2023
Elastic Weight Removal for Faithful and Abstractive Dialogue Generation Nico Daheim Nouha Dziri Mrinmaya Sachan Iryna Gurevych E. Ponti MoMe 15 30 0 30 Mar 2023
Extrapolative Controlled Sequence Generation via Iterative Refinement Vishakh Padmakumar Richard Yuanzhe Pang He He Ankur P. Parikh 11 9 0 08 Mar 2023
Systematic Rectification of Language Models via Dead-end Analysis Mengyao Cao Mehdi Fatemi Jackie C.K. Cheung Samira Shabanian KELM 19 15 0 27 Feb 2023
Guiding Large Language Models via Directional Stimulus Prompting Zekun Li Baolin Peng Pengcheng He Michel Galley Jianfeng Gao Xi Yan LLMAG LRM LM&Ro 21 91 0 22 Feb 2023
Pretraining Language Models with Human Preferences Tomasz Korbak Kejian Shi Angelica Chen Rasika Bhalerao C. L. Buckley Jason Phang Sam Bowman Ethan Perez ALM SyDa 17 205 0 16 Feb 2023