Enhancing Chat Language Models by Scaling High-quality Instructional Conversations

23 May 2023

Zhiyuan Liu

Maosong Sun

Bowen Zhou

ALM

ArXiv PDF HTML

Papers citing "Enhancing Chat Language Models by Scaling High-quality Instructional Conversations"

50 / 376 papers shown

Title
Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing Zhangchen Xu Fengqing Jiang Luyao Niu Yuntian Deng Radha Poovendran Yejin Choi Bill Yuchen Lin SyDa 27 110 0 12 Jun 2024
Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoning Jaehyun Nam Kyuyoung Kim Seunghyuk Oh Jihoon Tack Jaehyung Kim Jinwoo Shin LMTD 28 9 0 12 Jun 2024
Let's Go Real Talk: Spoken Dialogue Model for Face-to-Face Conversation Se Jin Park Chae Won Kim Hyeongseop Rha Minsu Kim Joanna Hong Jeong Hun Yeo Yong Man Ro CVBM AuLLM 37 6 0 12 Jun 2024
Fine-tuning with HED-IT: The impact of human post-editing for dialogical language models Daniela Occhipinti Michele Marchi Irene Mondella Huiyuan Lai F. Dell’Orletta Malvina Nissim Marco Guerini 18 1 0 11 Jun 2024
MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs Vera Neplenbroek Arianna Bisazza Raquel Fernández 29 6 0 11 Jun 2024
CoEvol: Constructing Better Responses for Instruction Finetuning through Multi-Agent Cooperation Renhao Li Minghuan Tan Derek F. Wong Min Yang LLMAG 19 1 0 11 Jun 2024
Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactions Cheng Tan Dongxin Lyu Siyuan Li Zhangyang Gao Jingxuan Wei Siqi Ma Zicheng Liu Stan Z. Li LLMAG 32 8 0 09 Jun 2024
Improving Alignment and Robustness with Circuit Breakers Andy Zou Long Phan Justin Wang Derek Duenas Maxwell Lin Maksym Andriushchenko Rowan Wang Zico Kolter Matt Fredrikson Dan Hendrycks AAML 36 70 0 06 Jun 2024
Self-Play with Adversarial Critic: Provable and Scalable Offline Alignment for Language Models Xiang Ji Sanjeev Kulkarni Mengdi Wang Tengyang Xie OffRL 27 4 0 06 Jun 2024
UltraMedical: Building Specialized Generalists in Biomedicine Kaiyan Zhang Sihang Zeng Ermo Hua Ning Ding Zhang-Ren Chen ... Xuekai Zhu Xingtai Lv Hu Jinfang Zhiyuan Liu Bowen Zhou LM&MA 39 19 0 06 Jun 2024
Wings: Learning Multimodal LLMs without Text-only Forgetting Yi-Kai Zhang Shiyin Lu Yang Li Yanqing Ma Qing-Guo Chen Zhao Xu Weihua Luo Kaifu Zhang De-Chuan Zhan Han-Jia Ye VLM 33 6 0 05 Jun 2024
Phased Instruction Fine-Tuning for Large Language Models Wei Pang Chuan Zhou Xiao-Hua Zhou Xiaojie Wang ALM 20 4 0 01 Jun 2024
Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Training Maximillian Chen Ruoxi Sun Sercan Ö. Arik Tomas Pfister LLMAG 24 6 0 31 May 2024
TS-Align: A Teacher-Student Collaborative Framework for Scalable Iterative Finetuning of Large Language Models Chen Zhang Chengguang Tang Dading Chong Ke Shi Guohua Tang Feng Jiang Haizhou Li 27 4 0 30 May 2024
MathChat: Benchmarking Mathematical Reasoning and Instruction Following in Multi-Turn Interactions Zhenwen Liang Dian Yu Wenhao Yu Wenlin Yao Zhihan Zhang Xiangliang Zhang Dong Yu LRM 35 9 0 29 May 2024
Evaluating Vision-Language Models on Bistable Images Artemis Panagopoulou Coby Melkin Chris Callison-Burch 39 0 0 29 May 2024
Self-Exploring Language Models: Active Preference Elicitation for Online Alignment Shenao Zhang Donghan Yu Hiteshi Sharma Ziyi Yang Shuohang Wang Hany Hassan Zhaoran Wang LRM 35 28 0 29 May 2024
Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment Jiaxiang Li Siliang Zeng Hoi-To Wai Chenliang Li Alfredo García Mingyi Hong 57 15 0 28 May 2024
Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models Chia-Yi Hsu Yu-Lin Tsai Chih-Hsun Lin Pin-Yu Chen Chia-Mu Yu Chun-ying Huang 42 30 0 27 May 2024
Triple Preference Optimization: Achieving Better Alignment with Less Data in a Single Step Optimization Amir Saeidi Shivanshu Verma Aswin Rrv Chitta Baral 27 5 0 26 May 2024
Automatically Generating Numerous Context-Driven SFT Data for LLMs across Diverse Granularity Shanghaoran Quan 27 3 0 26 May 2024
Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer Zhihan Liu Miao Lu Shenao Zhang Boyi Liu Hongyi Guo Yingxiang Yang Jose H. Blanchet Zhaoran Wang 33 41 0 26 May 2024
Efficient Adversarial Training in LLMs with Continuous Attacks Sophie Xhonneux Alessandro Sordoni Stephan Günnemann Gauthier Gidel Leo Schwinn AAML 37 43 0 24 May 2024
Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvement Xiyao Wang Jiuhai Chen Zhaoyang Wang Yuhang Zhou Yiyang Zhou ... Tianyi Zhou Tom Goldstein Parminder Bhatia Furong Huang Cao Xiao 58 33 0 24 May 2024
SimPO: Simple Preference Optimization with a Reference-Free Reward Yu Meng Mengzhou Xia Danqi Chen 57 335 0 23 May 2024
Annotation-Efficient Preference Optimization for Language Model Alignment Yuu Jinnai Ukyo Honda 38 0 0 22 May 2024
Disperse-Then-Merge: Pushing the Limits of Instruction Tuning via Alignment Tax Reduction Tingchen Fu Deng Cai Lemao Liu Shuming Shi Rui Yan MoMe 45 13 0 22 May 2024
360Zhinao Technical Report 360Zhinao Team 32 0 0 22 May 2024
Intuitive Fine-Tuning: Towards Simplifying Alignment into a Single Process Ermo Hua Biqing Qi Kaiyan Zhang Yue Yu Ning Ding Xingtai Lv Kai Tian Bowen Zhou 32 3 0 20 May 2024
Dynamic data sampler for cross-language transfer learning in large language models Yudong Li Yuhao Feng Wen Zhou Zhe Zhao Linlin Shen Cheng-An Hou Xianxu Hou 42 0 0 17 May 2024
CinePile: A Long Video Question Answering Dataset and Benchmark Ruchit Rawal Khalid Saifullah Ronen Basri David Jacobs Gowthami Somepalli Tom Goldstein 38 39 0 14 May 2024
LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Play Li-Chun Lu Shou-Jen Chen Tsung-Min Pai Chan-Hung Yu Hung-yi Lee Shao-Hua Sun LLMAG 43 38 0 10 May 2024
Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment Abhinav Agarwalla Abhay Gupta Alexandre Marques Shubhra Pandit Michael Goin ... Tuan Nguyen Mahmoud Salem Dan Alistarh Sean Lie Mark Kurtz MoE SyDa 23 11 0 06 May 2024
CRAFT: Extracting and Tuning Cultural Instructions from the Wild Bin Wang Geyu Lin Zhengyuan Liu Chengwei Wei Nancy F. Chen 29 3 0 06 May 2024
Aloe: A Family of Fine-tuned Open Healthcare LLMs Ashwin Kumar Gururajan Enrique Lopez-Cuena Jordi Bayarri-Planas Adrián Tormos Daniel Hinjos ... Lucia Urcelay-Ganzabal Marta Gonzalez-Mallo Sergio Álvarez Napagao Eduard Ayguadé-Parra Ulises Cortés Dario Garcia-Gasulla ELM LM&MA 24 12 0 03 May 2024
D2PO: Discriminator-Guided DPO with Response Evaluation Models Prasann Singhal Nathan Lambert S. Niekum Tanya Goyal Greg Durrett OffRL EGVM 36 4 0 02 May 2024
NeMo-Aligner: Scalable Toolkit for Efficient Model Alignment Gerald Shen Zhilin Wang Olivier Delalleau Jiaqi Zeng Yi Dong ... Sahil Jain Ali Taghibakhshi Markel Sanz Ausin Ashwath Aithal Oleksii Kuchaiev 24 13 0 02 May 2024
It's Difficult to be Neutral -- Human and LLM-based Sentiment Annotation of Patient Comments Petter Maehlum David Samuel R. Norman Elma Jelin Oyvind Bjertnaes Lilja Ovrelid Erik Velldal 26 3 0 29 Apr 2024
Mixture-of-Instructions: Aligning Large Language Models via Mixture Prompting Bowen Xu Shaoyu Wu Kai Liu Lulu Hu 26 1 0 29 Apr 2024
PatentGPT: A Large Language Model for Intellectual Property Zilong Bai Ruiji Zhang Linqing Chen Qijun Cai Yuan Zhong ... Fu Bian Xiaolong Gu Lisha Zhang Weilei Wang Changyang Tu 41 3 0 28 Apr 2024
Insights into Alignment: Evaluating DPO and its Variants Across Multiple Tasks Amir Saeidi Shivanshu Verma Chitta Baral Chitta Baral ALM 30 21 0 23 Apr 2024
A Survey on Self-Evolution of Large Language Models Zhengwei Tao Ting-En Lin Xiancai Chen Hangyu Li Yuchuan Wu Yongbin Li Zhi Jin Fei Huang Dacheng Tao Jingren Zhou LRM LM&Ro 46 21 0 22 Apr 2024
SnapKV: LLM Knows What You are Looking for Before Generation Yuhong Li Yingbing Huang Bowen Yang Bharat Venkitesh Acyr F. Locatelli Hanchen Ye Tianle Cai Patrick Lewis Deming Chen VLM 75 148 0 22 Apr 2024
Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data Fahim Tajwar Anika Singh Archit Sharma Rafael Rafailov Jeff Schneider Tengyang Xie Stefano Ermon Chelsea Finn Aviral Kumar 28 103 0 22 Apr 2024
ChatRetriever: Adapting Large Language Models for Generalized and Robust Conversational Dense Retrieval Kelong Mao Chenlong Deng Haonan Chen Fengran Mo Zheng Liu Tetsuya Sakai Zhicheng Dou KELM 26 11 0 21 Apr 2024
Aligning Language Models to Explicitly Handle Ambiguity Hyuhng Joon Kim Youna Kim Cheonbok Park Junyeob Kim Choonghyun Park Kang Min Yoo Sang-goo Lee Taeuk Kim 20 11 0 18 Apr 2024
Inductive-Deductive Strategy Reuse for Multi-Turn Instructional Dialogues Jiao Ou Jiayu Wu Che Liu Fuzheng Zhang Di Zhang Kun Gai 19 2 0 17 Apr 2024
Learn Your Reference Model for Real Good Alignment Alexey Gorbatovski Boris Shaposhnikov Alexey Malakhov Nikita Surnachev Yaroslav Aksenov Ian Maksimov Nikita Balagansky Daniil Gavrilov OffRL 47 25 0 15 Apr 2024
Best Practices and Lessons Learned on Synthetic Data for Language Models Ruibo Liu Jerry W. Wei Fangyu Liu Chenglei Si Yanzhe Zhang ... Steven Zheng Daiyi Peng Diyi Yang Denny Zhou Andrew M. Dai SyDa EgoV 36 85 0 11 Apr 2024
JetMoE: Reaching Llama2 Performance with 0.1M Dollars Yikang Shen Zhen Guo Tianle Cai Zengyi Qin MoE ALM 33 26 0 11 Apr 2024