Enhancing Chat Language Models by Scaling High-quality Instructional Conversations

23 May 2023

Zhiyuan Liu

Maosong Sun

Bowen Zhou

ALM

ArXiv PDF HTML

Papers citing "Enhancing Chat Language Models by Scaling High-quality Instructional Conversations"

50 / 376 papers shown

Title
Preference-Guided Reflective Sampling for Aligning Language Models Hai Ye Hwee Tou Ng 24 3 0 22 Aug 2024
REInstruct: Building Instruction Data from Unlabeled Corpus Shu Chen Xinyan Guan Y. Lu Hongyu Lin Xianpei Han Le Sun ALM SyDa 14 1 0 20 Aug 2024
PanoSent: A Panoptic Sextuple Extraction Benchmark for Multimodal Conversational Aspect-based Sentiment Analysis Meng Luo Hao Fei Bobo Li Shengqiong Wu Qian Liu Soujanya Poria Erik Cambria M. Lee W. Hsu 28 7 0 18 Aug 2024
Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization Yuxin Jiang Bo Huang Yufei Wang Xingshan Zeng Liangyou Li Yasheng Wang Xin Jiang Lifeng Shang Ruiming Tang Wei Wang 40 5 0 14 Aug 2024
LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs Yushi Bai Jiajie Zhang Xin Lv Linzhi Zheng Siqi Zhu Lei Hou Yuxiao Dong Jie Tang Juanzi Li VGen LLMAG ALM 34 37 0 13 Aug 2024
Med42-v2: A Suite of Clinical LLMs Clément Christophe Praveen K Kanithi Tathagata Raha Shadab Khan Marco AF Pimentel ELM LM&MA AI4MH 20 19 0 12 Aug 2024
In2Core: Leveraging Influence Functions for Coreset Selection in Instruction Finetuning of Large Language Models Ayrton San Joaquin Bin Wang Zhengyuan Liu Nicholas Asher Brian Lim Philippe Muller Nancy Chen 22 0 0 07 Aug 2024
Intermittent Semi-working Mask: A New Masking Paradigm for LLMs Mingcong Lu Jiangcai Zhu Wang Hao Zheng Li Shusheng Zhang Kailai Shao Chao Chen Nan Li Feng Wang Xin Lu 32 0 0 01 Aug 2024
ThinK: Thinner Key Cache by Query-Driven Pruning Yuhui Xu Zhanming Jie Hanze Dong Lei Wang Xudong Lu Aojun Zhou Amrita Saha Caiming Xiong Doyen Sahoo 64 14 0 30 Jul 2024
SaulLM-54B & SaulLM-141B: Scaling Up Domain Adaptation for the Legal Domain Pierre Colombo T. Pires Malik Boudiaf Rui Melo Dominic Culver Sofia Morgado Etienne Malaboeuf Gabriel Hautreux Johanne Charpentier Michael Desa ELM AILaw ALM 35 12 0 28 Jul 2024
LLAVADI: What Matters For Multimodal Large Language Models Distillation Shilin Xu Xiangtai Li Haobo Yuan Lu Qi Yunhai Tong Ming-Hsuan Yang 34 3 0 28 Jul 2024
Towards Effective and Efficient Continual Pre-training of Large Language Models Jie Chen Zhipeng Chen Jiapeng Wang Kun Zhou Yutao Zhu ... Rui Yan Zhewei Wei Di Hu Wenbing Huang Ji-Rong Wen KELM ALM CLL ELM LRM 44 4 0 26 Jul 2024
ALLaM: Large Language Models for Arabic and English M Saiful Bari Yazeed Alnumay Norah A. Alzahrani Nouf M. Alotaibi H. A. Alyahya ... Jeril Kuriakose Abdalghani Abujabal Nora Al-Twairesh Areeb Alowisheq Haidar Khan 31 11 0 22 Jul 2024
Understanding Reference Policies in Direct Preference Optimization Yixin Liu Pengfei Liu Arman Cohan 26 7 0 18 Jul 2024
PersLLM: A Personified Training Approach for Large Language Models Zheni Zeng Jiayi Chen Huimin Chen Yukun Yan Yuxuan Chen Zhenghao Liu Zhiyuan Liu Maosong Sun LLMAG 12 2 0 17 Jul 2024
The Oscars of AI Theater: A Survey on Role-Playing with Language Models Nuo Chen Yan Wang Yang Deng Jia Li 26 14 0 16 Jul 2024
A Survey on Symbolic Knowledge Distillation of Large Language Models Kamal Acharya Alvaro Velasquez H. Song SyDa 29 4 0 12 Jul 2024
GOFA: A Generative One-For-All Model for Joint Graph Language Modeling Lecheng Kong Jiarui Feng Hao Liu Chengsong Huang Jiaxin Huang Yixin Chen Muhan Zhang AI4CE 77 6 0 12 Jul 2024
Adapting LLMs to Hebrew: Unveiling DictaLM 2.0 with Enhanced Vocabulary and Instruction Capabilities Shaltiel Shmidman Avi Shmidman Amir DN Cohen Moshe Koppel 22 2 0 09 Jul 2024
Entropy Law: The Story Behind Data Compression and LLM Performance Mingjia Yin Chuhan Wu Yufei Wang Hao Wang Wei Guo Yasheng Wang Y. Liu Ruiming Tang Defu Lian Enhong Chen 28 19 0 09 Jul 2024
OffsetBias: Leveraging Debiased Data for Tuning Evaluators Junsoo Park Seungyeon Jwa Meiying Ren Daeyoung Kim Sanghyuk Choi ALM 29 29 0 09 Jul 2024
LIONs: An Empirically Optimized Approach to Align Language Models Xiao Yu Qingyang Wu Yu Li Zhou Yu ALM 27 3 0 09 Jul 2024
A Single Transformer for Scalable Vision-Language Modeling Yangyi Chen Xingyao Wang Hao Peng Heng Ji LRM 40 13 0 08 Jul 2024
LoCo: Low-Bit Communication Adaptor for Large-scale Model Training Xingyu Xie Zhijie Lin Kim-Chuan Toh Pan Zhou 18 2 0 05 Jul 2024
Securing Multi-turn Conversational Language Models Against Distributed Backdoor Triggers Terry Tong Jiashu Xu Qin Liu Muhao Chen AAML SILM 30 1 0 04 Jul 2024
LLM Roleplay: Simulating Human-Chatbot Interaction Hovhannes Tamoyan Hendrik Schuff Iryna Gurevych 31 8 0 04 Jul 2024
AgentInstruct: Toward Generative Teaching with Agentic Flows Arindam Mitra Luciano Del Corro Guoqing Zheng Shweti Mahajan Dany Rouhana ... Corby Rosset Fillipe Silva Hamed Khanpour Yash Lara Ahmed Awadallah SyDa 35 23 0 03 Jul 2024
Learning to Refine with Fine-Grained Natural Language Feedback Manya Wadhwa Xinyu Zhao Junyi Jessy Li Greg Durrett 20 11 0 02 Jul 2024
Efficient Sparse Attention needs Adaptive Token Release Chaoran Zhang Lixin Zou Dan Luo Min Tang Xiangyang Luo Zihao Li Chenliang Li 28 2 0 02 Jul 2024
DogeRM: Equipping Reward Models with Domain Knowledge through Model Merging Tzu-Han Lin Chen An Li Hung-yi Lee Yun-Nung Chen VLM ALM 26 4 0 01 Jul 2024
$$\text{Memory}^3$: Language Modeling with Explicit Memory$ $\text{Memory}^3$ : Language Modeling with Explicit Memory Hongkang Yang Zehao Lin Wenjin Wang Hao Wu Zhiyu Li ... Yu Yu Kai Chen Feiyu Xiong Linpeng Tang Weinan E 33 11 0 01 Jul 2024
Adaptive Draft-Verification for Efficient Large Language Model Decoding Xukun Liu Bowen Lei Ruqi Zhang Dongkuan Xu 30 3 0 27 Jun 2024
A Teacher Is Worth A Million Instructions Nikhil Kothari Ravindra Nayak Shreyas Shetty Amey Patil Nikesh Garera ALM 19 0 0 27 Jun 2024
DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graph Zhehao Zhang Jiaao Chen Diyi Yang LRM 32 7 0 25 Jun 2024
M2Lingual: Enhancing Multilingual, Multi-Turn Instruction Alignment in Large Language Models Rishabh Maheshwary Vikas Yadav Hoang Nguyen Khyati Mahajan Sathwik Tejaswi Madhusudhan 37 3 0 24 Jun 2024
Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models Xinrong Zhang Yingfa Chen Shengding Hu Xu Han Zihang Xu Yuanwei Xu Weilin Zhao Maosong Sun Zhiyuan Liu 29 9 0 22 Jun 2024
Steering Without Side Effects: Improving Post-Deployment Control of Language Models Asa Cooper Stickland Alexander Lyzhov Jacob Pfau Salsabila Mahdi Samuel R. Bowman LLMSV AAML 57 17 0 21 Jun 2024
TAGLAS: An atlas of text-attributed graph datasets in the era of large graph and language models Jiarui Feng Hao Liu Lecheng Kong Yixin Chen Muhan Zhang 24 2 0 20 Jun 2024
CityGPT: Empowering Urban Spatial Cognition of Large Language Models Jie Feng Yuwei Du Tianhui Liu Siqi Guo Yuming Lin Yong Li 30 10 0 20 Jun 2024
Do LLMs Have Distinct and Consistent Personality? TRAIT: Personality Testset designed for LLMs with Psychometrics Seungbeen Lee Seungwon Lim Seungju Han Giyeong Oh Hyungjoo Chae ... Beong-woo Kwak Yeonsoo Lee Dongha Lee Jinyoung Yeo Youngjae Yu 31 8 0 20 Jun 2024
Take the essence and discard the dross: A Rethinking on Data Selection for Fine-Tuning Large Language Models Ziche Liu Rui Ke Feng Jiang Feng Jiang Haizhou Li 58 1 0 20 Jun 2024
Finding Blind Spots in Evaluator LLMs with Interpretable Checklists Sumanth Doddapaneni Mohammed Safi Ur Rahman Khan Sshubam Verma Mitesh Khapra 34 11 0 19 Jun 2024
CleanGen: Mitigating Backdoor Attacks for Generation Tasks in Large Language Models Yuetai Li Zhangchen Xu Fengqing Jiang Luyao Niu D. Sahabandu Bhaskar Ramasubramanian Radha Poovendran SILM AAML 44 6 0 18 Jun 2024
Nemotron-4 340B Technical Report Nvidia : Bo Adler Niket Agarwal Ashwath Aithal ... Jimmy Zhang Jing Zhang Vivienne Zhang Yian Zhang Chen Zhu 36 54 0 17 Jun 2024
HARE: HumAn pRiors, a key to small language model Efficiency Lingyun Zhang Bin jin Gaojian Ge Lunhui Liu Xuewen Shen Mingyong Wu Houqian Zhang Yongneng Jiang Shiqi Chen Shi Pu ALM 27 0 0 17 Jun 2024
Self-Evolution Fine-Tuning for Policy Optimization Ruijun Chen Jiehao Liang Shiping Gao Fanqi Wan Xiaojun Quan 35 0 0 16 Jun 2024
GenQA: Generating Millions of Instructions from a Handful of Prompts Jiuhai Chen Rifaa Qadri Yuxin Wen Neel Jain John Kirchenbauer Tianyi Zhou Tom Goldstein ALM 35 13 0 14 Jun 2024
On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey Lin Long Rui Wang Ruixuan Xiao Junbo Zhao Xiao Ding Gang Chen Haobo Wang SyDa 48 88 0 14 Jun 2024
Mixture-of-Skills: Learning to Optimize Data Usage for Fine-Tuning Large Language Models Minghao Wu Thuy-Trang Vu Lizhen Qu Gholamreza Haffari 27 4 0 13 Jun 2024
HelpSteer2: Open-source dataset for training top-performing reward models Zhilin Wang Yi Dong Olivier Delalleau Jiaqi Zeng Gerald Shen Daniel Egert Jimmy J. Zhang Makesh Narsimhan Sreedhar Oleksii Kuchaiev AI4TS 44 82 0 12 Jun 2024