Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor

19 December 2022

Papers citing "Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor"

50 / 291 papers shown

Title
Prometheus: Inducing Fine-grained Evaluation Capability in Language Models Seungone Kim Jamin Shin Yejin Cho Joel Jang Shayne Longpre ... Sangdoo Yun Seongjin Shin Sungdong Kim James Thorne Minjoon Seo ALM LM&MA ELM 6 202 0 12 Oct 2023
InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining Boxin Wang Wei Ping Lawrence C. McAfee Peng-Tao Xu Bo Li M. Shoeybi Bryan Catanzaro RALM 13 44 0 11 Oct 2023
The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values Hannah Rose Kirk Andrew M. Bean Bertie Vidgen Paul Röttger Scott A. Hale ALM 11 40 0 11 Oct 2023
KwaiYiiMath: Technical Report Jia-Yi Fu Lei Lin Xiaoyang Gao Pengli Liu Zhengzong Chen ... Zijia Lin Fuzheng Zhang Zhongyuan Wang Di Zhang Kun Gai LRM ReLM RALM 38 2 0 11 Oct 2023
An Empirical Study of Instruction-tuning Large Language Models in Chinese Q. Si Tong Wang Zheng Lin Xu Zhang Yanan Cao Weiping Wang ALM 57 16 0 11 Oct 2023
FireAct: Toward Language Agent Fine-tuning Baian Chen Chang Shu Ehsan Shareghi Nigel Collier Karthik Narasimhan Shunyu Yao ALM LLMAG 99 96 0 09 Oct 2023
Ada-Instruct: Adapting Instruction Generators for Complex Reasoning Wanyun Cui Qianle Wang LRM 31 7 0 06 Oct 2023
InstructProtein: Aligning Human and Protein Language via Knowledge Instruction Zeyuan Wang Qiang Zhang Keyan Ding Ming Qin Zhuang Xiang Xiaotong Li Huajun Chen 16 28 0 05 Oct 2023
RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models Zekun Wang Zhongyuan Peng Haoran Que Jiaheng Liu Wangchunshu Zhou ... Wanli Ouyang Ke Xu Wenhu Chen Jie Fu Junran Peng LLMAG 26 80 0 01 Oct 2023
Adapting LLM Agents with Universal Feedback in Communication Kuan-Chieh Jackson Wang Yadong Lu Michael Santacroce Yeyun Gong Chao Zhang Yelong Shen LLMAG 22 7 0 01 Oct 2023
Self-Specialization: Uncovering Latent Expertise within Large Language Models Junmo Kang Hongyin Luo Yada Zhu Jacob A. Hansen James R. Glass David D. Cox Alan Ritter Rogerio Feris Leonid Karlinsky ALM MoMe 17 4 0 29 Sep 2023
Jointly Training Large Autoregressive Multimodal Models Emanuele Aiello L. Yu Yixin Nie Armen Aghajanyan Barlas Oğuz 11 29 0 27 Sep 2023
LayoutNUWA: Revealing the Hidden Layout Expertise of Large Language Models Zecheng Tang Chenfei Wu Juntao Li Nan Duan 3DV 20 8 0 18 Sep 2023
Can Large Language Models Understand Real-World Complex Instructions? Qi He Jie Zeng Wenhao Huang Lina Chen Jin Xiao ... Shisong Chen Yikai Zhang Zhouhong Gu Jiaqing Liang Yanghua Xiao ALM LRM ELM 90 50 0 17 Sep 2023
Exploring the impact of low-rank adaptation on the performance, efficiency, and regularization of RLHF Simeng Sun Dhawal Gupta Mohit Iyyer 6 17 0 16 Sep 2023
DoG-Instruct: Towards Premium Instruction-Tuning Data via Text-Grounded Instruction Wrapping Yongrui Chen Haiyun Jiang Xinting Huang Shuming Shi Guilin Qi SyDa 6 11 0 11 Sep 2023
Data-Juicer: A One-Stop Data Processing System for Large Language Models Daoyuan Chen Yilun Huang Zhijian Ma Hesen Chen Xuchen Pan ... Zhaoyang Liu Jinyang Gao Yaliang Li Bolin Ding Jingren Zhou SyDa VLM 15 29 0 05 Sep 2023
Donkii: Can Annotation Error Detection Methods Find Errors in Instruction-Tuning Datasets? Leon Weber-Genzel Robert Litschko Ekaterina Artemova Barbara Plank 6 2 0 04 Sep 2023
Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models Neha Sengupta Sunil Kumar Sahu Bokang Jia Satheesh Katipomu Haonan Li ... A. Jackson Hector Xuguang Ren Preslav Nakov Timothy Baldwin Eric P. Xing LRM 8 40 0 30 Aug 2023
Code Llama: Open Foundation Models for Code Baptiste Rozière Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat ... Hugo Touvron Louis Martin Nicolas Usunier Thomas Scialom Gabriel Synnaeve ELM ALM 58 1,881 0 24 Aug 2023
Harnessing the Power of David against Goliath: Exploring Instruction Data Generation without Using Closed-Source Models Yue Wang Xinrui Wang Juntao Li Jinxiong Chang Qishen Zhang Zhongyi Liu Guannan Zhang Min Zhang ALM 11 5 0 24 Aug 2023
Improving Translation Faithfulness of Large Language Models via Augmenting Instructions Yijie Chen Yanjun Liu Fandong Meng Yufeng Chen Jinan Xu Jie Zhou 25 25 0 24 Aug 2023
From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning Ming Li Yong Zhang Zhitao Li Jiuhai Chen Lichang Chen Ning Cheng Jianzong Wang Tianyi Zhou Jing Xiao 25 168 0 23 Aug 2023
From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Models Jing Yao Xiaoyuan Yi Xiting Wang Jindong Wang Xing Xie ALM 14 41 0 23 Aug 2023
Instruction Tuning for Large Language Models: A Survey Shengyu Zhang Linfeng Dong Xiaoya Li Sen Zhang Xiaofei Sun ... Jiwei Li Runyi Hu Tianwei Zhang Fei Wu Guoyin Wang LM&MA 16 524 0 21 Aug 2023
BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questions Wenbo Hu Y. Xu Y. Li W. Li Z. Chen Z. Tu MLLM VLM 28 120 0 19 Aug 2023
#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Models K. Lu Hongyi Yuan Zheng Yuan Runji Lin Junyang Lin Chuanqi Tan Chang Zhou Jingren Zhou ALM LRM 27 33 0 14 Aug 2023
Self-Alignment with Instruction Backtranslation Xian Li Ping Yu Chunting Zhou Timo Schick Omer Levy Luke Zettlemoyer Jason Weston M. Lewis SyDa 11 122 0 11 Aug 2023
A Preliminary Study of the Intrinsic Relationship between Complexity and Alignment Ying Zhao Yu Bowen Binyuan Hui Haiyang Yu Fei Huang Yongbin Li N. Zhang 23 22 0 10 Aug 2023
LoRA-FA: Memory-efficient Low-rank Adaptation for Large Language Models Fine-tuning Longteng Zhang Lin Zhang S. Shi X. Chu Bo-wen Li AI4CE 11 88 0 07 Aug 2023
HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attribution Ehsan Kamalloo A. Jafari Xinyu Crystina Zhang Nandan Thakur Jimmy J. Lin 16 41 0 31 Jul 2023
Exploring Format Consistency for Instruction Tuning Shi Liang Runchu Tian Kunlun Zhu Yujia Qin Huadong Wang Xin Cong Zhiyuan Liu Xiaojiang Liu Maosong Sun ALM 14 11 0 28 Jul 2023
FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets Seonghyeon Ye Doyoung Kim Sungdong Kim Hyeonbin Hwang Seungone Kim Yongrae Jo James Thorne Juho Kim Minjoon Seo ALM 27 96 0 20 Jul 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 70 10,890 0 18 Jul 2023
Effective Prompt Extraction from Language Models Yiming Zhang Nicholas Carlini Daphne Ippolito MIACV SILM 17 35 0 13 Jul 2023
A Comprehensive Overview of Large Language Models Humza Naveed Asad Ullah Khan Shi Qiu Muhammad Saqib Saeed Anwar Muhammad Usman Naveed Akhtar Nick Barnes Ajmal Saeed Mian OffRL 46 499 0 12 Jul 2023
Instruction Mining: When Data Mining Meets Large Language Model Finetuning Yihan Cao Yanbin Kang Chi Wang Lichao Sun ALM 6 29 0 12 Jul 2023
Empowering Cross-lingual Behavioral Testing of NLP Models with Typological Features Ester Hlavnova Sebastian Ruder 17 5 0 11 Jul 2023
What Matters in Training a GPT4-Style Language Model with Multimodal Inputs? Yan Zeng Hanbo Zhang Jiani Zheng Jiangnan Xia Guoqiang Wei Yang Wei Yuchen Zhang Tao Kong MLLM 16 71 0 05 Jul 2023
SCITUNE: Aligning Large Language Models with Scientific Multimodal Instructions Sameera Horawalavithana Sai Munikoti Ian Stewart Henry Kvinge MLLM 13 20 0 03 Jul 2023
Kosmos-2: Grounding Multimodal Large Language Models to the World Zhiliang Peng Wenhui Wang Li Dong Y. Hao Shaohan Huang Shuming Ma Furu Wei MLLM ObjD VLM 14 688 0 26 Jun 2023
Training Multimedia Event Extraction With Generated Images and Captions Zilin Du Yunxin Li Xu Guo Yidan Sun Boyang Albert Li DiffM 18 7 0 15 Jun 2023
MiniLLM: Knowledge Distillation of Large Language Models Yuxian Gu Li Dong Furu Wei Minlie Huang ALM 23 75 0 14 Jun 2023
ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases Qiaoyu Tang Ziliang Deng Hongyu Lin Xianpei Han Qiao Liang Boxi Cao Le Sun CLL SyDa 14 179 0 08 Jun 2023
How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources Yizhong Wang Hamish Ivison Pradeep Dasigi Jack Hessel Tushar Khot ... David Wadden Kelsey MacMillan Noah A. Smith Iz Beltagy Hannaneh Hajishirzi ALM ELM 11 364 0 07 Jun 2023
ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systems Yi Zhang Jan Deriu George Katsogiannis-Meimarakis Catherine Kosten Georgia Koutrika Kurt Stockinger 12 17 0 07 Jun 2023
Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learning Fan Yin Jesse Vig Philippe Laban Shafiq R. Joty Caiming Xiong Chien-Sheng Wu ALM 13 38 0 01 Jun 2023
Layout and Task Aware Instruction Prompt for Zero-shot Document Image Question Answering Wenjin Wang Yunhao Li Yixin Ou Yin Zhang VLM 14 24 0 01 Jun 2023
The False Promise of Imitating Proprietary LLMs Arnav Gudibande Eric Wallace Charles Burton Snell Xinyang Geng Hao Liu Pieter Abbeel Sergey Levine Dawn Song ALM 22 195 0 25 May 2023
Universal Self-Adaptive Prompting Xingchen Wan Ruoxi Sun Hootan Nakhost H. Dai Julian Martin Eisenschlos Sercan Ö. Arik Tomas Pfister LRM 31 9 0 24 May 2023