Muppet: Massive Multi-task Representations with Pre-Finetuning

26 January 2021

Luke Zettlemoyer

Papers citing "Muppet: Massive Multi-task Representations with Pre-Finetuning"

50 / 64 papers shown

Title
On the Role of Pre-trained Embeddings in Binary Code Analysis Alwin Maier Felix Weissberg Konrad Rieck 43 0 0 12 Feb 2025
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 49 3 0 24 Oct 2024
Pre-Calc: Learning to Use the Calculator Improves Numeracy in Language Models Vishruth Veerendranath Vishwa Shah Kshitish Ghate 30 0 0 22 Apr 2024
One-Shot Learning as Instruction Data Prospector for Large Language Models Yunshui Li Binyuan Hui Xiaobo Xia Jiaxi Yang Min Yang ... Ling-Hao Chen Junhao Liu Tongliang Liu Fei Huang Yongbin Li 38 31 0 16 Dec 2023
A Knowledge-enhanced Two-stage Generative Framework for Medical Dialogue Information Extraction Zefa Hu Ziyi Ni Jing Shi Shuang Xu Bo Xu MedIm 37 1 0 30 Jul 2023
ConaCLIP: Exploring Distillation of Fully-Connected Knowledge Interaction Graph for Lightweight Text-Image Retrieval Jiapeng Wang Chengyu Wang Xiaodan Wang Jun Huang Lianwen Jin VLM 31 4 0 28 May 2023
Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark Minje Choi Jiaxin Pei Sagar Kumar Chang Shu David Jurgens ALM LLMAG 29 69 0 24 May 2023
TaskWeb: Selecting Better Source Tasks for Multi-task NLP Joongwon Kim Akari Asai Gabriel Ilharco Hannaneh Hajishirzi 29 11 0 22 May 2023
LINGO : Visually Debiasing Natural Language Instructions to Support Task Diversity Anjana Arunkumar Shubham Sharma Rakhi Agrawal Sriramakrishnan Chandrasekaran Chris Bryan 34 0 0 12 Apr 2023
Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning Zhen Wang Rameswar Panda Leonid Karlinsky Rogerio Feris Huan Sun Yoon Kim VLM VPVLM 21 107 0 06 Mar 2023
Conversational Text-to-SQL: An Odyssey into State-of-the-Art and Challenges Ahead S. Parthasarathi Lu Zeng Dilek Z. Hakkani-Tür 44 2 0 21 Feb 2023
Privately Customizing Prefinetuning to Better Match User Data in Federated Learning Charlie Hou Hongyuan Zhan Akshat Shrivastava Sida I. Wang S. Livshits Giulia Fanti Daniel Lazar FedML 32 15 0 17 Feb 2023
Knowledge is a Region in Weight Space for Fine-tuned Language Models Almog Gueta Elad Venezian Colin Raffel Noam Slonim Yoav Katz Leshem Choshen 31 49 0 09 Feb 2023
CrossCodeBench: Benchmarking Cross-Task Generalization of Source Code Models Changan Niu Chuanyi Li Vincent Ng Bin Luo ELM ALM 34 9 0 08 Feb 2023
Curriculum-Guided Abstractive Summarization Sajad Sotudeh Hanieh Deilamsalehy Franck Dernoncourt Nazli Goharian 35 1 0 02 Feb 2023
A Stability Analysis of Fine-Tuning a Pre-Trained Model Z. Fu Anthony Man-Cho So Nigel Collier 23 3 0 24 Jan 2023
MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning Zhiyang Xu Ying Shen Lifu Huang MLLM 32 110 0 21 Dec 2022
MIGA: A Unified Multi-task Generation Framework for Conversational Text-to-SQL Yingwen Fu Wenjie Ou Zhou Yu Yue Lin 26 6 0 19 Dec 2022
ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning Shachar Don-Yehiya Elad Venezian Colin Raffel Noam Slonim Yoav Katz Leshem Choshen MoMe 28 52 0 02 Dec 2022
SciRepEval: A Multi-Format Benchmark for Scientific Document Representations Amanpreet Singh Mike DÁrcy Arman Cohan Doug Downey Sergey Feldman 22 79 0 23 Nov 2022
TCBERT: A Technical Report for Chinese Topic Classification BERT Ting Han Kunhao Pan Xinyu Chen Dingjie Song Yuchen Fan Xinyu Gao Ruyi Gan Jiaxing Zhang VLM 19 1 0 21 Nov 2022
Why Is It Hate Speech? Masked Rationale Prediction for Explainable Hate Speech Detection Jiyun Kim Byounghan Lee Kyung-ah Sohn 21 13 0 01 Nov 2022
Zero-Shot Text Classification with Self-Training Ariel Gera Alon Halfon Eyal Shnarch Yotam Perlitz L. Ein-Dor Noam Slonim VLM 28 59 0 31 Oct 2022
Exploring Mode Connectivity for Pre-trained Language Models Yujia Qin Cheng Qian Jing Yi Weize Chen Yankai Lin Xu Han Zhiyuan Liu Maosong Sun Jie Zhou 29 20 0 25 Oct 2022
Visualizing the Obvious: A Concreteness-based Ensemble Model for Noun Property Prediction Yue Yang Artemis Panagopoulou Marianna Apidianaki Mark Yatskar Chris Callison-Burch 23 2 0 24 Oct 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 62 2,989 0 20 Oct 2022
Transcending Scaling Laws with 0.1% Extra Compute Yi Tay Jason W. Wei Hyung Won Chung Vinh Q. Tran David R. So ... Donald Metzler Slav Petrov N. Houlsby Quoc V. Le Mostafa Dehghani LRM 42 68 0 20 Oct 2022
Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective Ping Yang Junjie Wang Ruyi Gan Xinyu Zhu Lin Zhang Ziwei Wu Xinyu Gao Jiaxing Zhang Tetsuya Sakai BDL 14 25 0 16 Oct 2022
Data-Efficiency with a Single GPU: An Exploration of Transfer Methods for Small Language Models Alon Albalak Akshat Shrivastava Chinnadhurai Sankar Adithya Sagar Mike Ross 32 3 0 08 Oct 2022
State-of-the-art generalisation research in NLP: A taxonomy and review Dieuwke Hupkes Mario Giulianelli Verna Dankers Mikel Artetxe Yanai Elazar ... Leila Khalatbari Maria Ryskina Rita Frieske Ryan Cotterell Zhijing Jin 114 93 0 06 Oct 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 28 109 0 31 Aug 2022
Few-shot Adaptation Works with UnpredicTable Data Jun Shern Chan Michael Pieler Jonathan Jao Jérémy Scheurer Ethan Perez 31 5 0 01 Aug 2022
Unified BERT for Few-shot Natural Language Understanding Junyu Lu Ping Yang Ruyi Gan Jing Yang Jiaxing Zhang 19 2 0 24 Jun 2022
Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization Puyuan Liu Chenyang Huang Lili Mou 30 20 0 28 May 2022
AANG: Automating Auxiliary Learning Lucio Dery Paul Michel M. Khodak Graham Neubig Ameet Talwalkar 41 9 0 27 May 2022
Eliciting and Understanding Cross-Task Skills with Task-Level Mixture-of-Experts Qinyuan Ye Juan Zha Xiang Ren MoE 18 12 0 25 May 2022
Leveraging QA Datasets to Improve Generative Data Augmentation Dheeraj Mekala Tu Vu Timo Schick Jingbo Shang 19 18 0 25 May 2022
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 127 100 0 24 May 2022
PreQuEL: Quality Estimation of Machine Translation Outputs in Advance Shachar Don-Yehiya Leshem Choshen Omri Abend 30 10 0 18 May 2022
Few-shot Mining of Naturally Occurring Inputs and Outputs Mandar Joshi Terra Blevins M. Lewis Daniel S. Weld Luke Zettlemoyer 30 1 0 09 May 2022
Improving In-Context Few-Shot Learning via Self-Supervised Training Mingda Chen Jingfei Du Ramakanth Pasunuru Todor Mihaylov Srini Iyer Ves Stoyanov Zornitsa Kozareva SSL AI4MH 32 63 0 03 May 2022
Exploring the Role of Task Transferability in Large-Scale Multi-Task Learning Vishakh Padmakumar Leonard Lausen Miguel Ballesteros Sheng Zha He He George Karypis 28 18 0 23 Apr 2022
IndicXNLI: Evaluating Multilingual Inference for Indian Languages Divyanshu Aggarwal V. Gupta Anoop Kunchukuttan 23 27 0 19 Apr 2022
CLUES: A Benchmark for Learning Classifiers using Natural Language Explanations Rakesh R Menon Sayan Ghosh Shashank Srivastava LRM ELM 26 9 0 14 Apr 2022
NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Tasks Swaroop Mishra Arindam Mitra Neeraj Varshney Bhavdeep Singh Sachdeva Peter Clark Chitta Baral A. Kalyan AIMat ReLM ELM LRM 27 102 0 12 Apr 2022
Fusing finetuned models for better pretraining Leshem Choshen Elad Venezian Noam Slonim Yoav Katz FedML AI4CE MoMe 45 87 0 06 Apr 2022
Geographic Adaptation of Pretrained Language Models Valentin Hofmann Goran Glavavs Nikola Ljubevsić J. Pierrehumbert Hinrich Schütze VLM 21 16 0 16 Mar 2022
Hyperdecoders: Instance-specific decoders for multi-task NLP Hamish Ivison Matthew E. Peters AI4CE 26 20 0 15 Mar 2022
SkillNet-NLU: A Sparsely Activated Model for General-Purpose Natural Language Understanding Fan Zhang Duyu Tang Yong Dai Cong Zhou Shuangzhi Wu Shuming Shi CLL MoE 33 12 0 07 Mar 2022
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? Sewon Min Xinxi Lyu Ari Holtzman Mikel Artetxe M. Lewis Hannaneh Hajishirzi Luke Zettlemoyer LLMAG LRM 31 1,400 0 25 Feb 2022