Continued Pretraining for Better Zero- and Few-Shot Promptability

Continued Pretraining for Better Zero- and Few-Shot Promptability

19 October 2022

IV RobertL.Logan

Pete Walsh

Akshita Bhagia

Dirk Groeneveld

Papers citing "Continued Pretraining for Better Zero- and Few-Shot Promptability"

16 / 16 papers shown

Title
Llama-3.1-FoundationAI-SecurityLLM-Base-8B Technical Report Paul Kassianik Baturay Saglam Alexander Chen Blaine Nelson Anu Vellore ... Hyrum Anderson Kojin Oshiba Omar Santos Yaron Singer Amin Karbasi PILM 56 0 0 28 Apr 2025
Efficient Continual Pre-training by Mitigating the Stability Gap Yiduo Guo Jie Fu Huishuai Zhang Dongyan Zhao Yikang Shen 30 12 0 21 Jun 2024
A Continued Pretrained LLM Approach for Automatic Medical Note Generation Dong Yuan Eti Rastogi Gautam Naik Sree Prasanna Rajagopal Sagar Goyal Fen Zhao Jai Chintagunta Jeff Ward LM&MA AI4MH 32 19 0 14 Mar 2024
MEDITRON-70B: Scaling Medical Pretraining for Large Language Models Zeming Chen Alejandro Hernández Cano Angelika Romanou Antoine Bonnet Kyle Matoba ... Axel Marmet Syrielle Montariol Mary-Anne Hartley Martin Jaggi Antoine Bosselut LM&MA AI4MH MedIm 12 174 0 27 Nov 2023
Create and Find Flatness: Building Flat Training Spaces in Advance for Continual Learning Wenhang Shi Yiren Chen Zhe Zhao Wei Lu Kimmo Yan Xiaoyong Du CLL 17 5 0 20 Sep 2023
UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation Daixuan Cheng Shaohan Huang Junyu Bi Yu-Wei Zhan Jianfeng Liu Yujing Wang Hao-Lun Sun Furu Wei Denvy Deng Qi Zhang RALM LRM 11 66 0 15 Mar 2023
HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation Hamish Ivison Akshita Bhagia Yizhong Wang Hannaneh Hajishirzi Matthew E. Peters 30 16 0 20 Dec 2022
HyperTuning: Toward Adapting Large Language Models without Back-propagation Jason Phang Yi Mao Pengcheng He Weizhu Chen 9 30 0 22 Nov 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 2,712 0 24 May 2022
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts Stephen H. Bach Victor Sanh Zheng-Xin Yong Albert Webson Colin Raffel ... Khalid Almubarak Xiangru Tang Dragomir R. Radev Mike Tian-Jian Jiang Alexander M. Rush VLM 212 335 0 02 Feb 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 203 1,651 0 15 Oct 2021
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer Tu Vu Brian Lester Noah Constant Rami Al-Rfou Daniel Matthew Cer VLM LRM 134 276 0 15 Oct 2021
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP Qinyuan Ye Bill Yuchen Lin Xiang Ren 209 167 0 18 Apr 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 278 3,784 0 18 Apr 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,898 0 31 Dec 2020
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 237 11,568 0 09 Mar 2017