Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping

15 February 2020

Papers citing "Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping"

44 / 94 papers shown

Title
Embedding Hallucination for Few-Shot Language Fine-tuning Yiren Jian Chongyang Gao Soroush Vosoughi 20 4 0 03 May 2022
Super-Prompting: Utilizing Model-Independent Contextual Data to Reduce Data Annotation Required in Visual Commonsense Tasks Navid Rezaei Marek Reformat VLM 17 2 0 25 Apr 2022
mGPT: Few-Shot Learners Go Multilingual Oleh Shliazhko Alena Fenogenova Maria Tikhonova Vladislav Mikhailov Anastasia Kozlova Tatiana Shavrina 31 148 0 15 Apr 2022
Reducing Model Jitter: Stable Re-training of Semantic Parsers in Production Environments Christopher Hidey Fei Liu Rahul Goel 11 4 0 10 Apr 2022
PERFECT: Prompt-free and Efficient Few-shot Learning with Language Models Rabeeh Karimi Mahabadi Luke Zettlemoyer James Henderson Marzieh Saeidi Lambert Mathias Ves Stoyanov Majid Yazdani VLM 21 69 0 03 Apr 2022
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time Mitchell Wortsman Gabriel Ilharco S. Gadre Rebecca Roelofs Raphael Gontijo-Lopes ... Hongseok Namkoong Ali Farhadi Y. Carmon Simon Kornblith Ludwig Schmidt MoMe 42 906 1 10 Mar 2022
Revisiting Parameter-Efficient Tuning: Are We Really There Yet? Guanzheng Chen Fangyu Liu Zaiqiao Meng Shangsong Liang 26 87 0 16 Feb 2022
A Differential Entropy Estimator for Training Neural Networks Georg Pichler Pierre Colombo Malik Boudiaf Günther Koliander Pablo Piantanida 15 21 0 14 Feb 2022
Diversity Enhanced Active Learning with Strictly Proper Scoring Rules Wei Tan Lan Du Wray L. Buntine 8 30 0 27 Oct 2021
SkullEngine: A Multi-stage CNN Framework for Collaborative CBCT Image Segmentation and Landmark Detection Qin Liu H. Deng C. Lian Xiaoyang Chen Deqiang Xiao ... Xu Chen Tianshu Kuang J. Gateno P. Yap J. Xia 15 25 0 07 Oct 2021
UoB at SemEval-2021 Task 5: Extending Pre-Trained Language Models to Include Task and Domain-Specific Information for Toxic Span Prediction E. Yan Harish Tayyar Madabushi 10 2 0 07 Oct 2021
KNN-BERT: Fine-Tuning Pre-Trained Models with KNN Classifier Linyang Li Demin Song Ruotian Ma Xipeng Qiu Xuanjing Huang 27 21 0 06 Oct 2021
CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models Yuan Yao Ao Zhang Zhengyan Zhang Zhiyuan Liu Tat-Seng Chua Maosong Sun MLLM VPVLM VLM 194 221 0 24 Sep 2021
Weakly Supervised Explainable Phrasal Reasoning with Neural Fuzzy Logic Zijun Wu Zi Xuan Zhang Atharva Naik Zhijian Mei Mauajama Firdaus Lili Mou LRM NAI 36 14 0 18 Sep 2021
Subword Mapping and Anchoring across Languages Giorgos Vernikos Andrei Popescu-Belis 62 12 0 09 Sep 2021
Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning Prasetya Ajie Utama N. Moosavi Victor Sanh Iryna Gurevych AAML 56 35 0 09 Sep 2021
Deep Reinforcement Learning at the Edge of the Statistical Precipice Rishabh Agarwal Max Schwarzer P. S. Castro Aaron Courville Marc G. Bellemare OffRL 25 630 0 30 Aug 2021
Rethinking Why Intermediate-Task Fine-Tuning Works Ting-Yun Chang Chi-Jen Lu LRM 19 29 0 26 Aug 2021
Linking Common Vulnerabilities and Exposures to the MITRE ATT&CK Framework: A Self-Distillation Approach Benjamin Ampel Sagar Samtani Steven Ullman Hsinchun Chen 22 35 0 03 Aug 2021
FewCLUE: A Chinese Few-shot Learning Evaluation Benchmark Liang Xu Xiaojing Lu Chenyang Yuan Xuanwei Zhang Huilin Xu ... Guoao Wei X. Pan Xin Tian Libo Qin Hai Hu ELM 24 56 0 15 Jul 2021
Noise Stability Regularization for Improving BERT Fine-tuning Hang Hua Xingjian Li Dejing Dou Chengzhong Xu Jiebo Luo 11 42 0 10 Jul 2021
The MultiBERTs: BERT Reproductions for Robustness Analysis Thibault Sellam Steve Yadlowsky Jason W. Wei Naomi Saphra Alexander DÁmour ... Iulia Turc Jacob Eisenstein Dipanjan Das Ian Tenney Ellie Pavlick 22 93 0 30 Jun 2021
A Closer Look at How Fine-tuning Changes BERT Yichu Zhou Vivek Srikumar 24 63 0 27 Jun 2021
Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models Robert L Logan IV Ivana Balavzević Eric Wallace Fabio Petroni Sameer Singh Sebastian Riedel VPVLM 20 207 0 24 Jun 2021
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers Rabeeh Karimi Mahabadi James Henderson Sebastian Ruder MoE 25 466 0 08 Jun 2021
PTR: Prompt Tuning with Rules for Text Classification Xu Han Weilin Zhao Ning Ding Zhiyuan Liu Maosong Sun VLM 33 513 0 24 May 2021
On the Importance of Effectively Adapting Pretrained Language Models for Active Learning Katerina Margatina Loïc Barrault Nikolaos Aletras 19 36 0 16 Apr 2021
How Many Data Points is a Prompt Worth? Teven Le Scao Alexander M. Rush VLM 32 295 0 15 Mar 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,916 0 31 Dec 2020
Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs Emanuele Bugliarello Ryan Cotterell Naoaki Okazaki Desmond Elliott 22 119 0 30 Nov 2020
Underspecification Presents Challenges for Credibility in Modern Machine Learning Alexander DÁmour Katherine A. Heller D. Moldovan Ben Adlam B. Alipanahi ... Kellie Webster Steve Yadlowsky T. Yun Xiaohua Zhai D. Sculley OffRL 21 669 0 06 Nov 2020
On the Transformer Growth for Progressive BERT Training Xiaotao Gu Liyuan Liu Hongkun Yu Jing Li C. L. P. Chen Jiawei Han VLM 61 51 0 23 Oct 2020
GiBERT: Introducing Linguistic Knowledge into BERT through a Lightweight Gated Injection Method Nicole Peinelt Marek Rei Maria Liakata 14 2 0 23 Oct 2020
The RELX Dataset and Matching the Multilingual Blanks for Cross-Lingual Relation Classification Abdullatif Köksal Arzucan Özgür 30 19 0 19 Oct 2020
Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data Jonathan Pilault Amine Elhattami C. Pal CLL MoE 19 89 0 19 Sep 2020
It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners Timo Schick Hinrich Schütze 15 951 0 15 Sep 2020
Differentially Private Language Models Benefit from Public Pre-training Gavin Kerrigan Dylan Slack Jens Tuyls 8 56 0 13 Sep 2020
Revisiting Few-sample BERT Fine-tuning Tianyi Zhang Felix Wu Arzoo Katiyar Kilian Q. Weinberger Yoav Artzi 30 441 0 10 Jun 2020
The Sensitivity of Language Models and Humans to Winograd Schema Perturbations Mostafa Abdou Vinit Ravishankar Maria Barrett Yonatan Belinkov Desmond Elliott Anders Søgaard ReLM LRM 54 34 0 04 May 2020
DQI: Measuring Data Quality in NLP Swaroop Mishra Anjana Arunkumar Bhavdeep Singh Sachdeva Chris Bryan Chitta Baral 25 30 0 02 May 2020
Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer Siddhant Garg Rohit Kumar Sharma Yingyu Liang 20 4 0 10 Apr 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 241 1,450 0 18 Mar 2020
FreeLB: Enhanced Adversarial Training for Natural Language Understanding Chen Zhu Yu Cheng Zhe Gan S. Sun Tom Goldstein Jingjing Liu AAML 221 436 0 25 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,943 0 20 Apr 2018