Pre-trained Models for Natural Language Processing: A Survey

18 March 2020

Xipeng Qiu

Tianxiang Sun

Xuanjing Huang

Papers citing "Pre-trained Models for Natural Language Processing: A Survey"

21 / 21 papers shown

Title
A Comprehensive Part-of-Speech Tagging to Standardize Central-Kurdish Language: A Research Guide for Kurdish Natural Language Processing Tasks S. Nazira Sabr Mustafa Talar Sabah Omar Salah Hwayyiz Rasool Nawzad Anwer Omer ... S. M. Asaad Sara Jamal Mohammed Twana Saeed Ali Fazil Shawrow H. Maghdid 19 0 0 28 Apr 2025
A BERT-Style Self-Supervised Learning CNN for Disease Identification from Retinal Images Xin Li Wenhui Zhu Peijie Qiu Oana Dumitrascu Amal Youssef Y. Wang SSL MedIm 69 0 0 25 Apr 2025
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 252 2,999 0 18 Apr 2021
WARP: Word-level Adversarial ReProgramming Karen Hambardzumyan Hrant Khachatrian Jonathan May AAML 229 302 0 01 Jan 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 223 1,649 0 31 Dec 2020
A Survey on Contextual Embeddings Qi Liu Matt J. Kusner Phil Blunsom 180 132 0 16 Mar 2020
Data Augmentation using Pre-trained Transformer Models Varun Kumar Ashutosh Choudhary Eunah Cho VLM 190 315 0 04 Mar 2020
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing Canwen Xu Wangchunshu Zhou Tao Ge Furu Wei Ming Zhou 175 184 0 07 Feb 2020
Adversarial Training for Aspect-Based Sentiment Analysis with BERT Akbar Karimi L. Rossi Andrea Prati 169 89 0 30 Jan 2020
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 225 1,382 0 21 Jan 2020
A Mutual Information Maximization Perspective of Language Representation Learning Lingpeng Kong Cyprien de Masson dÁutume Wang Ling Lei Yu Zihang Dai Dani Yogatama SSL 181 158 0 18 Oct 2019
FreeLB: Enhanced Adversarial Training for Natural Language Understanding Chen Zhu Yu Cheng Zhe Gan S. Sun Tom Goldstein Jingjing Liu AAML 177 390 0 25 Sep 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 221 1,436 0 17 Sep 2019
K-BERT: Enabling Language Representation with Knowledge Graph Weijie Liu Peng Zhou Zhe Zhao Zhiruo Wang Qi Ju Haotang Deng Ping Wang 192 683 0 17 Sep 2019
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT Sheng Shen Zhen Dong Jiayu Ye Linjian Ma Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer MQ 195 505 0 12 Sep 2019
Knowledge Enhanced Contextual Word Representations Matthew E. Peters Mark Neumann IV RobertL.Logan Roy Schwartz Vidur Joshi Sameer Singh Noah A. Smith 194 624 0 09 Sep 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 367 2,216 0 03 Sep 2019
Text Summarization with Pretrained Encoders Yang Liu Mirella Lapata MILM 212 1,282 0 22 Aug 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 267 6,003 0 20 Apr 2018
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 241 5,046 0 05 Nov 2016
Convolutional Neural Networks for Sentence Classification Yoon Kim AILaw VLM 217 12,850 0 25 Aug 2014