v1v2 (latest)

Dict-BERT: Enhancing Language Model Pre-training with Dictionary

13 October 2021

Papers citing "Dict-BERT: Enhancing Language Model Pre-training with Dictionary"

36 / 36 papers shown

Title
Threshold Filtering Packing for Supervised Fine-Tuning: Training Related Samples within Packs Jiancheng Dong Lei Jiang Wei Jin Lu Cheng 106 1 0 18 Aug 2024
Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts Wenhao Yu Chenguang Zhu Lianhui Qin Zhihan Zhang Tong Zhao Meng Jiang LRM 75 32 0 14 Mar 2022
KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering Donghan Yu Chenguang Zhu Yuwei Fang Wenhao Yu Shuohang Wang Yichong Xu Xiang Ren Yiming Yang Michael Zeng 87 90 0 08 Oct 2021
Lacking the embedding of a word? Look it up into a traditional dictionary Elena Sofia Ruzzetti Leonardo Ranaldi Michele Mastromattei Francesca Fallucchi Fabio Massimo Zanzotto 61 15 0 24 Sep 2021
Knowledge Enhanced Fine-Tuning for Better Handling Unseen Entities in Dialogue Generation Leyang Cui Yu-Huan Wu Shujie Liu Yue Zhang 146 23 0 12 Sep 2021
Does Knowledge Help General NLU? An Empirical Study Ruochen Xu Yuwei Fang Chenguang Zhu Michael Zeng ELM 67 9 0 01 Sep 2021
Pre-training Text-to-Text Transformers for Concept-centric Common Sense Wangchunshu Zhou Dong-Ho Lee Ravi Kiran Selvam Seyeon Lee Bill Yuchen Lin Xiang Ren LRM VLM 55 72 0 24 Oct 2020
A Survey of Knowledge-Enhanced Text Generation Wenhao Yu Chenguang Zhu Zaitang Li Zhiting Hu Qingyun Wang Heng Ji Meng Jiang 129 288 0 09 Oct 2020
JAKET: Joint Pre-training of Knowledge Graph and Language Understanding Donghan Yu Chenguang Zhu Yiming Yang Michael Zeng KELM 78 144 0 02 Oct 2020
KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning Ye Liu Yao Wan Lifang He Hao Peng Philip S. Yu 108 189 0 26 Sep 2020
Taking Notes on the Fly Helps BERT Pre-training Qiyu Wu Chen Xing Yatao Li Guolin Ke Di He Tie-Yan Liu 49 10 0 04 Aug 2020
Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarization Beliz Gunel Chenguang Zhu Michael Zeng Xuedong Huang HILM 76 42 0 27 Jun 2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith VLM AI4CE CLL 175 2,444 0 23 Apr 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 383 1,494 0 18 Mar 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 404 18,914 0 13 Feb 2020
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters Ruize Wang Duyu Tang Nan Duan Zhongyu Wei Xuanjing Huang Jianshu Ji Guihong Cao Daxin Jiang Ming Zhou KELM 131 556 0 05 Feb 2020
A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation Jian Guan Fei Huang Zhihao Zhao Xiaoyan Zhu Minlie Huang LRM SyDa 82 250 0 15 Jan 2020
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model Wenhan Xiong Jingfei Du William Yang Wang Veselin Stoyanov SSL KELM 102 201 0 20 Dec 2019
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation Xiaozhi Wang Tianyu Gao Zhaocheng Zhu Zhengyan Zhang Zhiyuan Liu Juan-Zi Li Jian Tang 140 672 0 13 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 530 20,386 0 23 Oct 2019
K-BERT: Enabling Language Representation with Knowledge Graph Weijie Liu Peng Zhou Zhe Zhao Zhiruo Wang Qi Ju Haotang Deng Ping Wang 313 794 0 17 Sep 2019
On Mutual Information Maximization for Representation Learning Michael Tschannen Josip Djolonga Paul Kishan Rubenstein Sylvain Gelly Mario Lucic SSL 186 502 0 31 Jul 2019
Representation Degeneration Problem in Training Natural Language Generation Models Jun Gao Di He Xu Tan Tao Qin Liwei Wang Tie-Yan Liu 72 270 0 28 Jul 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 711 24,597 0 26 Jul 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 238 8,456 0 19 Jun 2019
ERNIE: Enhanced Language Representation with Informative Entities Zhengyan Zhang Xu Han Zhiyuan Liu Xin Jiang Maosong Sun Qun Liu 127 1,403 0 17 May 2019
On Variational Bounds of Mutual Information Ben Poole Sherjil Ozair Aaron van den Oord Alexander A. Alemi George Tucker SSL 114 816 0 16 May 2019
Rare Words: A Major Problem for Contextualized Embeddings And How to Fix it by Attentive Mimicking Timo Schick Hinrich Schütze 99 97 0 14 Apr 2019
BioBERT: a pre-trained biomedical language representation model for biomedical text mining Jinhyuk Lee Wonjin Yoon Sungdong Kim Donghyeon Kim Sunkyu Kim Chan Ho So Jaewoo Kang OOD 195 5,688 0 25 Jan 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,407 0 11 Oct 2018
FRAGE: Frequency-Agnostic Word Representation Chengyue Gong Di He Xu Tan Tao Qin Liwei Wang Tie-Yan Liu OOD 87 144 0 18 Sep 2018
Representation Learning with Contrastive Predictive Coding Aaron van den Oord Yazhe Li Oriol Vinyals DRL SSL 356 10,377 0 10 Jul 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,208 0 20 Apr 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 840 132,854 0 12 Jun 2017
Definition Modeling: Learning to define word embeddings in natural language Thanapon Noraset Chen Liang Larry Birnbaum Doug Downey 73 125 0 01 Dec 2016
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 2.2K 150,501 0 22 Dec 2014