StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding

13 August 2019

Papers citing "StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding"

41 / 41 papers shown

Title
Beyond Semantics: Learning a Behavior Augmented Relevance Model with Self-supervised Learning Ze-jie Chen Wei-Neng Chen Jia Xu Zhongyi Liu Wei Zhang RALM 23 4 0 10 Aug 2023
On the (In)Effectiveness of Large Language Models for Chinese Text Correction Yinghui Li Haojing Huang Shirong Ma Yong-jia Jiang Y. Li F. Zhou Haitao Zheng Qingyu Zhou 31 43 0 18 Jul 2023
QUERT: Continual Pre-training of Language Model for Query Understanding in Travel Domain Search Jian Xie Yidan Liang Jingping Liu Yanghua Xiao Baohua Wu Shenghua Ni VLM LRM 30 8 0 11 Jun 2023
Zero-Shot Text Classification via Self-Supervised Tuning Chaoqun Liu Wenxuan Zhang Guizhen Chen Xiaobao Wu A. Luu Chip Hong Chang Lidong Bing VLM 32 11 0 19 May 2023
GeoGLUE: A GeoGraphic Language Understanding Evaluation Benchmark Dongyang Li Ruixue Ding Qiang-Wei Zhang Zheng Li Boli Chen ... Yao Xu Xin Li Ning Guo Fei Huang Xiaofeng He ELM VLM 29 5 0 11 May 2023
Going beyond research datasets: Novel intent discovery in the industry setting Aleksandra Chrabrowa Tsimur Hadeliya D. Kajtoch Robert Mroczkowski Piotr Rybak 8 2 0 09 May 2023
Interpretable multimodal sentiment analysis based on textual modality descriptions by using large-scale language models Sixia Li S. Okada 30 3 0 07 May 2023
Overview of the ICASSP 2023 General Meeting Understanding and Generation Challenge (MUG) Qinglin Zhang Chong Deng Jiaqing Liu Hai Yu Qian Chen Wen Wang Zhijie Yan Jinglin Liu Yi Ren Zhou Zhao 38 0 0 24 Mar 2023
A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT Yihan Cao Siyu Li Yixin Liu Zhiling Yan Yutong Dai Philip S. Yu Lichao Sun 29 504 0 07 Mar 2023
Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao AI4MH 52 238 0 19 Feb 2023
A Concept Knowledge Graph for User Next Intent Prediction at Alipay Yacheng He Qianghuai Jia Lin Yuan Ruopeng Li Yixin Ou Ningyu Zhang 13 5 0 02 Jan 2023
Paraphrase Identification with Deep Learning: A Review of Datasets and Methods Chao Zhou Cheng Qiu Daniel Ernesto Acuna 29 25 0 13 Dec 2022
Language Model Pre-training on True Negatives Zhuosheng Zhang Hai Zhao Masao Utiyama Eiichiro Sumita 22 2 0 01 Dec 2022
FCGEC: Fine-Grained Corpus for Chinese Grammatical Error Correction Lvxiaowei Xu Jian Wu Jiawei Peng Jiayu Fu Ming Cai 30 13 0 22 Oct 2022
Linguistic Rules-Based Corpus Generation for Native Chinese Grammatical Error Correction Shirong Ma Yinghui Li Rongyi Sun Qingyu Zhou Shulin Huang ... Ruiyang Liu Zhongli Li Yunbo Cao Haitao Zheng Ying Shen 13 26 0 19 Oct 2022
Knowing Where and What: Unified Word Block Pretraining for Document Understanding Song Tao Zijian Wang Tiantian Fan Canjie Luo Can Huang SSL 27 2 0 28 Jul 2022
Attention Mechanism in Neural Networks: Where it Comes and Where it Goes Derya Soydaner 3DV 36 149 0 27 Apr 2022
MuCGEC: a Multi-Reference Multi-Source Evaluation Dataset for Chinese Grammatical Error Correction Yue Zhang Zhenghua Li Zuyi Bao Jiacheng Li Bo-Wen Zhang Chen Li Fei Huang Min Zhang ELM 18 50 0 23 Apr 2022
Cross-Domain Generalization and Knowledge Transfer in Transformers Trained on Legal Data Jaromír Šavelka Hannes Westermann Karim Benyekhlef 23 15 0 15 Dec 2021
Linking-Enhanced Pre-Training for Table Semantic Parsing Bowen Qin Lihan Wang Binyuan Hui Ruiying Geng Zhen Cao Min Yang Jian Sun Yongbin Li 29 1 0 18 Nov 2021
Achieving Human Parity on Visual Question Answering Ming Yan Haiyang Xu Chenliang Li Junfeng Tian Bin Bi ... Ji Zhang Songfang Huang Fei Huang Luo Si Rong Jin 24 12 0 17 Nov 2021
ICDAR 2021 Competition on Document VisualQuestion Answering Rubèn Pérez Tito Minesh Mathew C. V. Jawahar Ernest Valveny Dimosthenis Karatzas 35 23 0 10 Nov 2021
MNet-Sim: A Multi-layered Semantic Similarity Network to Evaluate Sentence Similarity Manuela Nayantara Jeyaraj D. Kasthurirathna 11 3 0 09 Nov 2021
Monolingual and Cross-Lingual Acceptability Judgments with the Italian CoLA corpus Daniela Trotta R. Guarasci Elisa Leonardelli Sara Tonelli 42 30 0 24 Sep 2021
K-AID: Enhancing Pre-trained Language Models with Domain Knowledge for Question Answering Fu Sun Feng-Lin Li Ruize Wang Qianglong Chen Xingyi Cheng Ji Zhang VLM KELM 28 4 0 22 Sep 2021
AliMe MKG: A Multi-modal Knowledge Graph for Live-streaming E-commerce Guohai Xu Hehong Chen Feng-Lin Li Fu Sun Yunzhou Shi Zhixiong Zeng Wei Zhou Zhongzhou Zhao Ji Zhang 14 16 0 13 Sep 2021
Frustratingly Simple Pretraining Alternatives to Masked Language Modeling Atsuki Yamaguchi G. Chrysostomou Katerina Margatina Nikolaos Aletras 22 25 0 04 Sep 2021
How to Query Language Models? Leonard Adolphs S. Dhuliawala Thomas Hofmann KELM 16 15 0 04 Aug 2021
Memorization in Deep Neural Networks: Does the Loss Function matter? Deep Patel P. Sastry TDI 13 8 0 21 Jul 2021
HerBERT: Efficiently Pretrained Transformer-based Language Model for Polish Robert Mroczkowski Piotr Rybak Alina Wróblewska Ireneusz Gawlik 28 81 0 04 May 2021
Progressively Stacking 2.0: A Multi-stage Layerwise Training Method for BERT Training Speedup Cheng Yang Shengnan Wang Chao Yang Yuechuan Li Ru He Jingqiao Zhang 24 25 0 27 Nov 2020
CAPT: Contrastive Pre-Training for Learning Denoised Sequence Representations Fuli Luo Pengcheng Yang Shicheng Li Xuancheng Ren Xu Sun VLM SSL 13 16 0 13 Oct 2020
On Losses for Modern Language Models Stephane Aroca-Ouellette Frank Rudzicz 11 33 0 04 Oct 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 62 2,614 0 05 Jun 2020
Syntactic Structure Distillation Pretraining For Bidirectional Encoders A. Kuncoro Lingpeng Kong Daniel Fried Dani Yogatama Laura Rimell Chris Dyer Phil Blunsom 31 33 0 27 May 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 243 1,450 0 18 Mar 2020
Sentence Meta-Embeddings for Unsupervised Semantic Textual Similarity Nina Poerner Ulli Waltinger Hinrich Schütze AI4TS 24 20 0 09 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 71 19,422 0 23 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 59 6,370 0 26 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,743 0 26 Sep 2016