Syntactic Structure Distillation Pretraining For Bidirectional Encoders

Syntactic Structure Distillation Pretraining For Bidirectional Encoders

27 May 2020

Lingpeng Kong

Daniel Fried

ArXiv (abs)PDF HTML

Papers citing "Syntactic Structure Distillation Pretraining For Bidirectional Encoders"

16 / 16 papers shown

Title
Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examples Hezekiah J. Branch Jonathan Rodriguez Cefalu Jeremy McHugh Leyla Hujer Aditya Bahl Daniel del Castillo Iglesias Ron Heichman Ramesh Darwishi ELM SILM AAML 106 64 0 05 Sep 2022
Lexical Knowledge Internalization for Neural Dialog Generation Zhiyong Wu Wei Bi Xiang Li Lingpeng Kong B. Kao 83 2 0 04 May 2022
Syntax-informed Question Answering with Heterogeneous Graph Transformer Fangyi Zhu Lok You Tan See-Kiong Ng S. Bressan 112 3 0 01 Apr 2022
Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale Laurent Sartran Samuel Barrett A. Kuncoro Milovs Stanojević Phil Blunsom Chris Dyer 142 54 0 01 Mar 2022
Schrödinger's Tree -- On Syntax and Neural Language Models Artur Kulmizev Joakim Nivre 87 6 0 17 Oct 2021
Improved Latent Tree Induction with Distant Supervision via Span Constraints Zhiyang Xu Andrew Drozdov Jay Yoon Lee Timothy J. O'Gorman Subendhu Rongali Dylan Finkbeiner S. Suresh Mohit Iyyer Andrew McCallum 96 9 0 10 Sep 2021
Structural Guidance for Transformer Language Models Peng Qian Tahira Naseem R. Levy Ramón Fernández Astudillo 141 32 0 30 Jul 2021
Effective Batching for Recurrent Neural Network Grammars Hiroshi Noji Yohei Oseki GNN 86 18 0 31 May 2021
Syntactic Perturbations Reveal Representational Correlates of Hierarchical Phrase Structure in Pretrained Language Models Matteo Alleman J. Mamou Miguel Rio Hanlin Tang Yoon Kim SueYeon Chung NAI 130 17 0 15 Apr 2021
Structure Inducing Pre-Training Matthew B. A. McDermott Brendan Yap Peter Szolovits Marinka Zitnik 156 24 0 18 Mar 2021
On the Evolution of Syntactic Information Encoded by BERT's Contextualized Representations Laura Pérez-Mayos Roberto Carlini Miguel Ballesteros Leo Wanner 64 8 0 27 Jan 2021
Infusing Finetuning with Semantic Dependencies Zhaofeng Wu Hao Peng Noah A. Smith 100 37 0 10 Dec 2020
Do Syntax Trees Help Pre-trained Transformers Extract Information? Devendra Singh Sachan Yuhao Zhang Peng Qi William L. Hamilton 113 82 0 20 Aug 2020
Is Supervised Syntactic Parsing Beneficial for Language Understanding? An Empirical Investigation Goran Glavaš Ivan Vulić 126 69 0 15 Aug 2020
Transferring Inductive Biases through Knowledge Distillation Samira Abnar Mostafa Dehghani Willem H. Zuidema 143 61 0 31 May 2020
Towards Instance-Level Parser Selection for Cross-Lingual Transfer of Dependency Parsers Robert Litschko Ivan Vulić Zeljko Agic Goran Glavaš 63 5 0 16 Apr 2020