Pretrained Language Model Embryology: The Birth of ALBERT

6 October 2020

Papers citing "Pretrained Language Model Embryology: The Birth of ALBERT"

10 / 10 papers shown

Title
Gender Biases Unexpectedly Fluctuate in the Pre-training Stage of Masked Language Models Kenan Tang Hanchun Jiang AI4CE 18 1 0 26 Nov 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models Kushal Tirumala Aram H. Markosyan Luke Zettlemoyer Armen Aghajanyan TDI 26 185 0 22 May 2022
Silence is Sweeter Than Speech: Self-Supervised Model Using Silence to Store Speaker Information Chiyu Feng Po-Chun Hsu Hung-yi Lee SSL 20 8 0 08 May 2022
AraBART: a Pretrained Arabic Sequence-to-Sequence Model for Abstractive Summarization Moussa Kamal Eddine Nadi Tomeh Nizar Habash Joseph Le Roux Michalis Vazirgiannis 22 44 0 21 Mar 2022
Interpreting Language Models Through Knowledge Graph Extraction Vinitra Swamy Angelika Romanou Martin Jaggi 26 20 0 16 Nov 2021
The MultiBERTs: BERT Reproductions for Robustness Analysis Thibault Sellam Steve Yadlowsky Jason W. Wei Naomi Saphra Alexander DÁmour ... Iulia Turc Jacob Eisenstein Dipanjan Das Ian Tenney Ellie Pavlick 24 93 0 30 Jun 2021
IIITT@LT-EDI-EACL2021-Hope Speech Detection: There is always Hope in Transformers Karthik Puranik Adeep Hande R. Priyadharshini Sajeetha Thavareesan Bharathi Raja Chakravarthi 17 59 0 19 Apr 2021
When Do You Need Billions of Words of Pretraining Data? Yian Zhang Alex Warstadt Haau-Sing Li Samuel R. Bowman 21 136 0 10 Nov 2020
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 415 2,586 0 03 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018