BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance

7 November 2019

Papers citing "BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance"

37 / 37 papers shown

Title
(How) Do Language Models Track State? Belinda Z. Li Zifan Carl Guo Jacob Andreas LRM 46 0 0 04 Mar 2025
Distributional Scaling Laws for Emergent Capabilities Rosie Zhao Tian Qin David Alvarez-Melis Sham Kakade Naomi Saphra LRM 39 0 0 24 Feb 2025
The Curious Case of Arbitrariness in Machine Learning Prakhar Ganesh Afaf Taik G. Farnadi 59 2 0 28 Jan 2025
Survival of the Fittest Representation: A Case Study with Modular Addition Xiaoman Delores Ding Zifan Carl Guo Eric J. Michaud Ziming Liu Max Tegmark 48 3 0 27 May 2024
Acquiring Linguistic Knowledge from Multimodal Input Theodor Amariucai Alexander Scott Warstadt CLL 29 2 0 27 Feb 2024
Punctuation Restoration Improves Structure Understanding Without Supervision Junghyun Min Minho Lee Woochul Lee Yeonsoo Lee 59 1 0 13 Feb 2024
Position: Key Claims in LLM Research Have a Long Tail of Footnotes Anna Rogers A. Luccioni 53 19 0 14 Aug 2023
On The Impact of Machine Learning Randomness on Group Fairness Prakhar Ganesh Hong Chang Martin Strobel Reza Shokri FaML 30 30 0 09 Jul 2023
Cross-lingual Transfer Can Worsen Bias in Sentiment Analysis Seraphina Goldfarb-Tarrant Bjorn Ross Adam Lopez 33 7 0 22 May 2023
Similarity of Neural Network Models: A Survey of Functional and Representational Measures Max Klabunde Tobias Schumacher M. Strohmaier Florian Lemmerich 55 64 0 10 May 2023
Evaluating the Robustness of Machine Reading Comprehension Models to Low Resource Entity Renaming Clemencia Siro T. Ajayi 20 2 0 06 Apr 2023
Learning the Effects of Physical Actions in a Multi-modal Environment Gautier Dagan Frank Keller A. Lascarides LM&Ro 32 3 0 27 Jan 2023
Probing with Noise: Unpicking the Warp and Weft of Embeddings Filip Klubicka John D. Kelleher 30 4 0 21 Oct 2022
Monotonic Risk Relationships under Distribution Shifts for Regularized Risk Minimization Daniel LeJeune Jiayu Liu Reinhard Heckel 23 0 0 20 Oct 2022
GULP: a prediction-based metric between representations Enric Boix Adserà Hannah Lawrence George Stepaniants Philippe Rigollet 46 11 0 12 Oct 2022
State-of-the-art generalisation research in NLP: A taxonomy and review Dieuwke Hupkes Mario Giulianelli Verna Dankers Mikel Artetxe Yanai Elazar ... Leila Khalatbari Maria Ryskina Rita Frieske Ryan Cotterell Zhijing Jin 114 93 0 06 Oct 2022
Lost in Context? On the Sense-wise Variance of Contextualized Word Embeddings Yile Wang Yue Zhang 19 4 0 20 Aug 2022
Linear Connectivity Reveals Generalization Strategies Jeevesh Juneja Rachit Bansal Kyunghyun Cho João Sedoc Naomi Saphra 242 45 0 24 May 2022
mGPT: Few-Shot Learners Go Multilingual Oleh Shliazhko Alena Fenogenova Maria Tikhonova Vladislav Mikhailov Anastasia Kozlova Tatiana Shavrina 43 149 0 15 Apr 2022
Reducing Model Jitter: Stable Re-training of Semantic Parsers in Production Environments Christopher Hidey Fei Liu Rahul Goel 24 4 0 10 Apr 2022
Data-driven Model Generalizability in Crosslinguistic Low-resource Morphological Segmentation Zoey Liu Emily Tucker Prudhommeaux 43 4 0 05 Jan 2022
Building Human-like Communicative Intelligence: A Grounded Perspective M. Dubova 29 12 0 02 Jan 2022
How Emotionally Stable is ALBERT? Testing Robustness with Stochastic Weight Averaging on a Sentiment Analysis Task Urja Khurana Eric T. Nalisnick Antske Fokkens MoMe 27 6 0 18 Nov 2021
The Grammar-Learning Trajectories of Neural Language Models Leshem Choshen Guy Hacohen D. Weinshall Omri Abend 29 28 0 13 Sep 2021
Debiasing Methods in Natural Language Understanding Make Bias More Accessible Michael J. Mendelson Yonatan Belinkov 40 23 0 09 Sep 2021
Teaching Autoregressive Language Models Complex Tasks By Demonstration Gabriel Recchia 26 22 0 05 Sep 2021
The MultiBERTs: BERT Reproductions for Robustness Analysis Thibault Sellam Steve Yadlowsky Jason W. Wei Naomi Saphra Alexander DÁmour ... Iulia Turc Jacob Eisenstein Dipanjan Das Ian Tenney Ellie Pavlick 24 93 0 30 Jun 2021
How Reliable are Model Diagnostics? V. Aribandi Yi Tay Donald Metzler 19 19 0 12 May 2021
The NLP Cookbook: Modern Recipes for Transformer based Deep Learning Architectures Sushant Singh A. Mahmood AI4TS 60 92 0 23 Mar 2021
How Many Data Points is a Prompt Worth? Teven Le Scao Alexander M. Rush VLM 57 296 0 15 Mar 2021
Underspecification Presents Challenges for Credibility in Modern Machine Learning Alexander DÁmour Katherine A. Heller D. Moldovan Ben Adlam B. Alipanahi ... Kellie Webster Steve Yadlowsky T. Yun Xiaohua Zhai D. Sculley OffRL 77 670 0 06 Nov 2020
Improving Robustness by Augmenting Training Sentences with Predicate-Argument Structures N. Moosavi M. Boer Prasetya Ajie Utama Iryna Gurevych 19 13 0 23 Oct 2020
Compositional Networks Enable Systematic Generalization for Grounded Language Understanding Yen-Ling Kuo Boris Katz Andrei Barbu 31 22 0 06 Aug 2020
Syntactic Data Augmentation Increases Robustness to Inference Heuristics Junghyun Min R. Thomas McCoy Dipanjan Das Emily Pitler Tal Linzen 30 175 0 24 Apr 2020
Adversarial Filters of Dataset Biases Ronan Le Bras Swabha Swayamdipta Chandra Bhagavatula Rowan Zellers Matthew E. Peters Ashish Sabharwal Yejin Choi 36 220 0 10 Feb 2020
The Fine Line between Linguistic Generalization and Failure in Seq2Seq-Attention Models Noah Weber L. Shekhar Niranjan Balasubramanian 98 30 0 03 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018