Distributionally Robust Language Modeling

4 September 2019

Yonatan Oren

Shiori Sagawa

Tatsunori B. Hashimoto

Percy Liang

OOD

ArXiv PDF HTML

Papers citing "Distributionally Robust Language Modeling"

38 / 38 papers shown

Title
Class-Conditional Distribution Balancing for Group Robust Classification Miaoyun Zhao Qiang Zhang C. Li 67 1 0 24 Apr 2025
Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasets Tianjian Li Haoran Xu Weiting Tan Kenton Murray Daniel Khashabi 35 1 0 06 Oct 2024
ExPLoRA: Parameter-Efficient Extended Pre-Training to Adapt Vision Transformers under Domain Shifts Samar Khanna Medhanie Irgau David B. Lobell Stefano Ermon VLM 32 4 0 16 Jun 2024
Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models Samuel Marks Can Rager Eric J. Michaud Yonatan Belinkov David Bau Aaron Mueller 44 111 0 28 Mar 2024
To Pool or Not To Pool: Analyzing the Regularizing Effects of Group-Fair Training on Shared Models Cyrus Cousins I. E. Kumar Suresh Venkatasubramanian FedML 34 1 0 29 Feb 2024
Group Distributionally Robust Dataset Distillation with Risk Minimization Saeed Vahidian Mingyu Wang Jianyang Gu Vyacheslav Kungurtsev Wei Jiang Yiran Chen OOD DD 33 6 0 07 Feb 2024
REDUCR: Robust Data Downsampling Using Class Priority Reweighting William Bankes George Hughes Ilija Bogunovic Zi Wang 18 3 0 01 Dec 2023
Bias Amplification Enhances Minority Group Performance Gaotang Li Jiarui Liu Wei Hu 28 5 0 13 Sep 2023
From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework Yangyi Chen Hongcheng Gao Ganqu Cui Lifan Yuan Dehan Kong ... Longtao Huang H. Xue Zhiyuan Liu Maosong Sun Heng Ji AAML ELM 25 6 0 29 May 2023
Out-of-Distribution Generalization in Text Classification: Past, Present, and Future Linyi Yang Y. Song Xuan Ren Chenyang Lyu Yidong Wang Lingqiao Liu Jindong Wang Jennifer Foster Yue Zhang OOD 32 2 0 23 May 2023
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining Sang Michael Xie Hieu H. Pham Xuanyi Dong Nan Du Hanxiao Liu Yifeng Lu Percy Liang Quoc V. Le Tengyu Ma Adams Wei Yu MoMe MoE 31 174 0 17 May 2023
Diffusion Theory as a Scalpel: Detecting and Purifying Poisonous Dimensions in Pre-trained Language Models Caused by Backdoor or Bias Zhiyuan Zhang Deli Chen Hao Zhou Fandong Meng Jie Zhou Xu Sun 28 5 0 08 May 2023
Hyper-parameter Tuning for Fair Classification without Sensitive Attribute Access A. Veldanda Ivan Brugere Sanghamitra Dutta Alan Mishler S. Garg 28 5 0 02 Feb 2023
Foundation models in brief: A historical, socio-technical focus Johannes Schneider VLM 26 9 0 17 Dec 2022
Improving group robustness under noisy labels using predictive uncertainty Dongpin Oh Dae Lee Jeunghyun Byun Bonggun Shin UQCV 18 3 0 14 Dec 2022
The future is different: Large pre-trained language models fail in prediction tasks K. Cvejoski Ramses J. Sanchez C. Ojeda 22 3 0 01 Nov 2022
COCO-DR: Combating Distribution Shifts in Zero-Shot Dense Retrieval with Contrastive and Distributionally Robust Learning Yue Yu Chenyan Xiong Si Sun Chao Zhang Arnold Overwijk VLM OOD 48 22 0 27 Oct 2022
Tencent's Multilingual Machine Translation System for WMT22 Large-Scale African Languages Wenxiang Jiao Zhaopeng Tu Jiarui Li Wenxuan Wang Jen-tse Huang Shuming Shi 46 15 0 18 Oct 2022
Learning from a Biased Sample Roshni Sahoo Lihua Lei Stefan Wager 19 17 0 05 Sep 2022
Learning to Split for Automatic Bias Detection Yujia Bao Regina Barzilay 12 20 0 28 Apr 2022
Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations Polina Kirichenko Pavel Izmailov A. Wilson OOD 34 314 0 06 Apr 2022
Evaluating Distributional Distortion in Neural Language Modeling Benjamin LeBrun Alessandro Sordoni Timothy J. O'Donnell 17 22 0 24 Mar 2022
Challenges and Strategies in Cross-Cultural NLP Daniel Hershcovich Stella Frank Heather Lent Miryam de Lhoneux Mostafa Abdou ... Ruixiang Cui Constanza Fierro Katerina Margatina Phillip Rust Anders Søgaard 43 162 0 18 Mar 2022
Towards Group Robustness in the presence of Partial Group Labels Vishnu Suresh Lokhande Kihyuk Sohn Jinsung Yoon Madeleine Udell Chen-Yu Lee Tomas Pfister OOD 29 11 0 10 Jan 2022
Measure and Improve Robustness in NLP Models: A Survey Xuezhi Wang Haohan Wang Diyi Yang 139 130 0 15 Dec 2021
Editing a classifier by rewriting its prediction rules Shibani Santurkar Dimitris Tsipras Mahalaxmi Elango David Bau Antonio Torralba A. Madry KELM 175 89 0 02 Dec 2021
A comparison of approaches to improve worst-case predictive model performance over patient subpopulations Stephen R. Pfohl Haoran Zhang Yizhe Xu Agata Foryciarz Marzyeh Ghassemi N. Shah OOD 23 22 0 27 Aug 2021
The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models G. dÉon Jason dÉon J. R. Wright Kevin Leyton-Brown 20 74 0 01 Jul 2021
Learning Stable Classifiers by Transferring Unstable Features Yujia Bao Shiyu Chang Regina Barzilay OOD 19 8 0 15 Jun 2021
Examining and Combating Spurious Features under Distribution Shift Chunting Zhou Xuezhe Ma Paul Michel Graham Neubig OOD 27 65 0 14 Jun 2021
Detoxifying Language Models Risks Marginalizing Minority Voices Albert Xu Eshaan Pathak Eric Wallace Suchin Gururangan Maarten Sap Dan Klein 13 121 0 13 Apr 2021
Label-Imbalanced and Group-Sensitive Classification under Overparameterization Ganesh Ramachandra Kini Orestis Paraskevas Samet Oymak Christos Thrampoulidis 27 93 0 02 Mar 2021
PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains Eyal Ben-David Nadav Oved Roi Reichart VLM OOD 14 88 0 24 Feb 2021
Mind the Gap: Assessing Temporal Generalization in Neural Language Models Angeliki Lazaridou A. Kuncoro E. Gribovskaya Devang Agrawal Adam Liska ... Sebastian Ruder Dani Yogatama Kris Cao Susannah Young Phil Blunsom VLM 30 207 0 03 Feb 2021
Utility is in the Eye of the User: A Critique of NLP Leaderboards Kawin Ethayarajh Dan Jurafsky ELM 19 51 0 29 Sep 2020
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models Lifu Tu Garima Lalwani Spandana Gella He He LRM 19 184 0 14 Jul 2020
Measuring Robustness to Natural Distribution Shifts in Image Classification Rohan Taori Achal Dave Vaishaal Shankar Nicholas Carlini Benjamin Recht Ludwig Schmidt OOD 22 530 0 01 Jul 2020
An Investigation of Why Overparameterization Exacerbates Spurious Correlations Shiori Sagawa Aditi Raghunathan Pang Wei Koh Percy Liang 146 370 0 09 May 2020