Noise Stability Regularization for Improving BERT Fine-tuning

10 July 2021

Papers citing "Noise Stability Regularization for Improving BERT Fine-tuning"

27 / 27 papers shown

Title
Do we really have to filter out random noise in pre-training data for language models? Jinghan Ru Yuxin Xie Xianwei Zhuang Yuguo Yin Yuexian Zou 83 2 0 10 Feb 2025
Beyond Self-Consistency: Loss-Balanced Perturbation-Based Regularization Improves Industrial-Scale Ads Ranking Ilqar Ramazanli Hamid Eghbalzadeh Xiaoyi Liu Yang Wang Jiaxiang Fu Kaushik Rangadurai Sem Park Bo Long Xue Feng 44 0 0 05 Feb 2025
MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models Hang Hua Yunlong Tang Ziyun Zeng Liangliang Cao Zhengyuan Yang Hangfeng He Chenliang Xu Jiebo Luo VLM CoGe 36 9 0 13 Oct 2024
Co-Designing Binarized Transformer and Hardware Accelerator for Efficient End-to-End Edge Deployment Yuhao Ji Chao Fang Shaobo Ma Haikuo Shao Zhongfeng Wang MQ 36 1 0 16 Jul 2024
Fighting Randomness with Randomness: Mitigating Optimisation Instability of Fine-Tuning using Delayed Ensemble and Noisy Interpolation Branislav Pecher Ján Cegin Róbert Belanec Jakub Simko Ivan Srba M. Bieliková 37 1 0 18 Jun 2024
Generalization Measures for Zero-Shot Cross-Lingual Transfer Saksham Bassi Duygu Ataman Kyunghyun Cho 24 0 0 24 Apr 2024
FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction Hang Hua Jing Shi Kushal Kafle Simon Jenni Daoan Zhang John Collomosse Scott D. Cohen Jiebo Luo CoGe VLM 42 9 0 23 Apr 2024
V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuning Hang Hua Yunlong Tang Chenliang Xu Jiebo Luo VGen 60 25 0 18 Apr 2024
Video Understanding with Large Language Models: A Survey Yunlong Tang Jing Bi Siting Xu Luchuan Song Susan Liang ... Feng Zheng Jianguo Zhang Ping Luo Jiebo Luo Chenliang Xu VLM 50 82 0 29 Dec 2023
End-to-End Breast Cancer Radiotherapy Planning via LMMs with Consistency Embedding Kwanyoung Kim Y. Oh S. Park H. Byun Joongyo Lee Jin Sung Kim Yong Bae Kim Jong Chul Ye 23 0 0 27 Nov 2023
Rethinking the Construction of Effective Metrics for Understanding the Mechanisms of Pretrained Language Models You Li Jinhui Yin Yuming Lin 23 0 0 19 Oct 2023
FAM: Relative Flatness Aware Minimization Linara Adilova Amr Abourayya Jianning Li Amin Dada Henning Petzka Jan Egger Jens Kleesiek Michael Kamp ODL 16 1 0 05 Jul 2023
VideoXum: Cross-modal Visual and Textural Summarization of Videos Jingyang Lin Hang Hua Ming Chen Yikang Li Jenhao Hsiao C. Ho Jiebo Luo 28 30 0 21 Mar 2023
Measuring the Instability of Fine-Tuning Yupei Du D. Nguyen 18 4 0 15 Feb 2023
Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusion Haotian Ju Dongyue Li Aneesh Sharma Hongyang R. Zhang 19 40 0 09 Feb 2023
A Stability Analysis of Fine-Tuning a Pre-Trained Model Z. Fu Anthony Man-Cho So Nigel Collier 23 3 0 24 Jan 2023
HyPe: Better Pre-trained Language Model Fine-tuning with Hidden Representation Perturbation Hongyi Yuan Zheng Yuan Chuanqi Tan Fei Huang Songfang Huang 24 15 0 17 Dec 2022
On the Effectiveness of Parameter-Efficient Fine-Tuning Z. Fu Haoran Yang Anthony Man-Cho So Wai Lam Lidong Bing Nigel Collier 14 155 0 28 Nov 2022
BEBERT: Efficient and Robust Binary Ensemble BERT Jiayi Tian Chao Fang Hong Wang Zhongfeng Wang MQ 32 16 0 28 Oct 2022
We need to talk about random seeds Steven Bethard 31 8 0 24 Oct 2022
Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularization Hang Hua Xingjian Li Dejing Dou Chengzhong Xu Jiebo Luo 31 15 0 12 Jun 2022
Task-guided Disentangled Tuning for Pretrained Language Models Jiali Zeng Yu Jiang Shuangzhi Wu Yongjing Yin Mu Li DRL 17 3 0 22 Mar 2022
How Emotionally Stable is ALBERT? Testing Robustness with Stochastic Weight Averaging on a Sentiment Analysis Task Urja Khurana Eric T. Nalisnick Antske Fokkens MoMe 19 6 0 18 Nov 2021
Can the Transformer Be Used as a Drop-in Replacement for RNNs in Text-Generating GANs? Kevin Blin Andrei Kucharavy 8 2 0 26 Aug 2021
FreeLB: Enhanced Adversarial Training for Natural Language Understanding Chen Zhu Yu Cheng Zhe Gan S. Sun Tom Goldstein Jingjing Liu AAML 221 436 0 25 Sep 2019
Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models Cheolhyoung Lee Kyunghyun Cho Wanmo Kang MoE 235 205 0 25 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,950 0 20 Apr 2018