SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization

8 November 2019

Xiaodong Liu

Papers citing "SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization"

50 / 77 papers shown

Title
IM-BERT: Enhancing Robustness of BERT through the Implicit Euler Method Mihyeon Kim Juhyoung Park Youngbin Kim 24 0 0 11 May 2025
Impeding LLM-assisted Cheating in Introductory Programming Assignments via Adversarial Perturbation Saiful Islam Salim Rubin Yuchan Yang Alexander Cooper Suryashree Ray Saumya Debray Sazzadur Rahaman AAML 42 0 0 12 Oct 2024
Robust LLM safeguarding via refusal feature adversarial training L. Yu Virginie Do Karen Hambardzumyan Nicola Cancedda AAML 56 10 0 30 Sep 2024
GAMedX: Generative AI-based Medical Entity Data Extractor Using Large Language Models Mohammed-Khalil Ghali Abdelrahman Farrag Hajar Sakai Hicham El Baz Yu Jin Sarah Lam LM&MA MedIm 32 8 0 31 May 2024
From Robustness to Improved Generalization and Calibration in Pre-trained Language Models Josip Jukić Jan Snajder 23 0 0 31 Mar 2024
Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space Leo Schwinn David Dobre Sophie Xhonneux Gauthier Gidel Stephan Gunnemann AAML 47 36 0 14 Feb 2024
Black-Box Access is Insufficient for Rigorous AI Audits Stephen Casper Carson Ezell Charlotte Siegmann Noam Kolt Taylor Lynn Curtis ... Michael Gerovitch David Bau Max Tegmark David M. Krueger Dylan Hadfield-Menell AAML 13 76 0 25 Jan 2024
Dynamic Corrective Self-Distillation for Better Fine-Tuning of Pretrained Models Ibtihel Amara Vinija Jain Aman Chadha 28 0 0 12 Dec 2023
Weigh Your Own Words: Improving Hate Speech Counter Narrative Generation via Attention Regularization Helena Bonaldi Giuseppe Attanasio Debora Nozza Marco Guerini 16 6 0 05 Sep 2023
Efficient Discovery and Effective Evaluation of Visual Perceptual Similarity: A Benchmark and Beyond Oren Barkan Tal Reiss Jonathan Weill Ori Katz Roy Hirsch Itzik Malkiel Noam Koenigstein 27 6 0 28 Aug 2023
Out-of-Distribution Generalization in Text Classification: Past, Present, and Future Linyi Yang Y. Song Xuan Ren Chenyang Lyu Yidong Wang Lingqiao Liu Jindong Wang Jennifer Foster Yue Zhang OOD 20 2 0 23 May 2023
SHINE: Syntax-augmented Hierarchical Interactive Encoder for Zero-shot Cross-lingual Information Extraction Jun-Yu Ma Jia-Chen Gu Zhen-Hua Ling Quan Liu Cong Liu Guoping Hu 47 1 0 21 May 2023
LabelPrompt: Effective Prompt-based Learning for Relation Classification W. Zhang Xiaoning Song Zhenhua Feng Tianyang Xu Xiaojun Wu VLM 22 4 0 16 Feb 2023
HateProof: Are Hateful Meme Detection Systems really Robust? Piush Aggarwal Pranit Chawla Mithun Das Punyajoy Saha Binny Mathew Torsten Zesch Animesh Mukherjee AAML 22 8 0 11 Feb 2023
ZhichunRoad at Amazon KDD Cup 2022: MultiTask Pre-Training for E-Commerce Product Search Xuange Cui Wei Xiong Songlin Wang 23 1 0 31 Jan 2023
WIDER & CLOSER: Mixture of Short-channel Distillers for Zero-shot Cross-lingual Named Entity Recognition Jun-Yu Ma Beiduo Chen Jia-Chen Gu Zhen-Hua Ling Wu Guo Quan Liu Zhigang Chen Cong Liu 29 10 0 07 Dec 2022
Finetune like you pretrain: Improved finetuning of zero-shot vision models Sachin Goyal Ananya Kumar Sankalp Garg Zico Kolter Aditi Raghunathan CLIP VLM 27 136 0 01 Dec 2022
Language Model Pre-training on True Negatives Zhuosheng Zhang Hai Zhao Masao Utiyama Eiichiro Sumita 22 2 0 01 Dec 2022
Alignment-Enriched Tuning for Patch-Level Pre-trained Document Image Models Lei Wang Jian He Xingdong Xu Ning Liu Hui-juan Liu 27 2 0 27 Nov 2022
Precisely the Point: Adversarial Augmentations for Faithful and Informative Text Generation Wenhao Wu Wei Li Jiachen Liu Xinyan Xiao Sujian Li Yajuan Lyu 24 3 0 22 Oct 2022
TCAB: A Large-Scale Text Classification Attack Benchmark Kalyani Asthana Zhouhang Xie Wencong You Adam Noack Jonathan Brophy Sameer Singh Daniel Lowd 22 3 0 21 Oct 2022
Surgical Fine-Tuning Improves Adaptation to Distribution Shifts Yoonho Lee Annie S. Chen Fahim Tajwar Ananya Kumar Huaxiu Yao Percy Liang Chelsea Finn OOD 47 197 0 20 Oct 2022
ROSE: Robust Selective Fine-tuning for Pre-trained Language Models Lan Jiang Hao Zhou Yankai Lin Peng Li Jie Zhou R. Jiang AAML 24 8 0 18 Oct 2022
Short Text Pre-training with Extended Token Classification for E-commerce Query Understanding Haoming Jiang Tianyu Cao Zheng Li Cheng-hsin Luo Xianfeng Tang Qingyu Yin Danqing Zhang R. Goutam Bing Yin RALM 16 11 0 08 Oct 2022
InFi: End-to-End Learning to Filter Input for Resource-Efficiency in Mobile-Centric Inference Mu Yuan Lan Zhang Fengxiang He Xueting Tong Miao-Hui Song Zhengyuan Xu Xiang-Yang Li 16 2 0 28 Sep 2022
Linear Transformations for Cross-lingual Sentiment Analysis Pavel Přibáň Jakub Šmíd Adam Mištera Pavel Král 20 3 0 15 Sep 2022
Socially Enhanced Situation Awareness from Microblogs using Artificial Intelligence: A Survey Rabindra Lamsal Aaron Harwood M. Read 32 20 0 13 Sep 2022
Multi-Level Fine-Tuning, Data Augmentation, and Few-Shot Learning for Specialized Cyber Threat Intelligence Markus Bayer Tobias Frey Christian A. Reuter AAML 16 15 0 22 Jul 2022
Domain Confused Contrastive Learning for Unsupervised Domain Adaptation Quanyu Long Tianze Luo Wenya Wang Sinno Jialin Pan 49 8 0 10 Jul 2022
Dual Decomposition of Convex Optimization Layers for Consistent Attention in Medical Images Tom Ron M. Weiler-Sagie Tamir Hazan FAtt MedIm 19 6 0 06 Jun 2022
TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understanding Le Zhang Zichao Yang Diyi Yang 26 24 0 12 May 2022
Few-shot Mining of Naturally Occurring Inputs and Outputs Mandar Joshi Terra Blevins M. Lewis Daniel S. Weld Luke Zettlemoyer 25 1 0 09 May 2022
Embedding Hallucination for Few-Shot Language Fine-tuning Yiren Jian Chongyang Gao Soroush Vosoughi 20 4 0 03 May 2022
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals Payal Bajaj Chenyan Xiong Guolin Ke Xiaodong Liu Di He Saurabh Tiwary Tie-Yan Liu Paul N. Bennett Xia Song Jianfeng Gao 42 32 0 13 Apr 2022
Impossible Triangle: What's Next for Pre-trained Language Models? Chenguang Zhu Michael Zeng 16 1 0 13 Apr 2022
Incremental Few-Shot Learning via Implanting and Compressing Yiting Li H. Zhu Xijia Feng Zilong Cheng Jun Ma Cheng Xiang P. Vadakkepat T. Lee CLL VLM 19 2 0 19 Mar 2022
USTC-NELSLIP at SemEval-2022 Task 11: Gazetteer-Adapted Integration Network for Multilingual Complex Named Entity Recognition Beiduo Chen Jun-Yu Ma Jiajun Qi Wu Guo Zhen-Hua Ling Quan Liu 18 16 0 07 Mar 2022
Amortized Proximal Optimization Juhan Bae Paul Vicol Jeff Z. HaoChen Roger C. Grosse ODL 22 14 0 28 Feb 2022
Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Review Kyle Hamilton Aparna Nayak Bojan Bozic Luca Longo NAI 21 57 0 24 Feb 2022
NoisyTune: A Little Noise Can Help You Finetune Pretrained Language Models Better Chuhan Wu Fangzhao Wu Tao Qi Yongfeng Huang Xing Xie 22 58 0 24 Feb 2022
GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records Xi Yang Aokun Chen Nima M. Pournejatian Hoo-Chang Shin Kaleb E. Smith ... Duane A. Mitchell W. Hogan E. Shenkman Jiang Bian Yonghui Wu AI4MH LM&MA 37 499 0 02 Feb 2022
Identifying Adversarial Attacks on Text Classifiers Zhouhang Xie Jonathan Brophy Adam Noack Wencong You Kalyani Asthana Carter Perkins Sabrina Reis Sameer Singh Daniel Lowd AAML 16 9 0 21 Jan 2022
Transferability in Deep Learning: A Survey Junguang Jiang Yang Shu Jianmin Wang Mingsheng Long OOD 17 100 0 15 Jan 2022
Sharpness-Aware Minimization with Dynamic Reweighting Wenxuan Zhou Fangyu Liu Huan Zhang Muhao Chen AAML 19 8 0 16 Dec 2021
Measure and Improve Robustness in NLP Models: A Survey Xuezhi Wang Haohan Wang Diyi Yang 139 130 0 15 Dec 2021
MNet-Sim: A Multi-layered Semantic Similarity Network to Evaluate Sentence Similarity Manuela Nayantara Jeyaraj D. Kasthurirathna 11 3 0 09 Nov 2021
Smooth Imitation Learning via Smooth Costs and Smooth Policies Sapana Chaudhary Balaraman Ravindran 16 1 0 03 Nov 2021
CLLD: Contrastive Learning with Label Distance for Text Classification Jinhe Lan Qingyuan Zhan Chenhao Jiang Kunping Yuan Desheng Wang VLM 29 2 0 25 Oct 2021
Sharpness-Aware Minimization Improves Language Model Generalization Dara Bahri H. Mobahi Yi Tay 119 98 0 16 Oct 2021
SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systems Harrison Lee Raghav Gupta Abhinav Rastogi Yuan Cao Bin Zhang Yonghui Wu 64 33 0 13 Oct 2021