v1v2 (latest)

Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization

25 May 2021

Xiaodong Liu

Papers citing "Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization"

23 / 23 papers shown

Title
Adaptive LoRA Merge with Parameter Pruning for Low-Resource Generation Ryota Miyano Yuki Arase MoMe 25 0 0 30 May 2025
Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmetic Yifei He Yuzheng Hu Yong Lin Tong Zhang Han Zhao FedML MoMe 129 25 0 08 Jan 2025
Edge AI: A Taxonomy, Systematic Review and Future Directions S. Gill Muhammed Golec Jianmin Hu Minxian Xu Junhui Du ... Kejiang Ye Prabal Verma Surendra Kumar Félix Cuadrado Steve Uhlig 96 35 0 04 Jul 2024
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation Yixiao Li Yifan Yu Qingru Zhang Chen Liang Pengcheng He Weizhu Chen Tuo Zhao 120 76 0 20 Jun 2023
Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations Lifan Yuan Yangyi Chen Ganqu Cui Hongcheng Gao Fangyuan Zou Xingyi Cheng Heng Ji Zhiyuan Liu Maosong Sun 148 84 0 07 Jun 2023
Pruning Pre-trained Language Models with Principled Importance and Self-regularization Siyu Ren Kenny Q. Zhu 79 2 0 21 May 2023
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers Chen Liang Haoming Jiang Zheng Li Xianfeng Tang Bin Yin Tuo Zhao VLM 134 25 0 19 Feb 2023
Gradient-based Intra-attention Pruning on Pre-trained Language Models Ziqing Yang Yiming Cui Xin Yao Shijin Wang VLM 73 12 0 15 Dec 2022
Robust Lottery Tickets for Pre-trained Language Models Rui Zheng Rong Bao Yuhao Zhou Di Liang Sirui Wang Wei Wu Tao Gui Qi Zhang Xuanjing Huang AAML 87 14 0 06 Nov 2022
Compressing And Debiasing Vision-Language Pre-Trained Models for Visual Question Answering Q. Si Yuanxin Liu Zheng Lin Peng Fu Weiping Wang VLM 120 1 0 26 Oct 2022
Pruning Pre-trained Language Models Without Fine-Tuning Ting Jiang Deqing Wang Fuzhen Zhuang Ruobing Xie Feng Xia 131 10 0 12 Oct 2022
A Win-win Deal: Towards Sparse and Robust Pre-trained Language Models Yuanxin Liu Fandong Meng Zheng Lin JiangNan Li Peng Fu Yanan Cao Weiping Wang Jie Zhou 87 6 0 11 Oct 2022
State-of-the-art generalisation research in NLP: A taxonomy and review Dieuwke Hupkes Mario Giulianelli Verna Dankers Mikel Artetxe Yanai Elazar ... Leila Khalatbari Maria Ryskina Rita Frieske Ryan Cotterell Zhijing Jin 280 99 0 06 Oct 2022
Doge Tickets: Uncovering Domain-general Language Models by Playing Lottery Tickets Yi Yang Chen Zhang Benyou Wang Dawei Song LRM 100 6 0 20 Jul 2022
Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models Clara Na Sanket Vaibhav Mehta Emma Strubell 116 20 0 25 May 2022
Diverse Lottery Tickets Boost Ensemble from a Single Pretrained Model Sosuke Kobayashi Shun Kiyono Jun Suzuki Kentaro Inui MoMe 77 9 0 24 May 2022
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation Simiao Zuo Qingru Zhang Chen Liang Pengcheng He T. Zhao Weizhu Chen MoE 195 41 0 15 Apr 2022
Probing Structured Pruning on Multilingual Pre-trained Models: Settings, Algorithms, and Efficiency Yanyang Li Fuli Luo Runxin Xu Songfang Huang Fei Huang Liwei Wang 69 3 0 06 Apr 2022
No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models Chen Liang Haoming Jiang Simiao Zuo Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen T. Zhao 72 14 0 06 Feb 2022
From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression Runxin Xu Fuli Luo Chengyu Wang Baobao Chang Jun Huang Songfang Huang Fei Huang VLM 64 26 0 14 Dec 2021
Pruning Pretrained Encoders with a Multitask Objective Patrick Xia Richard Shin 67 0 0 10 Dec 2021
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey Bonan Min Hayley L Ross Elior Sulem Amir Pouran Ben Veyseh Thien Huu Nguyen Oscar Sainz Eneko Agirre Ilana Heinz Dan Roth LM&MA VLM AI4CE 197 1,100 0 01 Nov 2021
AxFormer: Accuracy-driven Approximation of Transformers for Faster, Smaller and more Accurate NLP Models Amrit Nagarajan Sanchari Sen Jacob R. Stevens A. Raghunathan 39 3 0 07 Oct 2020