Intriguing Properties of Compression on Multilingual Models

Intriguing Properties of Compression on Multilingual Models

4 November 2022

Orevaoghene Ahia

Gbemileke Onilude

Sebastian Gehrmann

Papers citing "Intriguing Properties of Compression on Multilingual Models"

18 / 18 papers shown

Title
As easy as PIE: understanding when pruning causes language models to disagree Pietro Tropeano Maria Maistro Tuukka Ruotsalo Christina Lioma 50 0 0 27 Mar 2025
You Never Know: Quantization Induces Inconsistent Biases in Vision-Language Foundation Models Eric Slyman Anirudh Kanneganti Sanghyun Hong Stefan Lee VLM MQ 19 0 0 26 Oct 2024
How Does Quantization Affect Multilingual LLMs? Kelly Marchisio Saurabh Dash Hongyu Chen Dennis Aumiller A. Ustun Sara Hooker Sebastian Ruder MQ 44 6 0 03 Jul 2024
Understanding the Effect of Model Compression on Social Bias in Large Language Models Gustavo Gonçalves Emma Strubell 12 9 0 09 Dec 2023
Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences Fred Hohman Mary Beth Kery Donghao Ren Dominik Moritz 16 6 0 06 Oct 2023
Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient Pruning of A Multilingual ASR Model Jiamin Xie Ke Li Jinxi Guo Andros Tjandra Shangguan Yuan Leda Sari Chunyang Wu J. Jia Jay Mahadeokar Ozlem Kalinli 10 2 0 22 Sep 2023
Evaluating the Social Impact of Generative AI Systems in Systems and Society Irene Solaiman Zeerak Talat William Agnew Lama Ahmad Dylan K. Baker ... Marie-Therese Png Shubham Singh A. Strait Lukas Struppek Arjun Subramonian ELM EGVM 23 101 0 09 Jun 2023
Intriguing Properties of Quantization at Scale Arash Ahmadian Saurabh Dash Hongyu Chen Bharat Venkitesh Stephen Gou Phil Blunsom A. Ustun Sara Hooker MQ 27 38 0 30 May 2023
FAIR-Ensemble: When Fairness Naturally Emerges From Deep Ensembling Wei-Yin Ko Daniel D'souza Karina Nguyen Randall Balestriero Sara Hooker FedML 11 11 0 01 Mar 2023
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 23 105 0 31 Aug 2022
What Do Compressed Multilingual Machine Translation Models Forget? Alireza Mohammadshahi Vassilina Nikoulina Alexandre Berard Caroline Brun James Henderson Laurent Besacier AI4CE 40 9 0 22 May 2022
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation Kaustubh D. Dhole Varun Gangal Sebastian Gehrmann Aadesh Gupta Zhenhao Li ... Tianbao Xie Usama Yaseen Michael A. Yee Jing Zhang Yue Zhang 156 86 0 06 Dec 2021
The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation Orevaoghene Ahia Julia Kreutzer Sara Hooker 102 50 0 06 Oct 2021
On the Prunability of Attention Heads in Multilingual BERT Aakriti Budhraja Madhura Pande Pratyush Kumar Mitesh M. Khapra 29 4 0 26 Sep 2021
BinaryBERT: Pushing the Limit of BERT Quantization Haoli Bai Wei Zhang Lu Hou Lifeng Shang Jing Jin Xin Jiang Qun Liu Michael Lyu Irwin King MQ 138 183 0 31 Dec 2020
Learning Light-Weight Translation Models from Deep Transformer Bei Li Ziyang Wang Hui Liu Quan Du Tong Xiao Chunliang Zhang Jingbo Zhu VLM 112 39 0 27 Dec 2020
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models Benjamin Muller Antonis Anastasopoulos Benoît Sagot Djamé Seddah LRM 106 165 0 24 Oct 2020
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT Sheng Shen Zhen Dong Jiayu Ye Linjian Ma Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer MQ 217 571 0 12 Sep 2019