v1v2v3 (latest)

mT5: A massively multilingual pre-trained text-to-text transformer

22 October 2020

ArXiv (abs)PDF HTML HuggingFace (4 upvotes)

Papers citing "mT5: A massively multilingual pre-trained text-to-text transformer"

50 / 1,560 papers shown

Title
On the Universality of Deep Contextual Language Models Shaily Bhatt Poonam Goyal Sandipan Dandapat Monojit Choudhury Sunayana Sitaram ELM 174 5 0 15 Sep 2021
Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extraction M. Yarmohammadi Shijie Wu Marc Marone Haoran Xu Seth Ebner ... Craig Harman Kenton W. Murray Aaron Steven White Mark Dredze Benjamin Van Durme 132 30 0 14 Sep 2021
MDAPT: Multilingual Domain Adaptive Pretraining in a Single Model Rasmus Jorgensen Mareike Hartmann Xiang Dai Desmond Elliott AI4CE 119 13 0 14 Sep 2021
Exploring a Unified Sequence-To-Sequence Transformer for Medical Product Safety Monitoring in Social Media Shivam Raval Hooman Sedghamiz Enrico Santus Tuka Alhanai M. Ghassemi Emmanuele Chersoni 100 22 0 13 Sep 2021
CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation Yunfan Shao Zhichao Geng Yitao Liu Junqi Dai Hang Yan Fei Yang Li Zhe Hujun Bao Xipeng Qiu MedIm 227 166 0 13 Sep 2021
Multilingual Translation via Grafting Pre-trained Language Models Zewei Sun Mingxuan Wang Lei Li AI4CE 389 22 0 11 Sep 2021
AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African LanguagesConference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Machel Reid Junjie Hu Graham Neubig Y. Matsuo 186 37 0 10 Sep 2021
What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained TransformersConference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Boseop Kim Hyoungseok Kim Sang-Woo Lee Gichang Lee Donghyun Kwak ... Jaewook Kang Inho Kang Jung-Woo Ha W. Park Nako Sung VLM 515 128 0 10 Sep 2021
PPT: Pre-trained Prompt Tuning for Few-shot LearningAnnual Meeting of the Association for Computational Linguistics (ACL), 2021 Yuxian Gu Xu Han Zhiyuan Liu Shiyu Huang VLM 302 464 0 09 Sep 2021
Translate & Fill: Improving Zero-Shot Multilingual Semantic Parsing with Synthetic DataConference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Massimo Nicosia Zhongdi Qu Yasemin Altun 132 26 0 09 Sep 2021
ARMAN: Pre-training with Semantically Selecting and Reordering of Sentences for Persian Abstractive SummarizationConference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Alireza Salemi Emad Kebriaei Ghazal Neisi Minaei A. Shakery CVBM 94 6 0 09 Sep 2021
Smelting Gold and Silver for Improved Multilingual AMR-to-Text GenerationConference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Leonardo F. R. Ribeiro Jonas Pfeiffer Yue Zhang Iryna Gurevych 206 11 0 08 Sep 2021
IndicBART: A Pre-trained Model for Indic Natural Language GenerationFindings (Findings), 2021 Mary Dabre Himani Shrotriya Anoop Kunchukuttan Ratish Puduppully Mitesh M. Khapra Pratyush Kumar 221 86 0 07 Sep 2021
You should evaluate your language model on marginal likelihood over tokenisationsConference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Kris Cao Laura Rimell 243 30 0 06 Sep 2021
Nearest Neighbour Few-Shot Learning for Cross-lingual Classification M Saiful Bari Batool Haider Saab Mansour VLM 90 15 0 06 Sep 2021
MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer Ilias Chalkidis Manos Fergadiotis Ion Androutsopoulos AILaw 343 130 0 02 Sep 2021
ReMeDi: Resources for Multi-domain, Multi-service, Medical DialoguesAnnual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2021 Guojun Yan Jiahuan Pei Sudipta Singha Roy Zhaochun Ren Xin Xin Huasheng Liang Maarten de Rijke Zhumin Chen 270 29 0 01 Sep 2021
Discovering Representation Sprachbund For Multilingual Pre-TrainingConference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Yiming Fan Yaobo Liang Alexandre Muzio Hany Hassan Houqiang Li Ming Zhou Nan Duan 104 10 0 01 Sep 2021
Boosting Cross-Lingual Transfer via Self-Learning with Uncertainty EstimationConference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Liyan Xu Xuchao Zhang Xujiang Zhao Haifeng Chen F. Chen Jinho Choi 121 15 0 01 Sep 2021
mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset L. Bonifacio Vitor Jeronymo Hugo Queiroz Abonizio Israel Campiotti Marzieh Fadaee R. Lotufo Rodrigo Nogueira 398 134 0 31 Aug 2021
AraT5: Text-to-Text Transformers for Arabic Language GenerationAnnual Meeting of the Association for Computational Linguistics (ACL), 2021 El Moatez Billah Nagoudi AbdelRahim Elmadany Muhammad Abdul-Mageed 287 153 0 31 Aug 2021
LOT: A Story-Centric Benchmark for Evaluating Chinese Long Text Understanding and GenerationTransactions of the Association for Computational Linguistics (TACL), 2021 Jian Guan Zhuoer Feng Yamei Chen Ru He Xiaoxi Mao Changjie Fan Shiyu Huang 189 34 0 30 Aug 2021
Code-switched inspired losses for generic spoken dialog representationsConference on Empirical Methods in Natural Language Processing (EMNLP), 2021 E. Chapuis Pierre Colombo Matthieu Labeau Chloe Clave 322 12 0 27 Aug 2021
Multi-Task Self-Training for Learning General RepresentationsIEEE International Conference on Computer Vision (ICCV), 2021 Golnaz Ghiasi Barret Zoph E. D. Cubuk Quoc V. Le Nayeon Lee SSL 180 110 0 25 Aug 2021
YANMTT: Yet Another Neural Machine Translation ToolkitAnnual Meeting of the Association for Computational Linguistics (ACL), 2021 Mary Dabre Eiichiro Sumita 201 14 0 25 Aug 2021
Are the Multilingual Models Better? Improving Czech Sentiment with TransformersRecent Advances in Natural Language Processing (RANLP), 2021 Pavel Přibáň J. Steinberger 134 11 0 24 Aug 2021
A Unified Transformer-based Framework for Duplex Text Normalization T. Lai Yang Zhang Evelina Bakhturina Boris Ginsburg Heng Ji 72 12 0 23 Aug 2021
Language Model Augmented Relevance Score Ruibo Liu Jason W. Wei Soroush Vosoughi 131 11 0 19 Aug 2021
MTG: A Benchmark Suite for Multilingual Text Generation Yiran Chen Zhenqiao Song Xianze Wu Danqing Wang Jingjing Xu Jiaze Chen Hao Zhou Lei Li LRM VLM 200 23 0 13 Aug 2021
AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing Katikapalli Subramanyam Kalyan A. Rajasekharan S. Sangeetha VLM LM&MA 287 308 0 12 Aug 2021
Compositional Generalization in Multilingual Semantic Parsing over WikidataTransactions of the Association for Computational Linguistics (TACL), 2021 Ruixiang Cui Rahul Aralikatte Heather Lent Daniel Hershcovich 220 15 0 07 Aug 2021
PARADISE: Exploiting Parallel Data for Multilingual Sequence-to-Sequence Pretraining Machel Reid Mikel Artetxe VLM 191 30 0 04 Aug 2021
Towards Universality in Multilingual Text Rewriting Xavier Garcia Noah Constant Mandy Guo Orhan Firat LRM 167 11 0 30 Jul 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language ProcessingACM Computing Surveys (CSUR), 2021 Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 753 4,777 0 28 Jul 2021
Goal-Oriented Script ConstructionInternational Conference on Natural Language Generation (INLG), 2021 Qing Lyu Li Zhang Chris Callison-Burch 173 35 0 28 Jul 2021
One Question Answering Model for Many Languages with Cross-lingual Dense Passage RetrievalNeural Information Processing Systems (NeurIPS), 2021 Akari Asai Xinyan Velocity Yu Jungo Kasai Hannaneh Hajishirzi RALM LRM 223 79 0 26 Jul 2021
Modelling Latent Translations for Cross-Lingual Transfer Edoardo Ponti Julia Kreutzer Ivan Vulić Siva Reddy 230 21 0 23 Jul 2021
MarIA: Spanish Language Models Asier Gutiérrez-Fandiño Jordi Armengol-Estapé Marc Pàmies Joan Llop-Palao Joaquín Silveira-Ocampo C. Carrino Aitor Gonzalez-Agirre Carme Armentano-Oller Carlos Rodríguez-Penagos Marta Villegas VLM 277 133 0 15 Jul 2021
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 624 758 0 14 Jul 2021
DaCy: A Unified Framework for Danish NLP Kenneth Enevoldsen Lasse Hansen Kristoffer Nielbo 151 13 0 12 Jul 2021
A Primer on Pretrained Multilingual Language Models Sumanth Doddapaneni Gowtham Ramesh Mitesh M. Khapra Anoop Kunchukuttan Pratyush Kumar LRM 197 83 0 01 Jul 2021
GlyphCRM: Bidirectional Encoder Representation for Chinese Character with its Glyph Yunxin Li Yu Zhao Baotian Hu Qingcai Chen Yang Xiang Xiaolong Wang Yuxin Ding Lin Ma 108 8 0 01 Jul 2021
Revisiting the Primacy of English in Zero-shot Cross-lingual Transfer Iulia Turc Kenton Lee Jacob Eisenstein Ming-Wei Chang Kristina Toutanova 131 60 0 30 Jun 2021
XLM-E: Cross-lingual Language Model Pre-training via ELECTRAAnnual Meeting of the Association for Computational Linguistics (ACL), 2021 Zewen Chi Shaohan Huang Li Dong Shuming Ma Bo Zheng ... Payal Bajaj Xia Song Xian-Ling Mao Heyan Huang Furu Wei 207 128 0 30 Jun 2021
Overview of BioASQ 2021: The ninth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question AnsweringConference and Labs of the Evaluation Forum (CLEF), 2021 A. Nentidis K. Bougiatiotis Carlos Rodríguez-Penagos Anastasia Krithara Marta Villegas Martin Krallinger George Giannakopoulos 121 51 0 28 Jun 2021
XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 LanguagesFindings (Findings), 2021 Tahmid Hasan Abhik Bhattacharjee Md. Saiful Islam Kazi Samin Mubasshir Yuan-Fang Li Yong-Bin Kang M. Rahman Rifat Shahriyar 286 442 0 25 Jun 2021
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders Shuming Ma Li Dong Shaohan Huang Dongdong Zhang Alexandre Muzio Saksham Singhal Hany Awadalla Xia Song Furu Wei SLR AI4CE 195 88 0 25 Jun 2021
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization Yi Tay Vinh Q. Tran Sebastian Ruder Jai Gupta Hyung Won Chung Dara Bahri Zhen Qin Simon Baumgartner Cong Yu Donald Metzler 318 185 0 23 Jun 2021
CPM-2: Large-scale Cost-effective Pre-trained Language ModelsAI Open (AO), 2021 Zhengyan Zhang Yuxian Gu Xu Han Shengqi Chen Chaojun Xiao ... Shiyu Huang Wentao Han Yang Liu Xiaoyan Zhu Maosong Sun MoE 306 96 0 20 Jun 2021
Transformers for Headline Selection for Russian News Clusters Pavel Voropaev Olga Sopilnyak 79 0 0 19 Jun 2021