v1v2v3 (latest)

mT5: A massively multilingual pre-trained text-to-text transformer

22 October 2020

ArXiv (abs)PDF HTML HuggingFace (4 upvotes)

Papers citing "mT5: A massively multilingual pre-trained text-to-text transformer"

50 / 1,560 papers shown

Title
Automatic Construction of Evaluation Suites for Natural Language Generation Datasets Simon Mille Kaustubh D. Dhole Saad Mahamood Laura Perez-Beltrachini Varun Gangal Mihir Kale Emiel van Miltenburg Sebastian Gehrmann ELM 159 25 0 16 Jun 2021
Specializing Multilingual Language Models: An Empirical Study Ethan C. Chau Noah A. Smith 371 29 0 16 Jun 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 788 3,355 0 15 Jun 2021
Challenges and Considerations with Code-Mixed NLP for Multilingual Societies Vivek Srivastava M. Singh 203 6 0 15 Jun 2021
Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense ReasoningAnnual Meeting of the Association for Computational Linguistics (ACL), 2021 Bill Yuchen Lin Seyeon Lee Xiaoyang Qiao Xiang Ren ReLM LRM 182 78 0 13 Jun 2021
Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word AlignmentAnnual Meeting of the Association for Computational Linguistics (ACL), 2021 Zewen Chi Li Dong Bo Zheng Shaohan Huang Xian-Ling Mao Heyan Huang Furu Wei 193 79 0 11 Jun 2021
Learning Multilingual Representation for Natural Language Understanding with Enhanced Cross-Lingual Supervision Yinpeng Guo Liangyou Li Xin Jiang Qun Liu 144 0 0 09 Jun 2021
VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation Linjie Li Jie Lei Zhe Gan Licheng Yu Yen-Chun Chen ... Tamara L. Berg Joey Tianyi Zhou Jingjing Liu Lijuan Wang Zicheng Liu VLM 227 117 0 08 Jun 2021
Investigating Transfer Learning in Multilingual Pre-trained Language Models through Chinese Natural Language InferenceFindings (Findings), 2021 Hai Hu He Zhou Zuoyu Tian Yiwen Zhang Yina Ma Yanting Li Yixin Nie Kyle Richardson 155 12 0 07 Jun 2021
A Simple Recipe for Multilingual Grammatical Error CorrectionAnnual Meeting of the Association for Computational Linguistics (ACL), 2021 S. Rothe Jonathan Mallinson Eric Malmi Sebastian Krause Aliaksei Severyn 192 185 0 07 Jun 2021
nmT5 -- Is parallel data still relevant for pre-training massively multilingual language models?Annual Meeting of the Association for Computational Linguistics (ACL), 2021 Mihir Kale Aditya Siddhant Noah Constant Melvin Johnson Rami Al-Rfou Linting Xue LRM 99 26 0 03 Jun 2021
A Dataset and Baselines for Multilingual Reply SuggestionAnnual Meeting of the Association for Computational Linguistics (ACL), 2021 Mozhi Zhang Wei Wang Budhaditya Deb Guoqing Zheng Milad Shokouhi Ahmed Hassan Awadallah LRM 128 10 0 03 Jun 2021
NewsEmbed: Modeling News through Pre-trained Document RepresentationsKnowledge Discovery and Data Mining (KDD), 2021 Jialu Liu Tianqi Liu Cong Yu VLM 303 12 0 01 Jun 2021
Towards More Equitable Question Answering Systems: How Much More Data Do You Need?Annual Meeting of the Association for Computational Linguistics (ACL), 2021 Arnab Debnath Navid Rajabi F. Alam Antonios Anastasopoulos 142 11 0 28 May 2021
ByT5: Towards a token-free future with pre-trained byte-to-byte modelsTransactions of the Association for Computational Linguistics (TACL), 2021 Linting Xue Aditya Barua Noah Constant Rami Al-Rfou Sharan Narang Mihir Kale Adam Roberts Colin Raffel 383 575 0 28 May 2021
Investigating Code-Mixed Modern Standard Arabic-Egyptian to English Machine Translation El Moatez Billah Nagoudi AbdelRahim Elmadany Muhammad Abdul-Mageed MoE 161 15 0 28 May 2021
Exploring Text-to-Text Transformers for English to Hinglish Machine Translation with Synthetic Code-Mixing Ganesh Jawahar El Moatez Billah Nagoudi Muhammad Abdul-Mageed L. Lakshmanan 213 31 0 18 May 2021
A cost-benefit analysis of cross-lingual transfer methods G. Rosa L. Bonifacio Leandro Rodrigues de Souza R. Lotufo Rodrigo Nogueira 201 14 0 14 May 2021
Continual Mixed-Language Pre-Training for Extremely Low-Resource Neural Machine TranslationFindings (Findings), 2021 Zihan Liu Genta Indra Winata Pascale Fung VLM CLL 204 62 0 09 May 2021
Which transformer architecture fits my data? A vocabulary bottleneck in self-attentionInternational Conference on Machine Learning (ICML), 2021 Noam Wies Yoav Levine Daniel Jannai Amnon Shashua 188 22 0 09 May 2021
XeroAlign: Zero-Shot Cross-lingual Transformer AlignmentFindings (Findings), 2021 Milan Gritta Ignacio Iacobacci 173 23 0 06 May 2021
Rethinking Search: Making Domain Experts out of DilettantesSIGIR Forum (SIGIR Forum), 2021 Donald Metzler Yi Tay Dara Bahri Marc Najork LRM 232 54 0 05 May 2021
Russian News Clustering and Headline Selection Shared Task I. Gusev I. Smurov 145 7 0 03 May 2021
Morph Call: Probing Morphosyntactic Content of Multilingual Transformers Vladislav Mikhailov O. Serikov Ekaterina Artemova 204 10 0 26 Apr 2021
XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and BeyondInternational Conference on Language Resources and Evaluation (LREC), 2021 Francesco Barbieri Luis Espinosa Anke Jose Camacho-Collados 361 290 0 25 Apr 2021
Generating abstractive summaries of Lithuanian news articles using a transformer modelInternational Conference on Information and Software Technologies (ICIST), 2021 Lukas Stankevicius M. Lukoševičius 115 3 0 23 Apr 2021
Should we Stop Training More Monolingual Models, and Simply Use Machine Translation Instead?Nordic Conference of Computational Linguistics (NoDaLiDa), 2021 T. Isbister F. Carlsson Magnus Sahlgren 153 29 0 21 Apr 2021
LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding Yiheng Xu Tengchao Lv Lei Cui Guoxin Wang Yijuan Lu D. Florêncio Cha Zhang Furu Wei MLLM VLM 241 161 0 18 Apr 2021
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled CorpusConference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Jesse Dodge Maarten Sap Ana Marasović William Agnew Gabriel Ilharco Dirk Groeneveld Margaret Mitchell Matt Gardner AILaw 285 552 0 18 Apr 2021
Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained EncodersConference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Guanhua Chen Shuming Ma Yun-Nung Chen Li Dong Dongdong Zhang Jianxiong Pan Wenping Wang Furu Wei 126 41 0 18 Apr 2021
MT6: Multilingual Pretrained Text-to-Text Transformer with Translation PairsConference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Zewen Chi Li Dong Shuming Ma Shaohan Huang Xian-Ling Mao Heyan Huang Furu Wei LRM 183 76 0 18 Apr 2021
Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue SystemsJournal of Artificial Intelligence Research (JAIR), 2021 E. Razumovskaia Goran Glavaš Olga Majewska Edoardo Ponti Anna Korhonen Ivan Vulić 445 37 0 17 Apr 2021
XTREME-R: Towards More Challenging and Nuanced Multilingual EvaluationConference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Sebastian Ruder Noah Constant Jan A. Botha Aditya Siddhant Orhan Firat ... Pengfei Liu Junjie Hu Dan Garrette Graham Neubig Melvin Johnson ELM AAML LRM 248 209 0 15 Apr 2021
Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic LanguagesTransactions of the Association for Computational Linguistics (TACL), 2021 Gowtham Ramesh Sumanth Doddapaneni Aravinth Bheemaraj Mayank Jobanputra AK Raghavan ... K. Deepak Vivek Raghavan Anoop Kunchukuttan Pratyush Kumar Mitesh Khapra LRM 295 263 0 12 Apr 2021
CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computing Ahmed Elnaggar Wei Ding Llion Jones Tom Gibbs Tamas B. Fehér Christoph Angerer Silvia Severini Florian Matthes B. Rost 211 82 0 06 Apr 2021
Quality at a Glance: An Audit of Web-Crawled Multilingual DatasetsTransactions of the Association for Computational Linguistics (TACL), 2021 Julia Kreutzer Isaac Caswell Lisa Wang Ahsan Wahab D. Esch ... Duygu Ataman Orevaoghene Ahia Oghenefego Ahia Sweta Agrawal Mofetoluwa Adeyemi 347 308 0 22 Mar 2021
#PraCegoVer: A Large Dataset for Image Captioning in PortugueseInternational Conference on Data Technologies and Applications (DATA), 2021 G. O. D. Santos Esther Luna Colombini Sandra Avila 194 12 0 21 Mar 2021
The Effect of Domain and Diacritics in Yorùbá-English Neural Machine TranslationMachine Translation Summit (MT Summit), 2021 David Ifeoluwa Adelani Dana Ruiter Jesujoba Oluwadara Alabi Damilola Adebonojo Adesina Ayeni Mofetoluwa Adeyemi Ayodele Awokoya C. España-Bonet 192 45 0 15 Mar 2021
CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language RepresentationTransactions of the Association for Computational Linguistics (TACL), 2021 J. Clark Dan Garrette Iulia Turc John Wieting 442 262 0 11 Mar 2021
WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine LearningAnnual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2021 Krishna Srinivasan K. Raman Jiecao Chen Michael Bendersky Marc Najork VLM 458 386 0 02 Mar 2021
Do Transformer Modifications Transfer Across Implementations and Applications?Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Sharan Narang Hyung Won Chung Yi Tay W. Fedus Thibault Févry ... Wei Li Nan Ding Jake Marcus Adam Roberts Colin Raffel 198 134 0 23 Feb 2021
The GEM Benchmark: Natural Language Generation, its Evaluation and MetricsIEEE Games Entertainment Media Conference (IEEE GEM), 2021 Sebastian Gehrmann Tosin Adewumi Karmanya Aggarwal Pawan Sasanka Ammanamanchi Aremu Anuoluwapo ... Nishant Subramani Wei Xu Diyi Yang Akhila Yerukola Jiawei Zhou VLM 706 300 0 02 Feb 2021
Distilling Large Language Models into Tiny and Effective Students using pQRNN P. Kaliamoorthi Aditya Siddhant Edward Li Melvin Johnson MQ 127 18 0 21 Jan 2021
What Makes Good In-Context Examples for GPT- $3$ ?Workshop on Knowledge Extraction and Integration for Deep Learning Architectures; Deep Learning Inside Out (DEELIO), 2021 Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 554 1,595 0 17 Jan 2021
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient SparsityJournal of machine learning research (JMLR), 2021 W. Fedus Barret Zoph Noam M. Shazeer MoE 564 2,998 0 11 Jan 2021
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual CorporaConference on Empirical Methods in Natural Language Processing (EMNLP), 2020 Ouyang Xuan Shuohuan Wang Chao Pang Yu Sun Hao Tian Hua Wu Haifeng Wang 416 112 0 31 Dec 2020
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL), 2020 Phillip Rust Jonas Pfeiffer Ivan Vulić Sebastian Ruder Iryna Gurevych 410 307 0 31 Dec 2020
XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders Shuming Ma Jian Yang Haoyang Huang Zewen Chi Li Dong ... Akiko Eriguchi Saksham Singhal Xia Song Arul Menezes Furu Wei LRM 138 36 0 31 Dec 2020
ARBERT & MARBERT: Deep Bidirectional Transformers for ArabicAnnual Meeting of the Association for Computational Linguistics (ACL), 2020 Muhammad Abdul-Mageed AbdelRahim Elmadany El Moatez Billah Nagoudi VLM 415 539 0 27 Dec 2020
Leveraging ParsBERT and Pretrained mT5 for Persian Abstractive Text SummarizationInternational Computer Society of Iran Computer Conference (CSIC), 2020 Mehrdad Farahani Mohammad Gharachorloo Mohammad Manthouri 107 32 0 21 Dec 2020

All Papers

mT5: A massively multilingual pre-trained text-to-text transformer

Papers citing "mT5: A massively multilingual pre-trained text-to-text transformer"