MASS: Masked Sequence to Sequence Pre-training for Language Generation

7 May 2019

Xu Tan

Papers citing "MASS: Masked Sequence to Sequence Pre-training for Language Generation"

50 / 196 papers shown

Title
A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectives Shuyu Li Shulei Ji Zihao Wang Songruoyao Wu Jiaxing Yu Kaipeng Zhang MGen VGen 70 1 0 01 Apr 2025
Towards Making Flowchart Images Machine Interpretable S. Kamath S Prajwal Gatti Yogesh Kumar Vikash Yadav Anand Mishra 53 5 0 29 Jan 2025
Effective Self-Mining of In-Context Examples for Unsupervised Machine Translation with LLMs Abdellah El Mekki Muhammad Abdul-Mageed LRM 31 0 0 14 Oct 2024
Salience-Based Adaptive Masking: Revisiting Token Dynamics for Enhanced Pre-training Hyesong Choi Hyejin Park Kwang Moo Yi Sungmin Cha Dongbo Min 39 9 0 12 Apr 2024
Returning to the Start: Generating Narratives with Related Endpoints Anneliese Brei Chao Zhao Snigdha Chaturvedi HAI 23 1 0 31 Mar 2024
Extending Multilingual Speech Synthesis to 100+ Languages without Transcribed Data Takaaki Saeki Gary Wang Nobuyuki Morioka Isaac Elias Kyle Kastner ... Andrew Rosenberg Bhuvana Ramabhadran Heiga Zen Francoise Beaufays Hadar Shemtov 38 13 0 29 Feb 2024
Large Language Models: A Survey Shervin Minaee Tomáš Mikolov Narjes Nikzad M. Asgari-Chenaghlu R. Socher Xavier Amatriain Jianfeng Gao ALM LM&MA ELM 131 369 0 09 Feb 2024
How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation? Rheeya Uppaal Yixuan Li Junjie Hu 37 4 0 31 Jan 2024
Boosting Summarization with Normalizing Flows and Aggressive Training Yu Yang Xiaotong Shen AI4CE TPM 19 0 0 01 Nov 2023
Neural Summarization of Electronic Health Records Koyena Pal Seyed Ali Bahrainian Laura Y. Mercurio Carsten Eickhoff 25 3 0 24 May 2023
When Does Monolingual Data Help Multilingual Translation: The Role of Domain and Model Scale Christos Baziotis Biao Zhang Alexandra Birch Barry Haddow 30 2 0 23 May 2023
SikuGPT: A Generative Pre-trained Model for Intelligent Information Processing of Ancient Texts from the Perspective of Digital Humanities Chang Liu Dongbo Wang Zhixiao Zhao Die Hu Mengcheng Wu ... Si Shen Bin Li Jiangfeng Liu Hai Zhang Lianzheng Zhao 19 9 0 16 Apr 2023
Bilex Rx: Lexical Data Augmentation for Massively Multilingual Machine Translation Alex Jones Isaac Caswell Ishan Saxena Orhan Firat 23 8 0 27 Mar 2023
JaCoText: A Pretrained Model for Java Code-Text Generation Jessica Nayeli López Espejel Mahaman Sanoussi Yahaya Alassan Walid Dahhane E. Ettifouri 29 3 0 22 Mar 2023
A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT Yihan Cao Siyu Li Yixin Liu Zhiling Yan Yutong Dai Philip S. Yu Lichao Sun 29 506 0 07 Mar 2023
On the Provable Advantage of Unsupervised Pretraining Jiawei Ge Shange Tang Jianqing Fan Chi Jin SSL 33 16 0 02 Mar 2023
The unreasonable effectiveness of few-shot learning for machine translation Xavier Garcia Yamini Bansal Colin Cherry George F. Foster M. Krikun Fan Feng Melvin Johnson Orhan Firat 29 102 0 02 Feb 2023
A Survey on Efficient Training of Transformers Bohan Zhuang Jing Liu Zizheng Pan Haoyu He Yuetian Weng Chunhua Shen 31 47 0 02 Feb 2023
Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise Zheng-Wen Lin Yeyun Gong Yelong Shen Tong Wu Zhihao Fan Chen Lin Nan Duan Weizhu Chen AI4CE DiffM VLM 35 60 0 22 Dec 2022
GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator Jian Yang Shuming Ma Li Dong Shaohan Huang Haoyang Huang Yuwei Yin Dongdong Zhang Liqun Yang Furu Wei Zhoujun Li SyDa AI4CE 32 25 0 20 Dec 2022
Open-world Story Generation with Structured Knowledge Enhancement: A Comprehensive Survey Yuxin Wang Jieru Lin Zhiwei Yu Wei Hu Börje F. Karlsson 20 17 0 09 Dec 2022
CUNI Systems for the WMT22 Czech-Ukrainian Translation Task Martin Popel Jindrich Libovický Jindřich Helcl 19 4 0 01 Dec 2022
Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention Zineng Tang Jaemin Cho Jie Lei Joey Tianyi Zhou VLM 24 9 0 21 Nov 2022
AF Adapter: Continual Pretraining for Building Chinese Biomedical Language Model Yongyu Yan Kui Xue Xiaoming Shi Qi Ye Jingping Liu Tong Ruan CLL 42 1 0 21 Nov 2022
Mask More and Mask Later: Efficient Pre-training of Masked Language Models by Disentangling the [MASK] Token Baohao Liao David Thulke Sanjika Hewavitharana Hermann Ney Christof Monz 28 9 0 09 Nov 2022
Contrastive Learning enhanced Author-Style Headline Generation Hui Liu Weidong Guo Yige Chen Xiangyang Li 15 5 0 07 Nov 2022
ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generation Junyi Li Tianyi Tang Wayne Xin Zhao J. Nie Ji-Rong Wen 22 17 0 24 Oct 2022
P $^3$ LM: Probabilistically Permuted Prophet Language Modeling for Generative Pre-Training Junwei Bao Yifan Wang Jiangyong Ying Yeyun Gong Jing Zhao Youzheng Wu Xiaodong He 37 1 0 22 Oct 2022
Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods Evan Crothers Nathalie Japkowicz H. Viktor DeLMO 38 107 0 13 Oct 2022
MTet: Multi-domain Translation for English and Vietnamese C. Ngo Trieu H. Trinh Long Phan H. Tran Tai Dang Hieu Duy Nguyen Minh Le Nguyen Minh-Thang Luong VLM 29 8 0 11 Oct 2022
TVLT: Textless Vision-Language Transformer Zineng Tang Jaemin Cho Yixin Nie Joey Tianyi Zhou VLM 51 28 0 28 Sep 2022
PePe: Personalized Post-editing Model utilizing User-generated Post-edits Jihyeon Janel Lee Taehee Kim Yunwon Tae Cheonbok Park Jaegul Choo 14 0 0 21 Sep 2022
Knowledge Is Flat: A Seq2Seq Generative Framework for Various Knowledge Graph Completion Chen Chen Yufei Wang Bing Li Kwok-Yan Lam 28 31 0 15 Sep 2022
Enhancing Semantic Understanding with Self-supervised Methods for Abstractive Dialogue Summarization Hyun-Yong Lee Jaewoong Yun Hyunjin Choi Seongho Joe Youngjune Gwon 18 3 0 01 Sep 2022
Interpreting Song Lyrics with an Audio-Informed Pre-trained Language Model Yixiao Zhang Junyan Jiang Gus Xia S. Dixon 25 9 0 24 Aug 2022
A Comprehensive Survey of Natural Language Generation Advances from the Perspective of Digital Deception Keenan I. Jones Enes ALTUNCU V. N. Franqueira Yi-Chia Wang Shujun Li DeLMO 34 3 0 11 Aug 2022
AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model Saleh Soltan Shankar Ananthakrishnan Jack G. M. FitzGerald Rahul Gupta Wael Hamza ... Mukund Sridhar Fabian Triefenbach Apurv Verma Gökhan Tür Premkumar Natarajan 51 82 0 02 Aug 2022
Transformers are Adaptable Task Planners Vidhi Jain Yixin Lin Eric Undersander Yonatan Bisk Akshara Rai 25 24 0 06 Jul 2022
E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao 46 27 0 30 May 2022
Simple Recurrence Improves Masked Language Models Tao Lei Ran Tian Jasmijn Bastings Ankur P. Parikh 77 4 0 23 May 2022
HyperTree Proof Search for Neural Theorem Proving Guillaume Lample Marie-Anne Lachaux Thibaut Lavril Xavier Martinet Amaury Hayat Gabriel Ebner Aurelien Rodriguez Timothée Lacroix AIMat 23 133 0 23 May 2022
When does Parameter-Efficient Transfer Learning Work for Machine Translation? A. Ustun Asa Cooper Stickland 37 7 0 23 May 2022
Building Machine Translation Systems for the Next Thousand Languages Ankur Bapna Isaac Caswell Julia Kreutzer Orhan Firat D. Esch ... Apurva Shah Yanping Huang Z. Chen Yonghui Wu Macduff Hughes 56 98 0 09 May 2022
Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation Xiangpeng Wei Heng Yu Yue Hu Rongxiang Weng Weihua Luo Jun Xie Rong Jin CLL 17 24 0 14 Apr 2022
Spatial-Temporal Parallel Transformer for Arm-Hand Dynamic Estimation Shuying Liu Wen-Xuan Wu Jiaxian Wu Yue-Hsun Lin 32 11 0 30 Mar 2022
A Well-Composed Text is Half Done! Composition Sampling for Diverse Conditional Generation Shashi Narayan Gonccalo Simoes Yao-Min Zhao Joshua Maynez Dipanjan Das Michael Collins Mirella Lapata 29 30 0 28 Mar 2022
ANNA: Enhanced Language Representation for Question Answering Changwook Jun Hansol Jang Myoseop Sim Hyun Kim Jooyoung Choi Kyungkoo Min Kyunghoon Bae 29 6 0 28 Mar 2022
EAG: Extract and Generate Multi-way Aligned Corpus for Complete Multi-lingual Neural Machine Translation Yulin Xu Zhen Yang Fandong Meng JieZhou 25 3 0 04 Mar 2022
Attend, Memorize and Generate: Towards Faithful Table-to-Text Generation in Few Shots Wenting Zhao Ye Liu Yao Wan Philip S. Yu 21 11 0 01 Mar 2022
Using natural language prompts for machine translation Xavier Garcia Orhan Firat AI4CE 25 30 0 23 Feb 2022