ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training

13 January 2020

Dayiheng Liu

Papers citing "ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training"

50 / 57 papers shown

Title
Looking beyond the next token Abitha Thankaraj Yiding Jiang J. Zico Kolter Yonatan Bisk LRM 51 1 0 15 Apr 2025
VocalNet: Speech LLM with Multi-Token Prediction for Faster and High-Quality Generation Yuhao Wang Heyang Liu Ziyang Cheng Ronghua Wu Qunshan Gu Yanfeng Wang Yu Wang 87 0 0 05 Apr 2025
SuperBPE: Space Travel for Language Models Alisa Liu J. Hayase Valentin Hofmann Sewoong Oh Noah A. Smith Yejin Choi 43 1 0 17 Mar 2025
CPRM: A LLM-based Continual Pre-training Framework for Relevance Modeling in Commercial Search Kaixin Wu Yixin Ji Z. Chen Qiang Wang Cunxiang Wang ... Jia Xu Zhongyi Liu Jinjie Gu Yuan Zhou Linjian Mo KELM CLL 92 0 0 02 Dec 2024
Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning Jiacheng Ye Jiahui Gao Shansan Gong Lin Zheng Xin Jiang Z. Li Lingpeng Kong DiffM LRM 42 15 0 18 Oct 2024
Machine Learning Predictors for Min-Entropy Estimation Javier Blanco-Romero Vicente Lorenzo Florina Almenáres Mendoza Daniel Díaz Sánchez 26 0 0 28 Jun 2024
Learning Regularities from Data using Spiking Functions: A Theory Canlin Zhang Xiuwen Liu 27 0 0 19 May 2024
CLIPSyntel: CLIP and LLM Synergy for Multimodal Question Summarization in Healthcare Akash Ghosh Arkadeep Acharya Raghav Jain Sriparna Saha Aman Chadha Setu Sinha 27 29 0 16 Dec 2023
Boosting Summarization with Normalizing Flows and Aggressive Training Yu Yang Xiaotong Shen AI4CE TPM 17 0 0 01 Nov 2023
Instruction Position Matters in Sequence Generation with Large Language Models Yanjun Liu Xianfeng Zeng Fandong Meng Jie Zhou LRM 35 8 0 23 Aug 2023
Learning Summary-Worthy Visual Representation for Abstractive Summarization in Video Zenan Xu Xiaojun Meng Yasheng Wang Qinliang Su Zexuan Qiu Xin Jiang Qun Liu 14 3 0 08 May 2023
Entity-Based Evaluation of Political Bias in Automatic Summarization Karen Zhou Chenhao Tan 24 1 0 03 May 2023
Bridging the Language Gap: Knowledge Injected Multilingual Question Answering Zhichao Duan Xiuxing Li Zhengyan Zhang Zhenyu Li Ning Liu Jianyong Wang 19 8 0 06 Apr 2023
Sequence-aware item recommendations for multiply repeated user-item interactions Juan Pablo Equihua Maged Ali Henrik Nordmark B. Lausen 6 0 0 02 Apr 2023
CISum: Learning Cross-modality Interaction to Enhance Multimodal Semantic Coverage for Multimodal Summarization Litian Zhang Xiaoming Zhang Ziming Guo Zhipeng Liu 17 7 0 20 Feb 2023
Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise Zheng-Wen Lin Yeyun Gong Yelong Shen Tong Wu Zhihao Fan Chen Lin Nan Duan Weizhu Chen AI4CE DiffM VLM 22 60 0 22 Dec 2022
Combining State-of-the-Art Models with Maximal Marginal Relevance for Few-Shot and Zero-Shot Multi-Document Summarization David Adams Gandharv Suri Yllias Chali VLM 14 3 0 19 Nov 2022
YORO -- Lightweight End to End Visual Grounding Chih-Hui Ho Srikar Appalaraju Bhavan A. Jasani R. Manmatha Nuno Vasconcelos ObjD 21 21 0 15 Nov 2022
Universal Evasion Attacks on Summarization Scoring Wenchuan Mu Kwan Hui Lim AAML 22 1 0 25 Oct 2022
ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generation Junyi Li Tianyi Tang Wayne Xin Zhao J. Nie Ji-Rong Wen 20 17 0 24 Oct 2022
P $^3$ LM: Probabilistically Permuted Prophet Language Modeling for Generative Pre-Training Junwei Bao Yifan Wang Jiangyong Ying Yeyun Gong Jing Zhao Youzheng Wu Xiaodong He 32 1 0 22 Oct 2022
Generative Language Models for Paragraph-Level Question Generation Asahi Ushio Fernando Alva-Manchego Jose Camacho-Collados ELM 11 45 0 08 Oct 2022
PROD: Progressive Distillation for Dense Retrieval Zhenghao Lin Yeyun Gong Xiao Liu Hang Zhang Chen Lin ... Jian Jiao Jing Lu Daxin Jiang Rangan Majumder Nan Duan 17 27 0 27 Sep 2022
PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecasting Hao Xue Flora D.Salim AI4TS 16 136 0 20 Sep 2022
CoHS-CQG: Context and History Selection for Conversational Question Generation Do Xuan Long Bowei Zou Liangming Pan Nancy F. Chen Shafiq R. Joty A. Aw SLR 29 10 0 14 Sep 2022
E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao 29 27 0 30 May 2022
Computational Storytelling and Emotions: A Survey Yusuke Mori Hiroaki Yamane Yusuke Mukuta Tatsuya Harada 22 2 0 23 May 2022
RankGen: Improving Text Generation with Large Ranking Models Kalpesh Krishna Yapei Chang John Wieting Mohit Iyyer AIMat 16 68 0 19 May 2022
Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages Felix Wu Kwangyoun Kim Shinji Watanabe Kyu Jeong Han Ryan T. McDonald Kilian Q. Weinberger Yoav Artzi SyDa 34 37 0 02 May 2022
Enhance Incomplete Utterance Restoration by Joint Learning Token Extraction and Text Generation Shumpei Inoue Tsun-Jui Liu Nguyen Hong Son Minh Le Nguyen 28 17 0 08 Apr 2022
A Well-Composed Text is Half Done! Composition Sampling for Diverse Conditional Generation Shashi Narayan Gonccalo Simoes Yao-Min Zhao Joshua Maynez Dipanjan Das Michael Collins Mirella Lapata 10 30 0 28 Mar 2022
A Feasibility Study of Answer-Agnostic Question Generation for Education Liam Dugan E. Miltsakaki Shriyash Upadhyay Etan Ginsberg Hannah Gonzalez Dayheon Choi Chuning Yuan Chris Callison-Burch 17 12 0 16 Mar 2022
NoisyTune: A Little Noise Can Help You Finetune Pretrained Language Models Better Chuhan Wu Fangzhao Wu Tao Qi Yongfeng Huang Xing Xie 17 58 0 24 Feb 2022
Multi-Narrative Semantic Overlap Task: Evaluation and Benchmark Naman Bansal Mousumi Akter Shubhra (Santu) Karmaker 29 0 0 14 Jan 2022
A Survey of Natural Language Generation Chenhe Dong Yinghui Li Haifan Gong M. Chen Junxin Li Ying Shen Min Yang 3DV 14 43 0 22 Dec 2021
BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese Nguyen Luong Tran Duong Minh Le Dat Quoc Nguyen 12 51 0 20 Sep 2021
Investigating Crowdsourcing Protocols for Evaluating the Factual Consistency of Summaries Xiangru Tang Alexander R. Fabbri Haoran Li Ziming Mao Griffin Adams Borui Wang Asli Celikyilmaz Yashar Mehdad Dragomir R. Radev HILM 11 19 0 19 Sep 2021
Generating Self-Contained and Summary-Centric Question Answer Pairs via Differentiable Reward Imitation Learning Li Zhou Kevin Small Yong Zhang Sandeep Atluri 32 2 0 10 Sep 2021
Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarization Bharath Chintagunta Namit Katariya X. Amatriain Anitha Kannan LM&MA MedIm 117 147 0 09 Sep 2021
Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization Tiezheng Yu Wenliang Dai Zihan Liu Pascale Fung 24 72 0 06 Sep 2021
Scheduled Sampling Based on Decoding Steps for Neural Machine Translation Yijin Liu Fandong Meng Yufeng Chen Jinan Xu Jie Zhou 18 16 0 30 Aug 2021
Generating Answer Candidates for Quizzes and Answer-Aware Question Generators Kristiyan Vachev Momchil Hardalov Georgi Karadzhov Georgi Georgiev Ivan Koychev Preslav Nakov AI4Ed 19 5 0 29 Aug 2021
Semantic-Based Self-Critical Training For Question Generation Loïc Kwate Dassi Kwate Dassi 15 0 0 26 Aug 2021
Enhanced Seq2Seq Autoencoder via Contrastive Learning for Abstractive Text Summarization Chujie Zheng Kunpeng Zhang Harry J. Wang Ling Fan Zhe Wang 14 6 0 26 Aug 2021
ComSum: Commit Messages Summarization and Meaning Preservation Leshem Choshen Idan Amit 17 4 0 23 Aug 2021
Reinforcement Learning for Abstractive Question Summarization with Question-aware Semantic Rewards S. Yadav D. Gupta Asma Ben Abacha Dina Demner-Fushman OffRL 8 33 0 01 Jul 2021
XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages Tahmid Hasan Abhik Bhattacharjee Md. Saiful Islam Kazi Samin Mubasshir Yuan-Fang Li Yong-Bin Kang M. Rahman Rifat Shahriyar 15 340 0 25 Jun 2021
How well do you know your summarization datasets? Priyam Tejaswin Dhruv Naik Peng Liu 10 26 0 21 Jun 2021
Straight to the Gradient: Learning to Use Novel Tokens for Neural Text Generation Xiang Lin Simeng Han Shafiq R. Joty 10 24 0 14 Jun 2021
Poolingformer: Long Document Modeling with Pooling Attention Hang Zhang Yeyun Gong Yelong Shen Weisheng Li Jiancheng Lv Nan Duan Weizhu Chen 29 98 0 10 May 2021