Sequence-Level Knowledge Distillation

25 June 2016

Papers citing "Sequence-Level Knowledge Distillation"

50 / 197 papers shown

Title
Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference Sneha Kudugunta Yanping Huang Ankur Bapna M. Krikun Dmitry Lepikhin Minh-Thang Luong Orhan Firat MoE 119 106 0 24 Sep 2021
The Volctrans GLAT System: Non-autoregressive Translation Meets WMT21 Lihua Qian Yi Zhou Zaixiang Zheng Yaoming Zhu Zehui Lin Jiangtao Feng Shanbo Cheng Lei Li Mingxuan Wang Hao Zhou 8 18 0 23 Sep 2021
TranslateLocally: Blazing-fast translation running on the local CPU Nikolay Bogoychev Jelmer Van der Linde Kenneth Heafield 14 3 0 21 Sep 2021
The NiuTrans System for WNGT 2020 Efficiency Task Chi Hu Bei Li Ye Lin Yinqiao Li Yanyang Li Chenglong Wang Tong Xiao Jingbo Zhu 15 7 0 16 Sep 2021
The NiuTrans System for the WMT21 Efficiency Task Chenglong Wang Chi Hu Yongyu Mu Zhongxiang Yan Siming Wu ... Hang Cao Bei Li Ye Lin Tong Xiao Jingbo Zhu 14 2 0 16 Sep 2021
AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translate Jongyoon Song Sungwon Kim Sungroh Yoon 66 37 0 14 Sep 2021
IndicBART: A Pre-trained Model for Indic Natural Language Generation Raj Dabre Himani Shrotriya Anoop Kunchukuttan Ratish Puduppully Mitesh M. Khapra Pratyush Kumar 36 70 0 07 Sep 2021
Survey of Low-Resource Machine Translation Barry Haddow Rachel Bawden Antonio Valerio Miceli Barone Jindvrich Helcl Alexandra Birch AIMat 29 147 0 01 Sep 2021
WeChat Neural Machine Translation Systems for WMT21 Xianfeng Zeng Yanjun Liu Ernan Li Qiu Ran Fandong Meng Peng Li Jinan Xu Jie Zhou 25 20 0 05 Aug 2021
The USYD-JD Speech Translation System for IWSLT 2021 Liang Ding Di Wu Dacheng Tao 27 16 0 24 Jul 2021
The NiuTrans End-to-End Speech Translation System for IWSLT 2021 Offline Task Chen Xu Xiaoqian Liu Xiaowen Liu Laohu Wang Canan Huang Tong Xiao Jingbo Zhu 29 5 0 06 Jul 2021
ESPnet-ST IWSLT 2021 Offline Speech Translation System H. Inaguma Shun Kiyono Nelson Enrique Yalta Soplin Pengcheng Guo Jun Suzuki Kevin Duh Shinji Watanabe 3DV 35 2 0 01 Jul 2021
The USTC-NELSLIP Systems for Simultaneous Speech Translation Task at IWSLT 2021 Dan Liu Mengge Du Xiaoxi Li Yuchen Hu Lirong Dai 11 20 0 01 Jul 2021
A Survey on Neural Speech Synthesis Xu Tan Tao Qin Frank Soong Tie-Yan Liu AI4TS 18 352 0 29 Jun 2021
Learning-based Framework for Sensor Fault-Tolerant Building HVAC Control with Model-assisted Learning Shichao Xu Yangyang Fu Yixuan Wang Zheng O’Neill Qi Zhu AI4CE 9 16 0 27 Jun 2021
Dealing with training and test segmentation mismatch: FBK@IWSLT2021 Sara Papi Marco Gaido Matteo Negri Marco Turchi 31 6 0 23 Jun 2021
Collaborative Training of Acoustic Encoders for Speech Recognition Varun K. Nagaraja Yangyang Shi Ganesh Venkatesh Ozlem Kalinli M. Seltzer Vikas Chandra 32 11 0 16 Jun 2021
Generate, Annotate, and Learn: NLP with Synthetic Text Xuanli He Islam Nassar J. Kiros Gholamreza Haffari Mohammad Norouzi 31 51 0 11 Jun 2021
Scalable Transformers for Neural Machine Translation Peng Gao Shijie Geng Yu Qiao Xiaogang Wang Jifeng Dai Hongsheng Li 23 13 0 04 Jun 2021
Diversifying Dialog Generation via Adaptive Label Smoothing Yida Wang Yinhe Zheng Yong-jia Jiang Minlie Huang 22 37 0 30 May 2021
Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey Danielle Saunders AI4CE 11 85 0 14 Apr 2021
The Curious Case of Hallucinations in Neural Machine Translation Vikas Raunak Arul Menezes Marcin Junczys-Dowmunt 11 189 0 14 Apr 2021
A Student-Teacher Architecture for Dialog Domain Adaptation under the Meta-Learning Setting Kun Qian Wei Wei Zhou Yu 13 8 0 06 Apr 2021
Compressing Visual-linguistic Model via Knowledge Distillation Zhiyuan Fang Jianfeng Wang Xiaowei Hu Lijuan Wang Yezhou Yang Zicheng Liu VLM 31 96 0 05 Apr 2021
Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translation Shuhao Gu Yang Feng Wanying Xie CLL AI4CE 25 27 0 25 Mar 2021
Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges Yoshitomo Matsubara Marco Levorato Francesco Restuccia 22 199 0 08 Mar 2021
An Efficient Transformer Decoder with Compressed Sub-layers Yanyang Li Ye Lin Tong Xiao Jingbo Zhu 22 29 0 03 Jan 2021
Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting Wangchunshu Zhou Tao Ge Canwen Xu Ke Xu Furu Wei LRM 16 15 0 02 Jan 2021
Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade Jiatao Gu X. Kong 17 135 0 31 Dec 2020
Neural Machine Translation: A Review of Methods, Resources, and Tools Zhixing Tan Shuo Wang Zonghan Yang Gang Chen Xuancheng Huang Maosong Sun Yang Liu 3DV AI4TS 13 105 0 31 Dec 2020
Understanding and Improving Lexical Choice in Non-Autoregressive Translation Liang Ding Longyue Wang Xuebo Liu Derek F. Wong Dacheng Tao Zhaopeng Tu 93 77 0 29 Dec 2020
Learning Light-Weight Translation Models from Deep Transformer Bei Li Ziyang Wang Hui Liu Quan Du Tong Xiao Chunliang Zhang Jingbo Zhu VLM 112 40 0 27 Dec 2020
Reinforced Multi-Teacher Selection for Knowledge Distillation Fei Yuan Linjun Shou J. Pei Wutao Lin Ming Gong Yan Fu Daxin Jiang 8 121 0 11 Dec 2020
Detecting Hallucinated Content in Conditional Neural Sequence Generation Chunting Zhou Graham Neubig Jiatao Gu Mona T. Diab P. Guzmán Luke Zettlemoyer Marjan Ghazvininejad HILM 36 194 0 05 Nov 2020
Pre-trained Summarization Distillation Sam Shleifer Alexander M. Rush 15 98 0 24 Oct 2020
Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor Xinyu Wang Yong-jia Jiang Zhaohui Yan Zixia Jia Nguyen Bach Tao Wang Zhongqiang Huang Fei Huang Kewei Tu 26 10 0 10 Oct 2020
Automated Concatenation of Embeddings for Structured Prediction Xinyu Wang Yong-jia Jiang Nguyen Bach Tao Wang Zhongqiang Huang Fei Huang Kewei Tu 35 172 0 10 Oct 2020
Lifelong Language Knowledge Distillation Yung-Sung Chuang Shang-Yu Su Yun-Nung Chen KELM CLL 6 49 0 05 Oct 2020
WeChat Neural Machine Translation Systems for WMT20 Fandong Meng Jianhao Yan Yijin Liu Yuan Gao Xia Zeng ... Peng Li Ming Chen Jie Zhou Sifan Liu Hao Zhou 19 21 0 01 Oct 2020
Teacher-Critical Training Strategies for Image Captioning Yiqing Huang Jiansheng Chen VLM 8 8 0 30 Sep 2020
TernaryBERT: Distillation-aware Ultra-low Bit BERT Wei Zhang Lu Hou Yichun Yin Lifeng Shang Xiao Chen Xin Jiang Qun Liu MQ 17 208 0 27 Sep 2020
Softmax Tempering for Training Neural Machine Translation Models Raj Dabre Atsushi Fujita 23 10 0 20 Sep 2020
Code-switching pre-training for neural machine translation Zhen Yang Bojie Hu Ambyera Han Shen Huang Qi Ju 19 71 0 17 Sep 2020
LRSpeech: Extremely Low-Resource Speech Synthesis and Recognition Jin Xu Xu Tan Yi Ren Tao Qin Jian Li Sheng Zhao Tie-Yan Liu VLM 16 90 0 09 Aug 2020
Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation Jungo Kasai Nikolaos Pappas Hao Peng James Cross Noah A. Smith 30 134 0 18 Jun 2020
Multi-fidelity Neural Architecture Search with Knowledge Distillation I. Trofimov Nikita Klyuchnikov Mikhail Salnikov Alexander N. Filippov Evgeny Burnaev 27 15 0 15 Jun 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 19 2,835 0 09 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 40 98 0 05 Jun 2020
End-to-End Speech-Translation with Knowledge Distillation: FBK@IWSLT2020 Marco Gaido Mattia Antonino Di Gangi Matteo Negri Marco Turchi 14 53 0 04 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 15 39,958 0 28 May 2020