Understanding Knowledge Distillation in Non-autoregressive Machine Translation

7 November 2019

Graham Neubig

Papers citing "Understanding Knowledge Distillation in Non-autoregressive Machine Translation"

42 / 42 papers shown

Title
FourierNAT: A Fourier-Mixing-Based Non-Autoregressive Transformer for Parallel Sequence Generation Andrew Kiruluta Eric Lundy Andreas Lemos AI4TS 39 0 0 04 Mar 2025
Decoupled Sequence and Structure Generation for Realistic Antibody Design Nayoung Kim Minsu Kim Sungsoo Ahn Jinkyoo Park 47 0 0 20 Jan 2025
Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison Tsz Kin Lam Marco Gaido Sara Papi L. Bentivogli Barry Haddow 31 0 0 04 Jan 2025
Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models Aviv Bick Kevin Y. Li Eric P. Xing J. Zico Kolter Albert Gu Mamba 50 24 0 19 Aug 2024
CTC-based Non-autoregressive Textless Speech-to-Speech Translation Qingkai Fang Zhengrui Ma Yan Zhou Min Zhang Yang Feng 52 0 0 11 Jun 2024
What Have We Achieved on Non-autoregressive Translation? Yafu Li Huajian Zhang Jianhao Yan Yongjing Yin Yue Zhang 31 1 0 21 May 2024
Sentence-Level or Token-Level? A Comprehensive Study on Knowledge Distillation Jingxuan Wei Linzhuang Sun Yichong Leng Xu Tan Bihui Yu Ruifeng Guo 43 3 0 23 Apr 2024
Non-autoregressive Sequence-to-Sequence Vision-Language Models Kunyu Shi Qi Dong Luis Goncalves Zhuowen Tu Stefano Soatto VLM 37 3 0 04 Mar 2024
Domain Adaptation of Multilingual Semantic Search -- Literature Review Anna Bringmann Anastasia Zhukova VLM 30 0 0 05 Feb 2024
Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning Jiasheng Ye Zaixiang Zheng Yu Bao Lihua Qian Quanquan Gu DiffM 54 14 0 23 Aug 2023
Improving Image Captioning Descriptiveness by Ranking and LLM-based Fusion Simone Bianco Luigi Celona Marco Donzella Paolo Napoletano 31 18 0 20 Jun 2023
Online Distillation for Pseudo-Relevance Feedback Sean MacAvaney Xi Wang 16 2 0 16 Jun 2023
Revisiting Non-Autoregressive Translation at Scale Zhihao Wang Longyue Wang Jinsong Su Junfeng Yao Zhaopeng Tu 22 3 0 25 May 2023
Multi-Granularity Optimization for Non-Autoregressive Translation Yafu Li Leyang Cui Yongjing Yin Yue Zhang 27 7 0 20 Oct 2022
A baseline revisited: Pushing the limits of multi-segment models for context-aware translation Suvodeep Majumde Stanislas Lauly Maria Nadejde Marcello Federico Georgiana Dinu 30 13 0 19 Oct 2022
Model Criticism for Long-Form Text Generation Yuntian Deng Volodymyr Kuleshov Alexander M. Rush 33 19 0 16 Oct 2022
CTC Alignments Improve Autoregressive Translation Brian Yan Siddharth Dalmia Yosuke Higuchi Graham Neubig Florian Metze A. Black Shinji Watanabe 44 33 0 11 Oct 2022
Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translation Chenze Shao Zhengrui Ma Yang Feng 34 14 0 11 Oct 2022
PROD: Progressive Distillation for Dense Retrieval Zhenghao Lin Yeyun Gong Xiao Liu Hang Zhang Chen Lin ... Jian Jiao Jing Lu Daxin Jiang Rangan Majumder Nan Duan 45 27 0 27 Sep 2022
Non-Autoregressive Machine Translation: It's Not as Fast as it Seems Jindvrich Helcl Barry Haddow Alexandra Birch 19 19 0 04 May 2022
$$\textit{latent}$-GLAT: Glancing at Latent Variables for Parallel Text Generation$ $\textit{latent}$ -GLAT: Glancing at Latent Variables for Parallel Text Generation Yu Bao Hao Zhou Shujian Huang Dongqi Wang Lihua Qian Xinyu Dai Jiajun Chen Lei Li 23 38 0 05 Apr 2022
PreTR: Spatio-Temporal Non-Autoregressive Trajectory Prediction Transformer Lina Achaji Thierno Barry Thibault Fouqueray Julien Moreau François Aioun François Charpillet 16 15 0 17 Mar 2022
Can Multilinguality benefit Non-autoregressive Machine Translation? Sweta Agrawal Julia Kreutzer Colin Cherry AI4CE 27 1 0 16 Dec 2021
Integrated Training for Sequence-to-Sequence Models Using Non-Autoregressive Transformer E. Tokarchuk Jan Rosendahl Weiyue Wang Pavel Petrushkov Tomer Lancewicki Shahram Khadivi Hermann Ney 25 2 0 27 Sep 2021
Partial to Whole Knowledge Distillation: Progressive Distilling Decomposed Knowledge Boosts Student Better Xuanyang Zhang X. Zhang Jian-jun Sun 23 1 0 26 Sep 2021
AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translate Jongyoon Song Sungwon Kim Sungroh Yoon 66 37 0 14 Sep 2021
MvSR-NAT: Multi-view Subset Regularization for Non-Autoregressive Machine Translation Pan Xie Zexian Li Xiaohui Hu 26 11 0 19 Aug 2021
The USYD-JD Speech Translation System for IWSLT 2021 Liang Ding Di Wu Dacheng Tao 29 16 0 24 Jul 2021
Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting Wangchunshu Zhou Tao Ge Canwen Xu Ke Xu Furu Wei LRM 16 15 0 02 Jan 2021
Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade Jiatao Gu X. Kong 20 135 0 31 Dec 2020
Neural Machine Translation: A Review of Methods, Resources, and Tools Zhixing Tan Shuo Wang Zonghan Yang Gang Chen Xuancheng Huang Maosong Sun Yang Liu 3DV AI4TS 15 105 0 31 Dec 2020
Understanding and Improving Lexical Choice in Non-Autoregressive Translation Liang Ding Longyue Wang Xuebo Liu Derek F. Wong Dacheng Tao Zhaopeng Tu 96 77 0 29 Dec 2020
Infusing Sequential Information into Conditional Masked Translation Model with Self-Review Mechanism Pan Xie Zhi Cui Xiuying Chen Xiaohui Hu Jianwei Cui Bin Wang 143 9 0 19 Oct 2020
Lifelong Language Knowledge Distillation Yung-Sung Chuang Shang-Yu Su Yun-Nung Chen KELM CLL 14 49 0 05 Oct 2020
Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation Jungo Kasai Nikolaos Pappas Hao Peng James Cross Noah A. Smith 30 134 0 18 Jun 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 19 2,835 0 09 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 45 98 0 05 Jun 2020
Non-Autoregressive Machine Translation with Latent Alignments Chitwan Saharia William Chan Saurabh Saxena Mohammad Norouzi 17 157 0 16 Apr 2020
Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning Zhaojiang Lin Andrea Madotto Pascale Fung 26 155 0 08 Apr 2020
Aligned Cross Entropy for Non-Autoregressive Machine Translation Marjan Ghazvininejad Vladimir Karpukhin Luke Zettlemoyer Omer Levy 30 115 0 03 Apr 2020
LAVA NAT: A Non-Autoregressive Translation Model with Look-Around Decoding and Vocabulary Attention Xiaoya Li Yuxian Meng Arianna Yuan Fei Wu Jiwei Li 32 12 0 08 Feb 2020
Semi-Autoregressive Training Improves Mask-Predict Decoding Marjan Ghazvininejad Omer Levy Luke Zettlemoyer 25 71 0 23 Jan 2020