Pre-training via Paraphrasing

26 June 2020

Luke Zettlemoyer

Papers citing "Pre-training via Paraphrasing"

25 / 25 papers shown

Title
Unveiling Attractor Cycles in Large Language Models: A Dynamical Systems View of Successive Paraphrasing Zhilin Wang Yafu Li Jianhao Yan Yu Cheng Yue Zhang 65 0 0 24 Feb 2025
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models Wenqi Fan Yujuan Ding Liang-bo Ning Shijie Wang Hengyun Li Dawei Yin Tat-Seng Chua Qing Li RALM 3DV 38 181 0 10 May 2024
LAMPAT: Low-Rank Adaption for Multilingual Paraphrasing Using Adversarial Training Khoi M. Le Trinh Pham Tho Quan A. Luu 11 7 0 09 Jan 2024
Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models Wenqi Jiang Marco Zeller R. Waleffe Torsten Hoefler Gustavo Alonso 47 16 0 15 Oct 2023
Towards Multi-Modal DBMSs for Seamless Querying of Texts and Tables Matthias Urban Carsten Binnig 28 5 0 26 Apr 2023
Advancing Multilingual Pre-training: TRIP Triangular Document-level Pre-training for Multilingual Language Models Hongyuan Lu Haoyang Huang Shuming Ma Dongdong Zhang W. Lam Furu Wei 19 4 0 15 Dec 2022
GNN-SL: Sequence Labeling Based on Nearest Examples via GNN Shuhe Wang Yuxian Meng Rongbin Ouyang Jiwei Li Tianwei Zhang Lingjuan Lyu Guoyin Wang 14 9 0 05 Dec 2022
BARTSmiles: Generative Masked Language Models for Molecular Representations Gayane Chilingaryan Hovhannes Tamoyan Ani Tevosyan N. Babayan L. Khondkaryan Karen Hambardzumyan Zaven Navoyan Hrant Khachatrian Armen Aghajanyan SSL 25 26 0 29 Nov 2022
Retrieval-Augmented Multimodal Language Modeling Michihiro Yasunaga Armen Aghajanyan Weijia Shi Rich James J. Leskovec Percy Liang M. Lewis Luke Zettlemoyer Wen-tau Yih RALM 11 95 0 22 Nov 2022
OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering Zhengbao Jiang Yi Mao Pengcheng He Graham Neubig Weizhu Chen LMTD 19 36 0 08 Jul 2022
An Empirical Study of Retrieval-enhanced Graph Neural Networks Dingmin Wang Shengchao Liu Hanchen Wang Bernardo Cuenca Grau Linfeng Song Jian Tang Song Le Qi Liu 9 0 0 01 Jun 2022
E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao 29 27 0 30 May 2022
Few-shot Mining of Naturally Occurring Inputs and Outputs Mandar Joshi Terra Blevins M. Lewis Daniel S. Weld Luke Zettlemoyer 22 1 0 09 May 2022
$k$ NN-NER: Named Entity Recognition with Nearest Neighbor Search Shuhe Wang Xiaoya Li Yuxian Meng Tianwei Zhang Rongbin Ouyang Jiwei Li Guoyin Wang 16 11 0 31 Mar 2022
A Survey on Retrieval-Augmented Text Generation Huayang Li Yixuan Su Deng Cai Yan Wang Lemao Liu RALM 31 197 0 02 Feb 2022
Unsupervised Dense Information Retrieval with Contrastive Learning Gautier Izacard Mathilde Caron Lucas Hosseini Sebastian Riedel Piotr Bojanowski Armand Joulin Edouard Grave RALM 17 805 0 16 Dec 2021
WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models Benjamin Minixhofer Fabian Paischer Navid Rekabsaz 22 72 0 13 Dec 2021
Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learning Seanie Lee Haebeom Lee Juho Lee S. Hwang MoMe CLL 27 16 0 06 Oct 2021
RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsing Vivek Gupta Akshat Shrivastava Adithya Sagar Armen Aghajanyan Denis Savenkov RALM 14 21 0 21 Sep 2021
Enhanced Seq2Seq Autoencoder via Contrastive Learning for Abstractive Text Summarization Chujie Zheng Kunpeng Zhang Harry J. Wang Ling Fan Zhe Wang 14 6 0 26 Aug 2021
What's in a Summary? Laying the Groundwork for Advances in Hospital-Course Summarization Griffin Adams Emily Alsentzer Mert Ketenci Jason Zucker Noémie Elhadad 33 46 0 12 Apr 2021
Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models Nora Kassner Philipp Dufter Hinrich Schütze 13 132 0 01 Feb 2021
GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing Tao Yu Chien-Sheng Wu Xi Victoria Lin Bailin Wang Y. Tan Xinyi Yang Dragomir R. Radev R. Socher Caiming Xiong LMTD 6 247 0 29 Sep 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,424 0 23 Jan 2020
MLQA: Evaluating Cross-lingual Extractive Question Answering Patrick Lewis Barlas Oğuz Ruty Rinott Sebastian Riedel Holger Schwenk ELM 242 490 0 16 Oct 2019