Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models

20 May 2020

Dan Jurafsky

Papers citing "Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models"

50 / 55 papers shown

Title
Tgea: An error-annotated dataset and benchmark tasks for text generation from pretrained language models Jie He Bo Peng Yi-Lun Liao Qun Liu Deyi Xiong 109 8 0 06 Mar 2025
Learning by Analogy: Enhancing Few-Shot Prompting for Math Word Problem Solving with Computational Graph-Based Retrieval Xiaocong Yang Jiacheng Lin Zhenting Wang Chengxiang Zhai ReLM 118 0 0 25 Nov 2024
Contrastive Learning Subspace for Text Clustering Qian Yong Chen Chen Xiabing Zhou 65 0 0 26 Aug 2024
LegalTurk Optimized BERT for Multi-Label Text Classification and NER Farnaz Zeidi Mehmet Fatih Amasyali Çiğdem Erol VLM 58 2 0 30 Jun 2024
DEPTH: Discourse Education through Pre-Training Hierarchically Zachary Bamberger Ofek Glick Chaim Baskin Yonatan Belinkov 126 0 0 13 May 2024
Towards Deviation-Robust Agent Navigation via Perturbation-Aware Contrastive Learning Bingqian Lin Yanxin Long Yi Zhu Fengda Zhu Xiaodan Liang QiXiang Ye Liang Lin 84 5 0 09 Mar 2024
CAMBranch: Contrastive Learning with Augmented MILPs for Branching Jiacheng Lin Meng Xu Zhihua Xiong Huangang Wang 128 4 0 06 Feb 2024
Contrastive Multi-Level Graph Neural Networks for Session-based Recommendation F. Wang Xingyu Gao Zhenyu Chen Lei Lyu 52 12 0 06 Nov 2023
Incorporating Domain Knowledge Graph into Multimodal Movie Genre Classification with Self-Supervised Attention and Contrastive Learning Jiaqi Li Guilin Qi Chuanyi Zhang Yongrui Chen Yiming Tan Chenlong Xia Ye Tian 81 3 0 12 Oct 2023
Enhancing Pre-Trained Language Models with Sentence Position Embeddings for Rhetorical Roles Recognition in Legal Opinions Anas Belfathi Nicolas Hernandez Laura Monceaux AILaw 66 4 0 08 Oct 2023
Improving the Generalization Ability in Essay Coherence Evaluation through Monotonic Constraints Chen Zheng Huan Zhang Yan Zhao Yuxuan Lai 64 0 0 25 Jul 2023
A Side-by-side Comparison of Transformers for English Implicit Discourse Relation Classification Bruce W. Lee Bongseok Yang J. Lee 76 0 0 07 Jul 2023
ContrastNER: Contrastive-based Prompt Tuning for Few-shot NER Amirhossein Layegh A. H. Payberah A. Soylu Dumitru Roman M. Matskin VLM 80 8 0 29 May 2023
COLA: Contextualized Commonsense Causal Reasoning from the Causal Inference Perspective Zhaowei Wang Quyet V. Do Hongming Zhang Jiayao Zhang Weiqi Wang Tianqing Fang Yangqiu Song Ginny Wong Simon See LRM 64 31 0 09 May 2023
D2CSE: Difference-aware Deep continuous prompts for Contrastive Sentence Embeddings HyunJae Lee VLM 49 0 0 18 Apr 2023
Modeling Sequential Sentence Relation to Improve Cross-lingual Dense Retrieval Shunyu Zhang Yaobo Liang Ming Gong Daxin Jiang Nan Duan 83 4 0 03 Feb 2023
Few-shot Object Detection with Refined Contrastive Learning Zeyu Shangguan Lian Huai Tong Liu Xingqun Jiang 74 7 0 24 Nov 2022
Beyond Model Interpretability: On the Faithfulness and Adversarial Robustness of Contrastive Textual Explanations Julia El Zini M. Awad AAML 62 2 0 17 Oct 2022
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling Haw-Shiuan Chang Ruei-Yao Sun Kathryn Ricci Andrew McCallum 108 15 0 10 Oct 2022
MENLI: Robust Evaluation Metrics from Natural Language Inference Yanran Chen Steffen Eger 107 18 0 15 Aug 2022
To Answer or Not to Answer? Improving Machine Reading Comprehension Model with Span-based Contrastive Learning Yunjie Ji Liangyu Chen Chenxiao Dou Baochang Ma Xiangang Li 82 5 0 02 Aug 2022
Evaluation Benchmarks for Spanish Sentence Representations Vladimir Araujo Andrés Carvallo Souvik Kundu J. Canete Marcelo Mendoza Robert E. Mercer Felipe Bravo-Marquez Marie-Francine Moens Alvaro Soto ELM 60 10 0 15 Apr 2022
Contrastive language and vision learning of general fashion concepts P. Chia Giuseppe Attanasio Federico Bianchi Silvia Terragni A. Magalhães Diogo Gonçalves C. Greco Jacopo Tagliabue CLIP 115 44 0 08 Apr 2022
A Dual-Contrastive Framework for Low-Resource Cross-Lingual Named Entity Recognition Yingwen Fu Nankai Lin Ziyu Yang Shengyi Jiang 96 4 0 02 Apr 2022
Language modeling via stochastic processes Rose E. Wang Esin Durmus Noah D. Goodman Tatsunori Hashimoto BDL AI4TS 117 25 0 21 Mar 2022
How does the pre-training objective affect what large language models learn about linguistic properties? Ahmed Alajrami Nikolaos Aletras 82 20 0 20 Mar 2022
RecursiveMix: Mixed Learning with History Lingfeng Yang Xiang Li Borui Zhao Renjie Song Jian Yang VLM 86 20 0 14 Mar 2022
The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking Hai-Tao Zheng Qingyu Zhou Yongqian Li Zhongli Li Ruiyang Liu Rongyi Sun Zizhen Wang Chao Li Yunbo Cao Haitao Zheng 135 70 0 02 Mar 2022
A Mutually Reinforced Framework for Pretrained Sentence Embeddings Junhan Yang Zheng Liu Shitao Xiao Jianxun Lian Lijun Wu Defu Lian Guangzhong Sun Xing Xie SSL 46 2 0 28 Feb 2022
DiscoScore: Evaluating Text Generation with BERT and Discourse Coherence Wei Zhao Michael Strube Steffen Eger 121 38 0 26 Jan 2022
PhysNLU: A Language Resource for Evaluating Natural Language Understanding and Explanation Coherence in Physics Jordan Meadows Zili Zhou André Freitas AIMat 117 3 0 12 Jan 2022
Long Context Question Answering via Supervised Contrastive Learning Avi Caciularu Ido Dagan Jacob Goldberger Arman Cohan RALM 92 24 0 16 Dec 2021
CLLD: Contrastive Learning with Label Distance for Text Classification Jinhe Lan Qingyuan Zhan Chenhao Jiang Kunping Yuan Desheng Wang VLM 64 2 0 25 Oct 2021
Local and Global Context-Based Pairwise Models for Sentence Ordering R. Manku A. Paul 110 3 0 08 Oct 2021
PoNet: Pooling Network for Efficient Token Mixing in Long Sequences Chao-Hong Tan Qian Chen Wen Wang Qinglin Zhang Siqi Zheng Zhenhua Ling ViT 51 12 0 06 Oct 2021
Augmenting BERT-style Models with Predictive Coding to Improve Discourse-level Representations Vladimir Araujo Andrés Villa Marcelo Mendoza Marie-Francine Moens Alvaro Soto 61 7 0 10 Sep 2021
Sequence Level Contrastive Learning for Text Summarization Shusheng Xu Xingxing Zhang Yi Wu Furu Wei 117 98 0 08 Sep 2021
Contrastive Representation Learning for Exemplar-Guided Paraphrase Generation Haoran Yang W. Lam Pijian Li 82 15 0 03 Sep 2021
Sequence Model with Self-Adaptive Sliding Window for Efficient Spoken Document Segmentation Qinglin Zhang Qian Chen Yali Li Jiaqing Liu Wen Wang 148 16 0 20 Jul 2021
CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding Dong Wang Ning Ding Pijian Li Haitao Zheng AAML 68 118 0 01 Jul 2021
Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions Dorottya Demszky Jing Liu Zid Mancenido Julie Cohen H. Hill Dan Jurafsky Tatsunori Hashimoto 117 67 0 07 Jun 2021
Bi-Granularity Contrastive Learning for Post-Training in Few-Shot Scene Ruikun Luo Guanhuan Huang Xiaojun Quan CLL 76 10 0 04 Jun 2021
CLEVE: Contrastive Pre-training for Event Extraction Ziqi Wang Xiaozhi Wang Xu Han Yankai Lin Lei Hou Zhiyuan Liu Peng Li Juan-Zi Li Jie Zhou 84 118 0 30 May 2021
Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherence Jian Guan Xiaoxi Mao Changjie Fan Zitao Liu Wenbiao Ding Minlie Huang AuLLM 95 83 0 19 May 2021
REPT: Bridging Language Models and Machine Reading Comprehension via Retrieval-Based Pre-training Fangkai Jiao Yangyang Guo Yilin Niu Feng Ji Feng-Lin Li Liqiang Nie LRM 69 12 0 10 May 2021
A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives Nils Rethmeier Isabelle Augenstein SSL VLM 159 94 0 25 Feb 2021
Improving Commonsense Causal Reasoning by Adversarial Training and Data Augmentation Ieva Staliunaite P. Gorinski Ignacio Iacobacci LRM 277 22 0 13 Jan 2021
Towards Modelling Coherence in Spoken Discourse Rajaswa Patil Yaman Kumar Singla R. Shah Mika Hama Roger Zimmermann AuLLM 121 8 0 31 Dec 2020
A Survey on Contrastive Self-supervised Learning Ashish Jaiswal Ashwin Ramesh Babu Mohammad Zaki Zadeh Debapriya Banerjee F. Makedon SSL 154 1,411 0 31 Oct 2020
SLM: Learning a Discourse Language Representation with Sentence Unshuffling Haejun Lee Drew A. Hudson Kangwook Lee Christopher D. Manning SSL 119 52 0 30 Oct 2020