A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives

25 February 2021

Papers citing "A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives"

8 / 8 papers shown

Title
Low Fidelity Visuo-Tactile Pretraining Improves Vision-Only Manipulation Performance Selam Gano Abraham George A. Farimani OnRL 28 0 0 21 Jun 2024
Dense Contrastive Visual-Linguistic Pretraining Lei Shi Kai Shuang Shijie Geng Peng Gao Zuohui Fu Gerard de Melo Yunpeng Chen Sen Su VLM SSL 40 6 0 24 Sep 2021
Contrastive Learning Inverts the Data Generating Process Roland S. Zimmermann Yash Sharma Steffen Schneider Matthias Bethge Wieland Brendel SSL 227 206 0 17 Feb 2021
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining Yu Meng Chenyan Xiong Payal Bajaj Saurabh Tiwary Paul N. Bennett Jiawei Han Xia Song 111 201 0 16 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 291 2,875 0 11 Feb 2021
CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding Yanru Qu Dinghan Shen Yelong Shen Sandra Sajeev Jiawei Han Weizhu Chen 115 65 0 16 Oct 2020
A Mutual Information Maximization Perspective of Language Representation Learning Lingpeng Kong Cyprien de Masson dÁutume Wang Ling Lei Yu Zihang Dai Dani Yogatama SSL 196 163 0 18 Oct 2019
Efficient Estimation of Word Representations in Vector Space Tomáš Mikolov Kai Chen G. Corrado J. Dean 3DV 228 29,632 0 16 Jan 2013