RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 3,776 papers shown

Title
Orchid: Flexible and Data-Dependent Convolution for Sequence Modeling Mahdi Karami Ali Ghodsi VLM 42 6 0 28 Feb 2024
Cause and Effect: Can Large Language Models Truly Understand Causality? Swagata Ashwani Kshiteesh Hegde Nishith Reddy Mannuru Mayank Jindal Dushyant Singh Sengar Krishna Chaitanya Rao Kathala Dishant Banga Vinija Jain Aman Chadha LRM 43 18 0 28 Feb 2024
Polos: Multimodal Metric Learning from Human Feedback for Image Captioning Yuiga Wada Kanta Kaneda Daichi Saito Komei Sugiura 34 24 0 28 Feb 2024
Transformer-based Parameter Estimation in Statistics Xiaoxin Yin David S. Yin 19 0 0 28 Feb 2024
Downstream Task Guided Masking Learning in Masked Autoencoders Using Multi-Level Optimization Han Guo Ramtin Hosseini Ruiyi Zhang Sai Ashish Somayajula Ranak Roy Chowdhury Rajesh K. Gupta Pengtao Xie 33 0 0 28 Feb 2024
Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey Xi Fang Weijie Xu Fiona Anting Tan Jiani Zhang Ziqing Hu Yanjun Qi Scott Nickleach Diego Socolinsky Srinivasan H. Sengamedu Christos Faloutsos LMTD ALM 37 66 0 27 Feb 2024
Acquiring Linguistic Knowledge from Multimodal Input Theodor Amariucai Alexander Scott Warstadt CLL 29 2 0 27 Feb 2024
Investigating Continual Pretraining in Large Language Models: Insights and Implications cCaugatay Yildiz Nishaanth Kanna Ravichandran Prishruit Punia Matthias Bethge B. Ermiş CLL KELM LRM 50 25 0 27 Feb 2024
Beyond Self-learned Attention: Mitigating Attention Bias in Transformer-based Models Using Attention Guidance Jiri Gesi Iftekhar Ahmed 51 0 0 26 Feb 2024
Two-stage Generative Question Answering on Temporal Knowledge Graph Using Large Language Models Yifu Gao Linbo Qiao Zhigang Kan Zhihua Wen Yongquan He Dongsheng Li 49 6 0 26 Feb 2024
Value Preferences Estimation and Disambiguation in Hybrid Participatory Systems Enrico Liscio Luciano Cavalcante Siebert Catholijn M. Jonker P. Murukannaiah 37 4 0 26 Feb 2024
From Text to Transformation: A Comprehensive Review of Large Language Models' Versatility Pravneet Kaur Gautam Siddharth Kashyap Ankit Kumar Md. Tabrez Nafis Sandeep Kumar Vikrant Shokeen LM&MA 48 54 0 25 Feb 2024
Training a Bilingual Language Model by Mapping Tokens onto a Shared Character Space Aviad Rom Kfir Bar 32 1 0 25 Feb 2024
IR2: Information Regularization for Information Retrieval Jianyou Wang Kaicheng Wang Xiaoyue Wang Weili Cao R. Paturi Leon Bergen 46 1 0 25 Feb 2024
Fine-tuning CLIP Text Encoders with Two-step Paraphrasing Hyunjae Kim Seunghyun Yoon Trung Bui Handong Zhao Quan Tran Franck Dernoncourt Jaewoo Kang CLIP 19 2 0 23 Feb 2024
Second-Order Fine-Tuning without Pain for LLMs:A Hessian Informed Zeroth-Order Optimizer Yanjun Zhao Sizhe Dang Haishan Ye Guang Dai Yi Qian Ivor W.Tsang 66 8 0 23 Feb 2024
Generalizing Reward Modeling for Out-of-Distribution Preference Learning Chen Jia 36 2 0 22 Feb 2024
The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations Aina Garí Soler Matthieu Labeau Chloé Clavel VLM 42 2 0 22 Feb 2024
Novi jezički modeli za srpski jezik Mihailo vSkorić 15 0 0 22 Feb 2024
Multi-modal Stance Detection: New Datasets and Model Bin Liang Ang Li Jingqian Zhao Lin Gui Min Yang Yue Yu Kam-Fai Wong Ruifeng Xu 34 4 0 22 Feb 2024
LLM-Assisted Content Conditional Debiasing for Fair Text Embedding Wenlong Deng Blair Chen Beidi Zhao Chiyu Zhang Xiaoxiao Li Christos Thrampoulidis 35 0 0 22 Feb 2024
Rethinking Scientific Summarization Evaluation: Grounding Explainable Metrics on Facet-aware Benchmark Xiuying Chen Tairan Wang Qingqing Zhu Taicheng Guo Shen Gao Zhiyong Lu Xin Gao Xiangliang Zhang 75 2 0 22 Feb 2024
COBIAS: Assessing the Contextual Reliability of Bias Benchmarks for Language Models Priyanshul Govil Hemang Jain Vamshi Bonagiri Aman Chadha Ponnurangam Kumaraguru Manas Gaur S. Dey 50 2 0 22 Feb 2024
Cost-Efficient Subjective Task Annotation and Modeling through Few-Shot Annotator Adaptation Preni Golazizian Ali Omrani Alireza S. Ziabari Morteza Dehghani 18 1 0 21 Feb 2024
Beyond Hate Speech: NLP's Challenges and Opportunities in Uncovering Dehumanizing Language Hezhao Zhang Lasana Harris N. Moosavi AILaw 41 1 0 21 Feb 2024
An Effective Incorporating Heterogeneous Knowledge Curriculum Learning for Sequence Labeling Xuemei Tang Qi Su 16 0 0 21 Feb 2024
Infrastructure Ombudsman: Mining Future Failure Concerns from Structural Disaster Response Towhid Chowdhury Soumyajit Datta Naveen Sharma Ashiqur R. KhudaBukhsh AI4CE 34 4 0 21 Feb 2024
Measuring Social Biases in Masked Language Models by Proxy of Prediction Quality Rahul Zalkikar Kanchan Chandra 29 1 0 21 Feb 2024
OLViT: Multi-Modal State Tracking via Attention-Based Embeddings for Video-Grounded Dialog Adnen Abdessaied Manuel von Hochmeister Andreas Bulling 40 2 0 20 Feb 2024
LoRA+: Efficient Low Rank Adaptation of Large Models Soufiane Hayou Nikhil Ghosh Bin Yu AI4CE 32 141 0 19 Feb 2024
Language Model Adaptation to Specialized Domains through Selective Masking based on Genre and Topical Characteristics Anas Belfathi Ygor Gallina Nicolas Hernandez Richard Dufour Laura Monceaux 31 1 0 19 Feb 2024
Acquiring Clean Language Models from Backdoor Poisoned Datasets by Downscaling Frequency Space Zongru Wu Zhuosheng Zhang Pengzhou Cheng Gongshen Liu AAML 44 4 0 19 Feb 2024
SIBO: A Simple Booster for Parameter-Efficient Fine-Tuning Zhihao Wen Jie Zhang Yuan Fang MoE 34 3 0 19 Feb 2024
Machine-Generated Text Localization Zhongping Zhang Wenda Qin Bryan A. Plummer DeLMO 34 5 0 19 Feb 2024
Puzzle Solving using Reasoning of Large Language Models: A Survey Panagiotis Giadikiaroglou Maria Lymperaiou Giorgos Filandrianos Giorgos Stamou ELM ReLM LRM 16 24 0 17 Feb 2024
Prompt-Based Bias Calibration for Better Zero/Few-Shot Learning of Language Models Kang He Yinghan Long Kaushik Roy 28 2 0 15 Feb 2024
Fast Vocabulary Transfer for Language Model Compression Leonidas Gee Andrea Zugarini Leonardo Rigutini Paolo Torroni 35 26 0 15 Feb 2024
Punctuation Restoration Improves Structure Understanding Without Supervision Junghyun Min Minho Lee Woochul Lee Yeonsoo Lee 56 1 0 13 Feb 2024
Event-Keyed Summarization William Gantt Alexander Martin Pavlo Kuchmiichuk Aaron Steven White 22 1 0 10 Feb 2024
Large Language Models: A Survey Shervin Minaee Tomáš Mikolov Narjes Nikzad M. Asgari-Chenaghlu R. Socher Xavier Amatriain Jianfeng Gao ALM LM&MA ELM 122 369 0 09 Feb 2024
Efficient Models for the Detection of Hate, Abuse and Profanity Christoph Tillmann Aashka Trivedi Bishwaranjan Bhattacharjee VLM 16 0 0 08 Feb 2024
Edu-ConvoKit: An Open-Source Library for Education Conversation Data Rose E. Wang Dorottya Demszky 15 9 0 07 Feb 2024
Comparison of Topic Modelling Approaches in the Banking Context Bayode Ogunleye Tonderai Maswera Laurence Hirsch J. Gaudoin T. Brunsdon AI4TS 35 41 0 05 Feb 2024
Domain Adaptation of Multilingual Semantic Search -- Literature Review Anna Bringmann Anastasia Zhukova VLM 30 0 0 05 Feb 2024
Adversarial Text Purification: A Large Language Model Approach for Defense Raha Moraffah Shubh Khandelwal Amrita Bhattacharjee Huan Liu DeLMO AAML 26 5 0 05 Feb 2024
From Data Creator to Data Reuser: Distance Matters C. Borgman Paul T. Groth 13 5 0 05 Feb 2024
DE $^3$ -BERT: Distance-Enhanced Early Exiting for BERT based on Prototypical Networks Jianing He Qi Zhang Weiping Ding Duoqian Miao Jun Zhao Liang Hu LongBing Cao 36 3 0 03 Feb 2024
Nomic Embed: Training a Reproducible Long Context Text Embedder Zach Nussbaum John X. Morris Brandon Duderstadt Andriy Mulyar 19 95 0 02 Feb 2024
DoubleMLDeep: Estimation of Causal Effects with Multimodal Data Sven Klaassen Jan Teichert-Kluge Philipp Bach Victor Chernozhukov Martin Spindler Suhas Vijaykumar BDL CML 18 6 0 01 Feb 2024
ALISON: Fast and Effective Stylometric Authorship Obfuscation Eric Xing Saranya Venkatraman Thai V. Le Dongwon Lee DeLMO 22 1 0 01 Feb 2024