RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 2,933 papers shown

Title
Small Language Models: Survey, Measurements, and Insights Zhenyan Lu Xiang Li Dongqi Cai Rongjie Yi Fangming Liu Xiwen Zhang Nicholas D. Lane Mengwei Xu ObjD LRM 51 36 0 24 Sep 2024
Language-based Audio Moment Retrieval Hokuto Munakata Taichi Nishimura Shota Nakada Tatsuya Komatsu 28 1 0 24 Sep 2024
FMDLlama: Financial Misinformation Detection based on Large Language Models Zhiwei Liu Xin Zhang Kailai Yang Qianqian Xie Jimin Huang Sophia Ananiadou ALM 22 2 0 24 Sep 2024
ToxiCraft: A Novel Framework for Synthetic Generation of Harmful Information Zheng Hui Zhaoxiao Guo Hang Zhao Juanyong Duan Congrui Huang 25 6 0 23 Sep 2024
The X Types -- Mapping the Semantics of the Twitter Sphere Ogen Schlachet Drukerman Einat Minkov 26 0 0 22 Sep 2024
Thought-Path Contrastive Learning via Premise-Oriented Data Augmentation for Logical Reading Comprehension Chenxu Wang Ping Jian Zhen Yang LRM 22 0 0 22 Sep 2024
LLM-Measure: Generating Valid, Consistent, and Reproducible Text-Based Measures for Social Science Research Yi Yang Hanyu Duan Jiaxin Liu Kar Yan Tam 16 0 0 19 Sep 2024
Smirk: An Atomically Complete Tokenizer for Molecular Foundation Models Alexius Wadell Anoushka Bhutani Venkatasubramanian Viswanathan 85 0 0 19 Sep 2024
Detecting LGBTQ+ Instances of Cyberbullying Muhammad Arslan Manuel Sandoval Madrigal Mohammed Abuhamad Deborah L. Hall Yasin N. Silva 21 0 0 18 Sep 2024
DocMamba: Efficient Document Pre-training with State Space Model Pengfei Hu Zhenrong Zhang Jiefeng Ma Shuhang Liu Jun Du Jianshu Zhang Mamba 35 1 0 18 Sep 2024
Mamba Fusion: Learning Actions Through Questioning Zhikang Dong Apoorva Beedu Jason Sheinkopf Irfan Essa Mamba 65 2 0 17 Sep 2024
Norm of Mean Contextualized Embeddings Determines their Variance Hiroaki Yamagiwa Hidetoshi Shimodaira 25 0 0 17 Sep 2024
Contextual Breach: Assessing the Robustness of Transformer-based QA Models Asir Saadat Nahian Ibn Asad Md Farhan Ishmam AAML 36 0 0 17 Sep 2024
Relative Representations: Topological and Geometric Perspectives Alejandro García-Castellanos G. Marchetti Danica Kragic Martina Scolamiero 48 0 0 17 Sep 2024
MusicLIME: Explainable Multimodal Music Understanding Theodoros Sotirou Vassilis Lyberatos Orfeas Menis-Mastromichalakis Giorgos Stamou 26 2 0 16 Sep 2024
Frequency-Guided Masking for Enhanced Vision Self-Supervised Learning Amin Karimi Monsefi Mengxi Zhou Nastaran Karimi Monsefi Ser-Nam Lim Wei-Lun Chao R. Ramnath 36 1 0 16 Sep 2024
Estimating Wage Disparities Using Foundation Models Keyon Vafa Susan Athey David M. Blei 70 1 0 15 Sep 2024
Enhancing adversarial robustness in Natural Language Inference using explanations Alexandros Koulakos Maria Lymperaiou Giorgos Filandrianos Giorgos Stamou SILM AAML 35 0 0 11 Sep 2024
Automated Speaking Assessment of Conversation Tests with Novel Graph-based Modeling on Spoken Response Coherence Jiun-Ting Li Bi-Cheng Yan Tien-Hong Lo Yi-Cheng Wang Yung-Chang Hsu Berlin Chen AuLLM 18 1 0 11 Sep 2024
Can Large Language Models Unlock Novel Scientific Research Ideas? Sandeep Kumar Tirthankar Ghosal Vinayak Goyal Asif Ekbal ALM LRM AI4CE 31 10 0 10 Sep 2024
Driving with Prior Maps: Unified Vector Prior Encoding for Autonomous Vehicle Mapping Shuang Zeng Xinyuan Chang Xinran Liu Zheng Pan Xing Wei 37 1 0 09 Sep 2024
RAGent: Retrieval-based Access Control Policy Generation Sakuna Jayasundara N. Arachchilage Giovanni Russello 44 1 0 08 Sep 2024
Constrained Multi-Layer Contrastive Learning for Implicit Discourse Relationship Recognition Yiheng Wu Junhui Li Muhua Zhu 19 0 0 07 Sep 2024
Context is the Key: Backdoor Attacks for In-Context Learning with Vision Transformers Gorka Abad S. Picek Lorenzo Cavallaro A. Urbieta SILM 37 0 0 06 Sep 2024
From Calculation to Adjudication: Examining LLM judges on Mathematical Reasoning Tasks Andreas Stephan D. Zhu Matthias Aßenmacher Xiaoyu Shen Benjamin Roth ELM 45 4 0 06 Sep 2024
Oddballness: universal anomaly detection with language models Filip Graliñski Ryszard Staruch Krzysztof Jurkiewicz 29 1 0 04 Sep 2024
Diversify-verify-adapt: Efficient and Robust Retrieval-Augmented Ambiguous Question Answering Yeonjun In Sungchul Kim Ryan A. Rossi Md Mehrab Tanjim Tong Yu Ritwik Sinha Chanyoung Park 30 0 0 04 Sep 2024
Expanding on EnCLAP with Auxiliary Retrieval Model for Automated Audio Captioning Jaeyeon Kim Jaeyoon Jung Minjeong Jeon Sang Hoon Woo Jinjoo Lee 24 1 0 02 Sep 2024
MaskMol: Knowledge-guided Molecular Image Pre-Training Framework for Activity Cliffs Zhixiang Cheng Hongxin Xiang Pengsen Ma Li Zeng Xin Jin ... Yang Deng Bosheng Song Xinxin Feng Changhui Deng Xiangxiang Zeng 24 0 0 02 Sep 2024
From Prediction to Application: Language Model-based Code Knowledge Tracing with Domain Adaptive Pre-Training and Automatic Feedback System with Pedagogical Prompting for Comprehensive Programming Education Unggi Lee Jiyeong Bae Yeonji Jung Minji Kang Gyuri Byun ... Sookbun Lee Jaekwon Park Taekyung Ahn Gunho Lee Hyeoncheol Kim AI4Ed KELM 26 1 0 31 Aug 2024
SYNTHEVAL: Hybrid Behavioral Testing of NLP Models with Synthetic CheckLists Raoyuan Zhao Abdullatif Köksal Yihong Liu Leonie Weissweiler Anna Korhonen Hinrich Schütze SyDa 36 1 0 30 Aug 2024
MoRe Fine-Tuning with 10x Fewer Parameters Wenxuan Tan Nicholas Roberts Tzu-Heng Huang Jitian Zhao John Cooper Samuel Guo Chengyu Duan Frederic Sala 23 0 0 30 Aug 2024
HyPA-RAG: A Hybrid Parameter Adaptive Retrieval-Augmented Generation System for AI Legal and Policy Applications Rishi Kalra Zekun Wu Ayesha Gulley Airlie Hilliard Xin Guan Adriano Soares Koshiyama Philip C. Treleaven RALM AILaw 47 5 0 29 Aug 2024
Can Optimization Trajectories Explain Multi-Task Transfer? David Mueller Mark Dredze Nicholas Andrews 53 1 0 26 Aug 2024
The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model design Artem Snegirev Maria Tikhonova Anna Maksimova Alena Fenogenova Alexander Abramov 24 4 0 22 Aug 2024
Counterfactuals As a Means for Evaluating Faithfulness of Attribution Methods in Autoregressive Language Models Sepehr Kamahi Yadollah Yaghoobzadeh 32 0 0 21 Aug 2024
Threshold Filtering Packing for Supervised Fine-Tuning: Training Related Samples within Packs Jiancheng Dong Lei Jiang Wei Jin Lu Cheng 36 1 0 18 Aug 2024
SelectLLM: Query-Aware Efficient Selection Algorithm for Large Language Models Kaushal Kumar Maurya KV Aditya Srivatsa Ekaterina Kochmar 38 2 0 16 Aug 2024
EmoDynamiX: Emotional Support Dialogue Strategy Prediction by Modelling MiXed Emotions and Discourse Dynamics Chenwei Wan Matthieu Labeau Chloé Clavel 35 0 0 16 Aug 2024
BERT's Conceptual Cartography: Mapping the Landscapes of Meaning Nina Haket Ryan Daniels 19 0 0 13 Aug 2024
Neural embedding of beliefs reveals the role of relative dissonance in human decision-making Byunghwee Lee Rachith Aiyappa Yong-Yeol Ahn Haewoon Kwak Jisun An 20 2 0 13 Aug 2024
LipidBERT: A Lipid Language Model Pre-trained on METiS de novo Lipid Library Tianhao Yu Cai Yao Zhuorui Sun Feng Shi Lin Zhang ... Xicheng Zhang Jiali Zou Wenshou Wang C. Lai Kai Wang 26 3 0 12 Aug 2024
Diffusion Guided Language Modeling Justin Lovelace Varsha Kishore Yiwei Chen Kilian Q. Weinberger 36 6 0 08 Aug 2024
LLM-DetectAIve: a Tool for Fine-Grained Machine-Generated Text Detection Mervat Abassy Kareem Elozeiri Alexander Aziz Minh Ngoc Ta Raj Vardhan Tomar ... Alham Fikri Aji Artem Shelmanov Nizar Habash Iryna Gurevych Preslav Nakov DeLMO 48 11 0 08 Aug 2024
Modelling Visual Semantics via Image Captioning to extract Enhanced Multi-Level Cross-Modal Semantic Incongruity Representation with Attention for Multimodal Sarcasm Detection Sajal Aggarwal Ananya Pandey Dinesh Kumar Vishwakarma 41 1 0 05 Aug 2024
StyEmp: Stylizing Empathetic Response Generation via Multi-Grained Prefix Encoder and Personality Reinforcement Yahui Fu Chenhui Chu Tatsuya Kawahara 29 2 0 05 Aug 2024
UnifiedNN: Efficient Neural Network Training on the Cloud Xingyu Lou Arthi Padmanabhan Spyridon Mastorakis FedML 31 0 0 02 Aug 2024
KNOWCOMP POKEMON Team at DialAM-2024: A Two-Stage Pipeline for Detecting Relations in Dialogical Argument Mining Zihao Zheng Zhaowei Wang Qing Zong Yangqiu Song LRM 40 1 0 29 Jul 2024
Banyan: Improved Representation Learning with Explicit Structure Mattia Opper N. Siddharth 31 1 0 25 Jul 2024
Generating Sample-Based Musical Instruments Using Neural Audio Codec Language Models S. Nercessian Johannes Imort Ninon Devis Frederik Blang 29 1 0 22 Jul 2024