Dissecting Contextual Word Embeddings: Architecture and Representation

27 August 2018

Luke Zettlemoyer

Papers citing "Dissecting Contextual Word Embeddings: Architecture and Representation"

50 / 62 papers shown

Title
Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey) S. Oota Zijiao Chen Manish Gupta R. Bapi G. Jobard F. Alexandre X. Hinaut 3DV AI4CE 49 11 0 31 Dec 2024
Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation Yiming Wang Pei Zhang Baosong Yang Derek F. Wong Rui-cang Wang LRM 48 4 0 17 Oct 2024
Monitoring Latent World States in Language Models with Propositional Probes Jiahai Feng Stuart Russell Jacob Steinhardt HILM 37 6 0 27 Jun 2024
Is Bigger and Deeper Always Better? Probing LLaMA Across Scales and Layers Nuo Chen Ning Wu Shining Liang Ming Gong Linjun Shou Dongmei Zhang Jia Li LRM 19 9 0 07 Dec 2023
From Heuristic to Analytic: Cognitively Motivated Strategies for Coherent Physical Commonsense Reasoning Zheyuan Zhang Shane Storks Fengyuan Hu Sungryull Sohn Moontae Lee Honglak Lee Joyce Chai LRM 34 3 0 24 Oct 2023
Disentangling the Linguistic Competence of Privacy-Preserving BERT Stefan Arnold Nils Kemmerzell Annika Schreiner 25 0 0 17 Oct 2023
DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers Anna Langedijk Hosein Mohebbi Gabriele Sarti Willem H. Zuidema Jaap Jumelet 21 10 0 05 Oct 2023
The EarlyBIRD Catches the Bug: On Exploiting Early Layers of Encoder Models for More Efficient Code Classification Anastasiia Grishina Max Hort Leon Moonen 22 6 0 08 May 2023
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models Hong Liu Sang Michael Xie Zhiyuan Li Tengyu Ma AI4CE 32 49 0 25 Oct 2022
Structural generalization is hard for sequence-to-sequence models Yuekun Yao Alexander Koller 24 21 0 24 Oct 2022
Probing with Noise: Unpicking the Warp and Weft of Embeddings Filip Klubicka John D. Kelleher 28 4 0 21 Oct 2022
On the Explainability of Natural Language Processing Deep Models Julia El Zini M. Awad 25 82 0 13 Oct 2022
PainPoints: A Framework for Language-based Detection of Chronic Pain and Expert-Collaborative Text-Summarization S. Fadnavis Amit Dhurandhar R. Norel Jenna M. Reinen C. Agurto E. Secchettin V. Schweiger Giovanni Perini Guillermo Cecchi 26 1 0 14 Sep 2022
TransPolymer: a Transformer-based language model for polymer property predictions Changwen Xu Yuyang Wang A. Farimani 19 86 0 03 Sep 2022
Probing via Prompting Jiaoda Li Ryan Cotterell Mrinmaya Sachan 29 13 0 04 Jul 2022
Knowledge Distillation of Transformer-based Language Models Revisited Chengqiang Lu Jianwei Zhang Yunfei Chu Zhengyu Chen Jingren Zhou Fei Wu Haiqing Chen Hongxia Yang VLM 25 10 0 29 Jun 2022
Assessing the Limits of the Distributional Hypothesis in Semantic Spaces: Trait-based Relational Knowledge and the Impact of Co-occurrences Mark Anderson Jose Camacho-Collados 30 0 0 16 May 2022
Fake news detection using parallel BERT deep neural networks Mahmood Farokhian V. Rafe H. Veisi GNN 20 14 0 10 Apr 2022
Interpretation of Black Box NLP Models: A Survey Shivani Choudhary N. Chatterjee S. K. Saha FAtt 32 10 0 31 Mar 2022
E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning Jiangjie Chen Rui Xu Ziquan Fu Wei Shi Zhongqiao Li Xinbo Zhang Changzhi Sun Lei Li Yanghua Xiao Hao Zhou ELM 23 35 0 16 Mar 2022
Does Entity Abstraction Help Generative Transformers Reason? Nicolas Angelard-Gontier Siva Reddy C. Pal 19 5 0 05 Jan 2022
Inducing Causal Structure for Interpretable Neural Networks Atticus Geiger Zhengxuan Wu Hanson Lu J. Rozner Elisa Kreiss Thomas F. Icard Noah D. Goodman Christopher Potts CML OOD 18 70 0 01 Dec 2021
Recent Advances in Automated Question Answering In Biomedical Domain K. D. Baksi 16 0 0 10 Nov 2021
Conditional probing: measuring usable information beyond a baseline John Hewitt Kawin Ethayarajh Percy Liang Christopher D. Manning 31 55 0 19 Sep 2021
A Relation-Oriented Clustering Method for Open Relation Extraction Jun Zhao Tao Gui Qi Zhang Yaqian Zhou 37 33 0 15 Sep 2021
What do pre-trained code models know about code? Anjan Karmakar Romain Robbes ELM 24 87 0 25 Aug 2021
Do Vision Transformers See Like Convolutional Neural Networks? M. Raghu Thomas Unterthiner Simon Kornblith Chiyuan Zhang Alexey Dosovitskiy ViT 46 924 0 19 Aug 2021
Theoretical foundations and limits of word embeddings: what types of meaning can they capture? Alina Arseniev-Koehler 28 19 0 22 Jul 2021
What do End-to-End Speech Models Learn about Speaker, Language and Channel Information? A Layer-wise and Neuron-level Analysis Shammur A. Chowdhury Nadir Durrani Ahmed M. Ali 25 12 0 01 Jul 2021
Classifying vaccine sentiment tweets by modelling domain-specific representation and commonsense knowledge into context-aware attentive GRU Usman Naseem Matloob Khushi Jinman Kim A. Dunn 16 12 0 17 Jun 2021
LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond Daniel Loureiro A. Jorge Jose Camacho-Collados 33 26 0 26 May 2021
Local Interpretations for Explainable Natural Language Processing: A Survey Siwen Luo Hamish Ivison S. Han Josiah Poon MILM 33 48 0 20 Mar 2021
When is it permissible for artificial intelligence to lie? A trust-based approach Tae Wan Kim Tong Lu Lu Kyusong Lee Zhaoqi Cheng Yanhan Tang J. N. Hooker 16 4 0 09 Mar 2021
Language Modelling as a Multi-Task Problem Leon Weber Jaap Jumelet Elia Bruni Dieuwke Hupkes 18 13 0 27 Jan 2021
Automated Coding of Under-Studied Medical Concept Domains: Linking Physical Activity Reports to the International Classification of Functioning, Disability, and Health Denis R. Newman-Griffis Eric Fosler-Lussier 24 18 0 27 Nov 2020
Dynamic Contextualized Word Embeddings Valentin Hofmann J. Pierrehumbert Hinrich Schütze 29 51 0 23 Oct 2020
Mixed-Precision Embedding Using a Cache J. Yang Jianyu Huang Jongsoo Park P. T. P. Tang Andrew Tulloch 11 36 0 21 Oct 2020
Towards Interpreting BERT for Reading Comprehension Based QA Sahana Ramnath Preksha Nema Deep Sahni Mitesh M. Khapra 34 30 0 18 Oct 2020
Mischief: A Simple Black-Box Attack Against Transformer Architectures Adrian de Wynter AAML 24 1 0 16 Oct 2020
Neural Databases James Thorne Majid Yazdani Marzieh Saeidi Fabrizio Silvestri Sebastian Riedel A. Halevy NAI 26 9 0 14 Oct 2020
Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data Jonathan Pilault Amine Elhattami C. Pal CLL MoE 19 89 0 19 Sep 2020
Analysis and Evaluation of Language Models for Word Sense Disambiguation Daniel Loureiro Kiamehr Rezaee Mohammad Taher Pilehvar Jose Camacho-Collados 13 13 0 26 Aug 2020
An exploration of the encoding of grammatical gender in word embeddings Hartger Veeman A. Basirat FaML 12 1 0 05 Aug 2020
BERTology Meets Biology: Interpreting Attention in Protein Language Models Jesse Vig Ali Madani L. Varshney Caiming Xiong R. Socher Nazneen Rajani 15 288 0 26 Jun 2020
On Incorporating Structural Information to improve Dialogue Response Generation Nikita Moghe Priyesh Vijayan Balaraman Ravindran Mitesh M. Khapra 22 6 0 28 May 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 15 39,979 0 28 May 2020
Probing Contextual Language Models for Common Ground with Visual Representations Gabriel Ilharco Rowan Zellers Ali Farhadi Hannaneh Hajishirzi 22 14 0 01 May 2020
oLMpics -- On what Language Model Pre-training Captures Alon Talmor Yanai Elazar Yoav Goldberg Jonathan Berant LRM 17 300 0 31 Dec 2019
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model Wenhan Xiong Jingfei Du William Yang Wang Veselin Stoyanov SSL KELM 36 201 0 20 Dec 2019
On the Linguistic Representational Power of Neural Machine Translation Models Yonatan Belinkov Nadir Durrani Fahim Dalvi Hassan Sajjad James R. Glass MILM 25 68 0 01 Nov 2019