What Context Features Can Transformer Language Models Use?

15 June 2021

Papers citing "What Context Features Can Transformer Language Models Use?"

44 / 44 papers shown

Title
LIFBench: Evaluating the Instruction Following Performance and Stability of Large Language Models in Long-Context Scenarios Xiaodong Wu Minhao Wang Yichen Liu Xiaoming Shi He Yan Xiangju Lu Junmin Zhu Wei Zhang 133 3 0 11 Nov 2024
Assessing Brittleness of Image-Text Retrieval Benchmarks from Vision-Language Models Perspective Mariya Hendriksen Shuo Zhang R. Reinanda Mohamed Yahya Edgar Meij Maarten de Rijke 38 0 0 21 Jul 2024
LLM-Mediated Domain-Specific Voice Agents: The Case of TextileBot Shu Zhong Elia Gatti James Hardwick Miriam Ribul Youngjun Cho Marianna Obrist 36 3 0 15 Jun 2024
A Human-Computer Collaborative Tool for Training a Single Large Language Model Agent into a Network through Few Examples Lihang Pan Yuxuan Li Chun Yu Yuanchun Shi LLMAG 38 1 0 24 Apr 2024
Contextual Feature Extraction Hierarchies Converge in Large Language Models and the Brain Gavin Mischler Yinghao Aaron Li Stephan Bickel A. Mehta N. Mesgarani 17 23 0 31 Jan 2024
Revisiting Topic-Guided Language Models Carolina Zheng Keyon Vafa David M. Blei BDL 27 1 0 04 Dec 2023
LILO: Learning Interpretable Libraries by Compressing and Documenting Code Gabriel Grand L. Wong Matthew Bowers Theo X. Olausson Muxin Liu Joshua B. Tenenbaum Jacob Andreas 16 21 0 30 Oct 2023
Can Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining? Fei-Yue Wang Liang Ding Jun Rao Ye Liu Li Shen Changxing Ding 32 15 0 24 Aug 2023
Lost in the Middle: How Language Models Use Long Contexts Nelson F. Liu Kevin Lin John Hewitt Ashwin Paranjape Michele Bevilacqua Fabio Petroni Percy Liang RALM 27 1,389 0 06 Jul 2023
Bring Your Own Data! Self-Supervised Evaluation for Large Language Models Neel Jain Khalid Saifullah Yuxin Wen John Kirchenbauer Manli Shu Aniruddha Saha Micah Goldblum Jonas Geiping Tom Goldstein ALM ELM 22 23 0 23 Jun 2023
MTCue: Learning Zero-Shot Control of Extra-Textual Attributes by Leveraging Unstructured Context in Neural Machine Translation S. Vincent R. Flynn Carolina Scarton 18 4 0 25 May 2023
Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models Natalie Shapira Mosh Levy S. Alavi Xuhui Zhou Yejin Choi Yoav Goldberg Maarten Sap Vered Shwartz LLMAG ELM 20 113 0 24 May 2023
Don't Take This Out of Context! On the Need for Contextual Models and Evaluations for Stylistic Rewriting Akhila Yerukola Xuhui Zhou Elizabeth Clark Maarten Sap 23 6 0 24 May 2023
Revisiting Entropy Rate Constancy in Text Vivek Verma Nicholas Tomlin Dan Klein 14 4 0 20 May 2023
Language Model Behavior: A Comprehensive Survey Tyler A. Chang Benjamin Bergen VLM LRM LM&MA 27 102 0 20 Mar 2023
RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions Yunlong Wang Shuyuan Shen Brian Y. Lim 28 88 0 19 Feb 2023
Position Matters! Empirical Study of Order Effect in Knowledge-grounded Dialogue Hsuan Su Shachi H. Kumar Sahisnu Mazumder Wenda Chen R. Manuvinakurike Eda Okur Saurav Sahay L. Nachman Shang-Tse Chen Hung-yi Lee 23 3 0 12 Feb 2023
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers K. Choromanski Shanda Li Valerii Likhosherstov Kumar Avinava Dubey Shengjie Luo Di He Yiming Yang Tamás Sarlós Thomas Weingarten Adrian Weller 23 8 0 03 Feb 2023
Black-box language model explanation by context length probing Ondřej Cífka Antoine Liutkus MILM LRM 6 6 0 30 Dec 2022
Identifying and Manipulating the Personality Traits of Language Models Graham Caron Shashank Srivastava 10 37 0 20 Dec 2022
Local Structure Matters Most in Most Languages Louis Clouâtre Prasanna Parthasarathi Amal Zouaq Sarath Chandar 26 1 0 09 Nov 2022
Detecting Languages Unintelligible to Multilingual Models through Local Structure Probes Louis Clouâtre Prasanna Parthasarathi Amal Zouaq Sarath Chandar 33 3 0 09 Nov 2022
Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionality Anuj Diwan Layne Berry Eunsol Choi David F. Harwath Kyle Mahowald CoGe 101 41 0 01 Nov 2022
Characterizing Verbatim Short-Term Memory in Neural Language Models K. Armeni C. Honey Tal Linzen KELM RALM 25 3 0 24 Oct 2022
Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs Maarten Sap Ronan Le Bras Daniel Fried Yejin Choi 22 205 0 24 Oct 2022
When and why vision-language models behave like bags-of-words, and what to do about it? Mert Yuksekgonul Federico Bianchi Pratyusha Kalluri Dan Jurafsky James Y. Zou VLM CoGe 28 362 0 04 Oct 2022
HYPRO: A Hybridly Normalized Probabilistic Model for Long-Horizon Prediction of Event Sequences Siqiao Xue X. Shi James Y. Zhang Hongyuan Mei AI4TS 19 34 0 04 Oct 2022
How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Models Hai Dang Lukas Mecke Florian Lehmann Sven Goller Daniel Buschek 12 95 0 03 Sep 2022
Context Limitations Make Neural Language Models More Human-Like Tatsuki Kuribayashi Yohei Oseki Ana Brassard Kentaro Inui 44 29 0 23 May 2022
Can language models learn from explanations in context? Andrew Kyle Lampinen Ishita Dasgupta Stephanie C. Y. Chan Kory Matthewson Michael Henry Tessler Antonia Creswell James L. McClelland Jane X. Wang Felix Hill LRM ReLM 31 283 0 05 Apr 2022
Word Order Does Matter (And Shuffled Language Models Know It) Vinit Ravishankar Mostafa Abdou Artur Kulmizev Anders Søgaard 17 44 0 21 Mar 2022
When classifying grammatical role, BERT doesn't care about word order... except when it matters Isabel Papadimitriou Richard Futrell Kyle Mahowald MILM 22 29 0 11 Mar 2022
Simple Local Attentions Remain Competitive for Long-Context Tasks Wenhan Xiong Barlas Ouguz Anchit Gupta Xilun Chen Diana Liskovich Omer Levy Wen-tau Yih Yashar Mehdad 36 29 0 14 Dec 2021
Quantifying the Task-Specific Information in Text-Based Classifications Zining Zhu Aparna Balagopalan Marzyeh Ghassemi Frank Rudzicz 28 4 0 17 Oct 2021
AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts Tongshuang Wu Michael Terry Carrie J. Cai LLMAG AI4CE LRM 24 444 0 04 Oct 2021
Shaking Syntactic Trees on the Sesame Street: Multilingual Probing with Controllable Perturbations Ekaterina Taktasheva Vladislav Mikhailov Ekaterina Artemova 8 13 0 28 Sep 2021
Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformers Stella Frank Emanuele Bugliarello Desmond Elliott 30 81 0 09 Sep 2021
A Bayesian Framework for Information-Theoretic Probing Tiago Pimentel Ryan Cotterell 20 24 0 08 Sep 2021
Do Prompt-Based Models Really Understand the Meaning of their Prompts? Albert Webson Ellie Pavlick LRM 30 351 0 02 Sep 2021
Local Structure Matters Most: Perturbation Study in NLU Louis Clouâtre Prasanna Parthasarathi Amal Zouaq Sarath Chandar 17 13 0 29 Jul 2021
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little Koustuv Sinha Robin Jia Dieuwke Hupkes J. Pineau Adina Williams Douwe Kiela 34 243 0 14 Apr 2021
Diagnosing Vision-and-Language Navigation: What Really Matters Wanrong Zhu Yuankai Qi P. Narayana Kazoo Sone Sugato Basu X. Wang Qi Wu M. Eckstein W. Wang LM&Ro 22 50 0 30 Mar 2021
Out of Order: How Important Is The Sequential Order of Words in a Sentence in Natural Language Understanding Tasks? Thang M. Pham Trung Bui Long Mai Anh Totti Nguyen 207 122 0 30 Dec 2020
Language GANs Falling Short Massimo Caccia Lucas Page-Caccia W. Fedus Hugo Larochelle Joelle Pineau Laurent Charlin 117 215 0 06 Nov 2018