Do Prompt-Based Models Really Understand the Meaning of their Prompts?

2 September 2021

Papers citing "Do Prompt-Based Models Really Understand the Meaning of their Prompts?"

50 / 254 papers shown

Title
What's the Difference? Supporting Users in Identifying the Effects of Prompt and Model Changes Through Token Patterns Michael A. Hedderich Anyi Wang Raoyuan Zhao Florian Eichin Barbara Plank 30 0 0 22 Apr 2025
DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation Eliya Habba Ofir Arviv Itay Itzhak Yotam Perlitz Elron Bandel Leshem Choshen Michal Shmueli-Scheuer Gabriel Stanovsky 64 1 0 03 Mar 2025
ANPMI: Assessing the True Comprehension Capabilities of LLMs for Multiple Choice Questions Gyeongje Cho Yeonkyoung So Jaejin Lee ELM 59 0 0 26 Feb 2025
Better Aligned with Survey Respondents or Training Data? Unveiling Political Leanings of LLMs on U.S. Supreme Court Cases Shanshan Xu T. Y. S. S. Santosh Yanai Elazar Quirin Vogel Barbara Plank Matthias Grabmair AILaw 78 0 0 25 Feb 2025
In-context learning of evolving data streams with tabular foundational models Afonso Lourenço João Gama Eric P. Xing Goreti Marreiros 56 0 0 24 Feb 2025
Vector-ICL: In-context Learning with Continuous Vector Representations Yufan Zhuang Chandan Singh Liyuan Liu Jingbo Shang Jianfeng Gao 52 3 0 21 Feb 2025
A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics Kai He Rui Mao Qika Lin Yucheng Ruan Xiang Lan Mengling Feng Erik Cambria LM&MA AILaw 85 148 0 28 Jan 2025
Zero-shot and Few-shot Learning with Instruction-following LLMs for Claim Matching in Automated Fact-checking Dina Pisarevskaya Arkaitz Zubiaga 48 0 0 18 Jan 2025
Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation H. Yang Qianghua Zhao Lei Li AI4CE LRM 66 1 0 05 Dec 2024
Enhancing Clinical Trial Patient Matching through Knowledge Augmentation with Multi-Agents Hanwen Shi Jin Zhang Kunpeng Zhang 76 0 0 22 Nov 2024
Belief in the Machine: Investigating Epistemological Blind Spots of Language Models Mirac Suzgun Tayfun Gur Federico Bianchi Daniel E. Ho Thomas F. Icard Dan Jurafsky James Zou 29 1 0 28 Oct 2024
Reasoning or a Semblance of it? A Diagnostic Study of Transitive Reasoning in LLMs Houman Mehrafarin Arash Eshghi Ioannis Konstas LRM 18 0 0 26 Oct 2024
Causality for Large Language Models Anpeng Wu Kun Kuang Minqin Zhu Yingrong Wang Yujia Zheng Kairong Han B. Li Guangyi Chen Fei Wu Kun Zhang LRM 44 6 0 20 Oct 2024
Are LLMs Models of Distributional Semantics? A Case Study on Quantifiers Zhang Enyan Zewei Wang Michael A. Lepori Ellie Pavlick Helena Aparicio 11 1 0 17 Oct 2024
One Step at a Time: Combining LLMs and Static Analysis to Generate Next-Step Hints for Programming Tasks Anastasiia Birillo Elizaveta Artser Anna Potriasaeva Ilya Vlasov Katsiaryna Dzialets Yaroslav Golubev Igor Gerasimov Hieke Keuning T. Bryksin 36 3 0 11 Oct 2024
Towards a Benchmark for Large Language Models for Business Process Management Tasks Kiran Busch Henrik Leopold 43 1 0 04 Oct 2024
HUT: A More Computation Efficient Fine-Tuning Method With Hadamard Updated Transformation Geyuan Zhang Xiaofei Zhou Chuheng Chen 14 0 0 20 Sep 2024
The representation landscape of few-shot learning and fine-tuning in large language models Diego Doimo Alessandro Serra A. Ansuini Alberto Cazzaniga 88 4 0 05 Sep 2024
Self-Instructed Derived Prompt Generation Meets In-Context Learning: Unlocking New Potential of Black-Box LLMs Zhuo Li Yuhao Du Jinpeng Hu Xiang Wan Anningzhe Gao 22 2 0 03 Sep 2024
Machine Unlearning in Generative AI: A Survey Zheyuan Liu Guangyao Dou Zhaoxuan Tan Yijun Tian Meng-Long Jiang MU 29 13 0 30 Jul 2024
GPT Assisted Annotation of Rhetorical and Linguistic Features for Interpretable Propaganda Technique Detection in News Text Kyle Hamilton Luca Longo Bojan Bozic 26 1 0 16 Jul 2024
Open (Clinical) LLMs are Sensitive to Instruction Phrasings Alberto Mario Ceballos Arroyo Monica Munnangi Jiuding Sun Karen Y.C. Zhang Denis Jered McInerney Byron C. Wallace Silvio Amir LM&MA 18 8 0 12 Jul 2024
Enhancing the Capability and Robustness of Large Language Models through Reinforcement Learning-Driven Query Refinement Zisu Huang Xiaohua Wang Feiran Zhang Zhibo Xu Cenyuan Zhang Xiaoqing Zheng Xuanjing Huang AAML LRM 24 4 0 01 Jul 2024
PrExMe! Large Scale Prompt Exploration of Open Source LLMs for Machine Translation and Summarization Evaluation Christoph Leiter Steffen Eger 27 7 0 26 Jun 2024
Benchmarking Mental State Representations in Language Models Matteo Bortoletto Constantin Ruhdorfer Lei Shi Andreas Bulling AI4MH LRM 36 4 0 25 Jun 2024
It Is Not About What You Say, It Is About How You Say It: A Surprisingly Simple Approach for Improving Reading Comprehension Sagi Shaier Lawrence E Hunter K. Wense 39 3 0 24 Jun 2024
APEER: Automatic Prompt Engineering Enhances Large Language Model Reranking Can Jin Hongwu Peng Shiyu Zhao Zhenting Wang Wujiang Xu Ligong Han Jiahui Zhao Kai Zhong Sanguthevar Rajasekaran Dimitris N. Metaxas KELM 40 31 0 20 Jun 2024
Text Serialization and Their Relationship with the Conventional Paradigms of Tabular Machine Learning Kyoka Ono Simon A. Lee LMTD 22 7 0 19 Jun 2024
A Personalised Learning Tool for Physics Undergraduate Students Built On a Large Language Model for Symbolic Regression Yufan Zhu Zi-Yu Khoo Jonathan Sze Choong Low Stephane Bressan AI4Ed 25 1 0 17 Jun 2024
Probing the Decision Boundaries of In-context Learning in Large Language Models Siyan Zhao Tung Nguyen Aditya Grover 32 6 0 17 Jun 2024
Effective Generative AI: The Human-Algorithm Centaur S. Saghafian Lihi Idan 38 7 0 16 Jun 2024
Bayesian Statistical Modeling with Predictors from LLMs Michael Franke Polina Tsvilodub Fausto Carcassi 29 4 0 13 Jun 2024
Talking Heads: Understanding Inter-layer Communication in Transformer Language Models Jack Merullo Carsten Eickhoff Ellie Pavlick 51 12 0 13 Jun 2024
Do Prompts Really Prompt? Exploring the Prompt Understanding Capability of Whisper Chih-Kai Yang Kuan Po Huang Hung-yi Lee 40 3 0 09 Jun 2024
Are LLMs classical or nonmonotonic reasoners? Lessons from generics Alina Leidinger R. Rooij Ekaterina Shutova LRM 26 3 0 05 Jun 2024
FreezeAsGuard: Mitigating Illegal Adaptation of Diffusion Models via Selective Tensor Freezing Kai Huang Wei Gao 19 2 0 24 May 2024
From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks Jacob Russin Sam Whitman McGrath Danielle J. Williams Lotem Elber-Dorozko AI4CE 59 2 0 24 May 2024
Lessons from the Trenches on Reproducible Evaluation of Language Models Stella Biderman Hailey Schoelkopf Lintang Sutawika Leo Gao J. Tow ... Xiangru Tang Kevin A. Wang Genta Indra Winata Franccois Yvon Andy Zou ELM ALM 120 16 3 23 May 2024
CIVICS: Building a Dataset for Examining Culturally-Informed Values in Large Language Models Giada Pistilli Alina Leidinger Yacine Jernite Atoosa Kasirzadeh A. Luccioni Margaret Mitchell 21 2 0 22 May 2024
Exploring Ordinality in Text Classification: A Comparative Study of Explicit and Implicit Techniques Siva Rajesh Kasa Aniket Goel Karan Gupta Sumegh Roychowdhury Anish Bhanushali Nikhil Pattisapu Prasanna Srinivasa Murthy 28 1 0 20 May 2024
MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generation Dongjun Lee Choongwon Park Jaehyuk Kim Heesoo Park 37 37 0 13 May 2024
ACORN: Aspect-wise Commonsense Reasoning Explanation Evaluation Ana Brassard Benjamin Heinzerling Keito Kudo Keisuke Sakaguchi Kentaro Inui LRM 37 0 0 08 May 2024
Learning To See But Forgetting To Follow: Visual Instruction Tuning Makes LLMs More Prone To Jailbreak Attacks Georgios Pantazopoulos Amit Parekh Malvina Nikandrou Alessandro Suglia 21 5 0 07 May 2024
A Human-Computer Collaborative Tool for Training a Single Large Language Model Agent into a Network through Few Examples Lihang Pan Yuxuan Li Chun Yu Yuanchun Shi LLMAG 27 1 0 24 Apr 2024
What Makes Multimodal In-Context Learning Work? Folco Bertini Baldassini Mustafa Shukor Matthieu Cord Laure Soulier Benjamin Piwowarski 32 18 0 24 Apr 2024
NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surrounding Chunkit Chan Cheng Jiayang Yauwai Yim Zheye Deng Wei Fan Haoran Li Xin Liu Hongming Zhang Weiqi Wang Yangqiu Song LLMAG 27 19 0 21 Apr 2024
Reinforcement of Explainability of ChatGPT Prompts by Embedding Breast Cancer Self-Screening Rules into AI Responses Yousef Khan Ahmed Abdeen Hamed 25 2 0 21 Apr 2024
CAUS: A Dataset for Question Generation based on Human Cognition Leveraging Large Language Models Minjung Shin Donghyun Kim Jeh-Kwang Ryu LRM 14 1 0 18 Apr 2024
Epigraphics: Message-Driven Infographics Authoring Tongyu Zhou Jeff Huang G. Chan 27 8 0 15 Apr 2024
LLMs in Biomedicine: A study on clinical Named Entity Recognition Masoud Monajatipoor Jiaxin Yang Joel Stremmel Melika Emami Fazlolah Mohaghegh Mozhdeh Rouhsedaghat Kai-Wei Chang LM&MA 24 5 0 10 Apr 2024