Prompt Consistency for Zero-Shot Task Generalization

29 April 2022

Chunting Zhou

Junxian He

Xuezhe Ma

Taylor Berg-Kirkpatrick

Graham Neubig

VLM

ArXiv PDF HTML

Papers citing "Prompt Consistency for Zero-Shot Task Generalization"

50 / 60 papers shown

Title
REFINE-AF: A Task-Agnostic Framework to Align Language Models via Self-Generated Instructions using Reinforcement Learning from Automated Feedback Aniruddha Roy Pretam Ray Abhilash Nandy Somak Aditya Pawan Goyal ALM 29 0 0 10 May 2025
reWordBench: Benchmarking and Improving the Robustness of Reward Models with Transformed Inputs Zhaofeng Wu Michihiro Yasunaga Andrew Cohen Yoon Kim Asli Celikyilmaz Marjan Ghazvininejad 38 1 0 14 Mar 2025
The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems Richard Ren Arunim Agarwal Mantas Mazeika Cristina Menghini Robert Vacareanu ... Matias Geralnik Adam Khoja Dean Lee Summer Yue Dan Hendrycks HILM ALM 88 0 0 05 Mar 2025
Same Question, Different Words: A Latent Adversarial Framework for Prompt Robustness Tingchen Fu Fazl Barez AAML 62 0 0 03 Mar 2025
Improving Consistency in Large Language Models through Chain of Guidance Harsh Raj Vipul Gupta Domenic Rosati Subhabrata Majumdar LLMAG LRM 63 3 0 21 Feb 2025
Enhancing Semantic Consistency of Large Language Models through Model Editing: An Interpretability-Oriented Approach J. Yang Dapeng Chen Yajing Sun Rongjun Li Zhiyong Feng Wei Peng 51 5 0 19 Jan 2025
On the Consistency of Video Large Language Models in Temporal Comprehension Minjoon Jung Junbin Xiao Byoung-Tak Zhang Angela Yao 87 2 0 20 Nov 2024
A Survey on the Honesty of Large Language Models Siheng Li Cheng Yang Taiqiang Wu Chufan Shi Yuji Zhang ... Jie Zhou Yujiu Yang Ngai Wong Xixin Wu Wai Lam HILM 32 4 0 27 Sep 2024
Latent Space Interpretation for Stylistic Analysis and Explainable Authorship Attribution Milad Alshomary Narutatsu Ri Marianna Apidianaki Ajay Patel Smaranda Muresan Kathleen McKeown 28 0 0 11 Sep 2024
Explicit Inductive Inference using Large Language Models Tianyang Liu Tianyi Li Liang Cheng Mark Steedman 28 0 0 26 Aug 2024
A Novel Metric for Measuring the Robustness of Large Language Models in Non-adversarial Scenarios Samuel Ackerman Ella Rabinovich E. Farchi Ateret Anaby-Tavor 28 1 0 04 Aug 2024
Application of Prompt Learning Models in Identifying the Collaborative Problem Solving Skills in an Online Task Mengxiao Zhu Xin Wang Xiantao Wang Zihang Chen Wei Huang 25 1 0 17 Jul 2024
Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generation Xinglin Wang Yiwei Li Shaoxiong Feng Peiwen Yuan Boyuan Pan Heda Wang Yao Hu Kan Li 28 10 0 02 Jul 2024
Leveraging Machine-Generated Rationales to Facilitate Social Meaning Detection in Conversations Ritam Dutt Zhen Wu Kelly Shi Divyanshu Sheth Prakhar Gupta Carolyn Rose 38 2 0 27 Jun 2024
On the Worst Prompt Performance of Large Language Models Bowen Cao Deng Cai Zhisong Zhang Yuexian Zou Wai Lam ALM LRM 30 5 0 08 Jun 2024
SQL-to-Schema Enhances Schema Linking in Text-to-SQL Sun Yang Qiong Su Zhishuai Li Ziyue Li Hangyu Mao Chenxi Liu Rui Zhao 21 10 0 15 May 2024
Deconstructing In-Context Learning: Understanding Prompts via Corruption Namrata Shivagunde Vladislav Lialin Sherin Muckatira Anna Rumshisky 36 2 0 02 Apr 2024
PURPLE: Making a Large Language Model a Better SQL Writer Tonghui Ren Yuankai Fan Zhenying He Ren Huang Jiaqi Dai Can Huang Yinan Jing Kai Zhang Yifan Yang Xiaoyang Sean Wang 23 21 0 29 Mar 2024
Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought James Chua Edward Rees Hunar Batra Samuel R. Bowman Julian Michael Ethan Perez Miles Turpin LRM 39 13 0 08 Mar 2024
diff History for Neural Language Agents Ulyana Piterbarg Lerrel Pinto Rob Fergus 27 3 0 12 Dec 2023
How Many Validation Labels Do You Need? Exploring the Design Space of Label-Efficient Model Ranking Zhengyu Hu Jieyu Zhang Yue Yu Yuchen Zhuang Hui Xiong 26 5 0 04 Dec 2023
Predicting Question-Answering Performance of Large Language Models through Semantic Consistency Ella Rabinovich Samuel Ackerman Orna Raz E. Farchi Ateret Anaby-Tavor 218 17 0 02 Nov 2023
Open Visual Knowledge Extraction via Relation-Oriented Multimodality Model Prompting Hejie Cui Xinyu Fang Zihan Zhang Ran Xu Xuan Kan Xin Liu Yue Yu Manling Li Yangqiu Song Carl Yang VLM 22 4 0 28 Oct 2023
Improving Language Models Meaning Understanding and Consistency by Learning Conceptual Roles from Dictionary Myeongjun Jang Thomas Lukasiewicz 22 4 0 24 Oct 2023
Test-Time Self-Adaptive Small Language Models for Question Answering Soyeong Jeong Jinheon Baek Sukmin Cho Sung Ju Hwang Jong C. Park 24 2 0 20 Oct 2023
Knowledge-Augmented Language Model Verification Jinheon Baek Soyeong Jeong Minki Kang Jong C. Park Sung Ju Hwang RALM 34 13 0 19 Oct 2023
Benchmarking and Improving Generator-Validator Consistency of Language Models Xiang Lisa Li Vaishnavi Shrivastava Siyan Li Tatsunori Hashimoto Percy Liang 14 27 0 03 Oct 2023
Prompting Segmentation with Sound Is Generalizable Audio-Visual Source Localizer Yaoting Wang Weisong Liu Guangyao Li Jian Ding Di Hu Xi Li VLM 13 18 0 13 Sep 2023
Semantic Consistency for Assuring Reliability of Large Language Models Harsh Raj Vipul Gupta Domenic Rosati S. Majumdar HILM 102 14 0 17 Aug 2023
Improving Generalization of Image Captioning with Unsupervised Prompt Learning Hongchen Wei Zhenzhong Chen VLM 31 3 0 05 Aug 2023
Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification Sha Li Ruining Zhao Manling Li Heng Ji Chris Callison-Burch Jiawei Han 41 29 0 05 Jul 2023
SummQA at MEDIQA-Chat 2023:In-Context Learning with GPT-4 for Medical Summarization Yash Mathur Sanketh Rangreji Raghav Kapoor Medha Palavalli Amanda Bertsch Matthew R. Gormley AI4MH 38 14 0 30 Jun 2023
Prompt to be Consistent is Better than Self-Consistent? Few-Shot and Zero-Shot Fact Verification with Pre-trained Language Models Fengzhu Zeng Wei Gao 17 5 0 05 Jun 2023
Two Failures of Self-Consistency in the Multi-Step Reasoning of LLMs Angelica Chen Jason Phang Alicia Parrish Vishakh Padmakumar Chen Zhao Sam Bowman Kyunghyun Cho ReLM LRM 25 28 0 23 May 2023
LM vs LM: Detecting Factual Errors via Cross Examination Roi Cohen May Hamri Mor Geva Amir Globerson HILM 32 117 0 22 May 2023
Distill or Annotate? Cost-Efficient Fine-Tuning of Compact Models Junmo Kang Wei-ping Xu Alan Ritter 44 15 0 02 May 2023
Improving Diffusion Models for Scene Text Editing with Dual Encoders Jiabao Ji Guanhua Zhang Zhaowen Wang Bairu Hou Zhifei Zhang Brian L. Price Shiyu Chang DiffM 32 29 0 12 Apr 2023
Self-Instruct: Aligning Language Models with Self-Generated Instructions Yizhong Wang Yeganeh Kordi Swaroop Mishra Alisa Liu Noah A. Smith Daniel Khashabi Hannaneh Hajishirzi ALM SyDa LRM 22 2,055 0 20 Dec 2022
One Embedder, Any Task: Instruction-Finetuned Text Embeddings Hongjin Su Weijia Shi Jungo Kasai Yizhong Wang Yushi Hu Mari Ostendorf Wen-tau Yih Noah A. Smith Luke Zettlemoyer Tao Yu 27 278 0 19 Dec 2022
Understanding Zero-Shot Adversarial Robustness for Large-Scale Models Chengzhi Mao Scott Geng Junfeng Yang Xin Eric Wang Carl Vondrick VLM 36 59 0 14 Dec 2022
Doubly Right Object Recognition: A Why Prompt for Visual Rationales Chengzhi Mao Revant Teotia Amrutha Sundar Sachit Menon Junfeng Yang Xin Eric Wang Carl Vondrick 15 29 0 12 Dec 2022
Discovering Latent Knowledge in Language Models Without Supervision Collin Burns Haotian Ye Dan Klein Jacob Steinhardt 55 322 0 07 Dec 2022
Measuring Reliability of Large Language Models through Semantic Consistency Harsh Raj Domenic Rosati S. Majumdar HILM 22 30 0 10 Nov 2022
Zero-Label Prompt Selection Chonghua Liao Yanan Zheng Zhilin Yang VLM 8 6 0 09 Nov 2022
Zero-Shot Text Classification with Self-Training Ariel Gera Alon Halfon Eyal Shnarch Yotam Perlitz L. Ein-Dor Noam Slonim VLM 28 59 0 31 Oct 2022
Don't Prompt, Search! Mining-based Zero-Shot Learning with Language Models Mozes van de Kar Mengzhou Xia Danqi Chen Mikel Artetxe 33 19 0 26 Oct 2022
Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalization Yuxian Gu Pei Ke Xiaoyan Zhu Minlie Huang ALM 31 17 0 17 Oct 2022
Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks Zhenhailong Wang Xiaoman Pan Dian Yu Dong Yu Jianshu Chen Heng Ji VLM 38 9 0 01 Oct 2022
On the Relation between Sensitivity and Accuracy in In-context Learning Yanda Chen Chen Zhao Zhou Yu Kathleen McKeown He He 182 77 0 16 Sep 2022
Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models Manli Shu Weili Nie De-An Huang Zhiding Yu Tom Goldstein Anima Anandkumar Chaowei Xiao VLM VPVLM 186 280 0 15 Sep 2022