Demystifying Prompts in Language Models via Perplexity Estimation

8 December 2022

Luke Zettlemoyer

Papers citing "Demystifying Prompts in Language Models via Perplexity Estimation"

41 / 141 papers shown

Title
Hint-enhanced In-Context Learning wakes Large Language Models up for knowledge-intensive tasks Yifan Wang Qingyan Guo Xinzhe Ni Chufan Shi Lemao Liu Haiyun Jiang Yujiu Yang ReLM RALM 17 8 0 03 Nov 2023
Improving Input-label Mapping with Demonstration Replay for In-context Learning Zhuocheng Gong Jiahao Liu Qifan Wang Jingang Wang Xunliang Cai Dongyan Zhao Rui Yan 16 1 0 30 Oct 2023
Unnatural language processing: How do language models handle machine-generated prompts? Corentin Kervadec Francesca Franzon Marco Baroni 10 5 0 24 Oct 2023
Bridging Information-Theoretic and Geometric Compression in Language Models Emily Cheng Corentin Kervadec Marco Baroni 16 16 0 20 Oct 2023
Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting Melanie Sclar Yejin Choi Yulia Tsvetkov Alane Suhr 19 295 0 17 Oct 2023
Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learning Zhe Yang Damai Dai Peiyi Wang Zhifang Sui 34 8 0 12 Oct 2023
Dynamic Demonstrations Controller for In-Context Learning Fei Zhao Taotian Pang Zhen Wu Zheng Ma Shujian Huang Xinyu Dai 27 5 0 30 Sep 2023
Leveraging Medical Knowledge Graphs Into Large Language Models for Diagnosis Prediction: Design and Application Study Yanjun Gao Ruizhe Li John R. Caskey Dmitriy Dligach T. Miller M. Churpek Majid Afshar Dmitriy Dligach Majid Afshar LM&MA 14 5 0 28 Aug 2023
Metric-Based In-context Learning: A Case Study in Text Simplification Subhadra Vadlamannati Gözde Gül Sahin 6 1 0 27 Jul 2023
Performance of Large Language Models in a Computer Science Degree Program Tim Krüger Michael Gref ELM ALM 13 8 0 24 Jul 2023
In-Context Learning Learns Label Relationships but Is Not Conventional Learning Jannik Kossen Y. Gal Tom Rainforth 22 27 0 23 Jul 2023
InstructEval: Systematic Evaluation of Instruction Selection Methods Anirudh Ajith Chris Pan Mengzhou Xia A. Deshpande Karthik Narasimhan ELM 17 16 0 01 Jul 2023
Pushing the Limits of ChatGPT on NLP Tasks Xiaofei Sun Linfeng Dong Xiaoya Li Zhen Wan Shuhe Wang ... Jiwei Li Fei Cheng Lingjuan Lyu Fei Wu Guoyin Wang AI4MH LRM 27 29 0 16 Jun 2023
Language models are not naysayers: An analysis of language models on negation benchmarks Thinh Hung Truong Timothy Baldwin Karin Verspoor Trevor Cohn 14 53 0 14 Jun 2023
Beyond Confidence: Reliable Models Should Also Consider Atypicality Mert Yuksekgonul Linjun Zhang James Y. Zou Carlos Guestrin 19 20 0 29 May 2023
Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrations Wei-Lin Chen Cheng-Kuang Wu Yun-Nung Chen Hsin-Hsi Chen 8 27 0 24 May 2023
Frugal Prompting for Dialog Models Bishal Santra Sakya Basak Abhinandan De Manish Gupta Pawan Goyal 20 2 0 24 May 2023
Improving Probability-based Prompt Selection Through Unified Evaluation and Analysis Sohee Yang Jonghyeon Kim Joel Jang Seonghyeon Ye Hyunji Lee Minjoon Seo 17 7 0 24 May 2023
In-Context Demonstration Selection with Cross Entropy Difference Dan Iter Reid Pryzant Ruochen Xu Shuohang Wang Yang Liu Yichong Xu Chenguang Zhu 18 9 0 24 May 2023
Evaluation of African American Language Bias in Natural Language Generation Nicholas Deas Jessica A. Grieser Shana Kleiner D. Patton Elsbeth Turcan Kathleen McKeown 6 14 0 23 May 2023
Active Learning Principles for In-Context Learning with Large Language Models Katerina Margatina Timo Schick Nikolaos Aletras Jane Dwivedi-Yu 20 39 0 23 May 2023
Dr.ICL: Demonstration-Retrieved In-context Learning Man Luo Xin Xu Zhuyun Dai Panupong Pasupat Mehran Kazemi Chitta Baral Vaiva Imbrasaite Vincent Zhao RALM 21 48 0 23 May 2023
CTQScorer: Combining Multiple Features for In-context Example Selection for Machine Translation Aswanth Kumar Ratish Puduppully Raj Dabre Anoop Kunchukuttan 28 12 0 23 May 2023
Enhancing Few-shot Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies Linyong Nan Yilun Zhao Weijin Zou Narutatsu Ri Jaesung Tae Ellen Zhang Arman Cohan Dragomir R. Radev LMTD 12 45 0 21 May 2023
Large Language Models can be Guided to Evade AI-Generated Text Detection Ning Lu Shengcai Liu Ruidan He Qi Wang Yew-Soon Ong Ke Tang SILM 20 42 0 18 May 2023
SatLM: Satisfiability-Aided Language Models Using Declarative Prompting Xi Ye Qiaochu Chen Işıl Dillig Greg Durrett ReLM ReCod LRM 31 61 0 16 May 2023
In-context Learning as Maintaining Coherency: A Study of On-the-fly Machine Translation Using Large Language Models Suzanna Sia Kevin Duh LRM 13 17 0 05 May 2023
Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4 Kent K. Chang Mackenzie Cramer Sandeep Soni David Bamman RALM 138 109 0 28 Apr 2023
Large Language Model Instruction Following: A Survey of Progresses and Challenges Renze Lou Kai Zhang Wenpeng Yin ALM LRM 19 19 0 18 Mar 2023
A Theory of Emergent In-Context Learning as Implicit Structure Induction Michael Hahn Navin Goyal LRM 8 73 0 14 Mar 2023
Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models Kaitlyn Zhou Dan Jurafsky Tatsunori Hashimoto 11 49 0 26 Feb 2023
In-context Example Selection with Influences Nguyen Tai Eric Wong 9 48 0 21 Feb 2023
Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting Xi Ye Greg Durrett LRM ReLM 10 10 0 09 Feb 2023
A Survey on In-context Learning Qingxiu Dong Lei Li Damai Dai Ce Zheng Jingyuan Ma ... Zhiyong Wu Baobao Chang Xu Sun Lei Li Zhifang Sui ReLM AIMat 20 443 0 31 Dec 2022
Toward Human Readable Prompt Tuning: Kubrick's The Shining is a good movie, and a good prompt too? Weijia Shi Xiaochuang Han Hila Gonen Ari Holtzman Yulia Tsvetkov Luke Zettlemoyer 21 41 0 20 Dec 2022
Data Curation Alone Can Stabilize In-context Learning Ting-Yun Chang Robin Jia 14 51 0 20 Dec 2022
Extensible Prompts for Language Models on Zero-shot Language Style Customization Tao Ge Jing Hu Li Dong Shaoguang Mao Yanqiu Xia Xun Wang Si-Qing Chen Furu Wei VLM 30 6 0 01 Dec 2022
On the Relation between Sensitivity and Accuracy in In-context Learning Yanda Chen Chen Zhao Zhou Yu Kathleen McKeown He He 180 77 0 16 Sep 2022
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts Stephen H. Bach Victor Sanh Zheng-Xin Yong Albert Webson Colin Raffel ... Khalid Almubarak Xiangru Tang Dragomir R. Radev Mike Tian-Jian Jiang Alexander M. Rush VLM 212 335 0 02 Feb 2022
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 278 3,784 0 18 Apr 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,898 0 31 Dec 2020