Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery

7 February 2023

Papers citing "Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery"

45 / 195 papers shown

Title
Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation Yangsibo Huang Samyak Gupta Mengzhou Xia Kai Li Danqi Chen AAML 19 264 0 10 Oct 2023
Teaching Language Models to Hallucinate Less with Synthetic Tasks Erik Jones Hamid Palangi Clarisse Simoes Varun Chandrasekaran Subhabrata Mukherjee Arindam Mitra Ahmed Hassan Awadallah Ece Kamar HILM 13 23 0 10 Oct 2023
Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations Zeming Wei Yifei Wang Ang Li Yichuan Mo Yisen Wang 40 233 0 10 Oct 2023
Understanding prompt engineering may not require rethinking generalization Victor Akinwande Yiding Jiang Dylan Sam J. Zico Kolter VLM VPVLM 112 7 0 06 Oct 2023
Navigating Cultural Chasms: Exploring and Unlocking the Cultural POV of Text-To-Image Models Mor Ventura Eyal Ben-David Anna Korhonen Roi Reichart 18 12 0 03 Oct 2023
Zero-Shot Continuous Prompt Transfer: Generalizing Task Semantics Across Language Models Zijun Wu Yongkang Wu Lili Mou VLM 14 2 0 02 Oct 2023
What's the Magic Word? A Control Theory of LLM Prompting Aman Bhargava Cameron Witkowski Manav Shah Matt W. Thomson LLMAG 45 29 0 02 Oct 2023
Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts Zhi-Yi Chin Chieh-Ming Jiang Ching-Chun Huang Pin-Yu Chen Wei-Chen Chiu DiffM 11 65 0 12 Sep 2023
Large Language Models as Optimizers Chengrun Yang Xuezhi Wang Yifeng Lu Hanxiao Liu Quoc V. Le Denny Zhou Xinyun Chen ODL 19 369 0 07 Sep 2023
Baseline Defenses for Adversarial Attacks Against Aligned Language Models Neel Jain Avi Schwarzschild Yuxin Wen Gowthami Somepalli John Kirchenbauer Ping Yeh-Chiang Micah Goldblum Aniruddha Saha Jonas Geiping Tom Goldstein AAML 8 335 0 01 Sep 2023
Manipulating Embeddings of Stable Diffusion Prompts Niklas Deckers Julia Peters Martin Potthast DiffM 27 9 0 23 Aug 2023
Reverse Stable Diffusion: What prompt was used to generate this image? Florinel-Alin Croitoru Vlad Hondru Radu Tudor Ionescu M. Shah VLM DiffM 23 5 0 02 Aug 2023
ImageBrush: Learning Visual In-Context Instructions for Exemplar-Based Image Manipulation Yasheng Sun Yifan Yang Houwen Peng Yifei Shen Yuqing Yang Hang-Rui Hu Lili Qiu Hideki Koike DiffM LM&Ro 27 33 0 02 Aug 2023
Guiding Image Captioning Models Toward More Specific Captions Simon Kornblith Lala Li Zirui Wang Thao Nguyen 19 15 0 31 Jul 2023
Universal and Transferable Adversarial Attacks on Aligned Language Models Andy Zou Zifan Wang Nicholas Carlini Milad Nasr J. Zico Kolter Matt Fredrikson 16 1,236 0 27 Jul 2023
Visual Instruction Inversion: Image Editing via Visual Prompting Thao Nguyen Yuheng Li Utkarsh Ojha Yong Jae Lee DiffM 21 22 0 26 Jul 2023
AdjointDPM: Adjoint Sensitivity Method for Gradient Backpropagation of Diffusion Probabilistic Models Jiachun Pan Jun Hao Liew Vincent Y. F. Tan Jiashi Feng Hanshu Yan DiffM 16 9 0 20 Jul 2023
Soft Prompt Tuning for Augmenting Dense Retrieval with Large Language Models Zhiyuan Peng Xuyang Wu Qifan Wang Yihan Fang VLM RALM 38 11 0 17 Jul 2023
Personalization for BERT-based Discriminative Speech Recognition Rescoring J. Kolehmainen Yile Gu Aditya Gourav Prashanth Gurunath Shivakumar Ankur Gandhe Ariya Rastrow I. Bulyko 25 2 0 13 Jul 2023
Text Descriptions are Compressive and Invariant Representations for Visual Learning Zhili Feng Anna Bair J. Zico Kolter VLM 14 6 0 10 Jul 2023
Text + Sketch: Image Compression at Ultra Low Rates E. Lei Yiugit Berkay Uslu Hamed Hassani Shirin Saeedi Bidokhti DiffM 23 36 0 04 Jul 2023
Mass-Producing Failures of Multimodal Systems with Language Models Shengbang Tong Erik Jones Jacob Steinhardt 22 33 0 21 Jun 2023
Explore, Establish, Exploit: Red Teaming Language Models from Scratch Stephen Casper Jason Lin Joe Kwon Gatlen Culp Dylan Hadfield-Menell AAML 8 83 0 15 Jun 2023
User-friendly Image Editing with Minimal Text Input: Leveraging Captioning and Injection Techniques Sunwoo Kim Wooseok Jang Hyunsung Kim Junho Kim Yunjey Choi Seung Wook Kim Gayeong Lee DiffM 28 6 0 05 Jun 2023
StyleDrop: Text-to-Image Generation in Any Style Kihyuk Sohn Nataniel Ruiz Kimin Lee Daniel Castro Chin Irina Blok ... Yuanzhen Li Yuan Hao Irfan Essa Michael Rubinstein Dilip Krishnan 4 141 0 01 Jun 2023
Discovering Failure Modes of Text-guided Diffusion Models via Adversarial Search Qihao Liu Adam Kortylewski Yutong Bai Song Bai Alan Yuille DiffM 28 12 0 01 Jun 2023
The Hidden Language of Diffusion Models Hila Chefer Oran Lang Mor Geva Volodymyr Polosukhin Assaf Shocher Michal Irani Inbar Mosseri Lior Wolf DiffM 20 26 0 01 Jun 2023
Understanding and Mitigating Copying in Diffusion Models Gowthami Somepalli Vasu Singla Micah Goldblum Jonas Geiping Tom Goldstein DiffM 16 124 0 31 May 2023
Submodular Minimax Optimization: Finding Effective Sets Loay Mualem Ethan R. Elenberg Moran Feldman Amin Karbasi 14 5 0 26 May 2023
HUB: Guiding Learned Optimizers with Continuous Prompt Tuning Gaole Dai Wei Yu Wu Ziyu Wang Jie Fu Shanghang Zhang Tiejun Huang AIFin 9 0 0 26 May 2023
ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Models Yu-xin Zhang Weiming Dong Fan Tang Nisha Huang Haibin Huang Chongyang Ma Tong-Yee Lee Oliver Deussen Changsheng Xu DiffM 25 75 0 25 May 2023
Universal Self-Adaptive Prompting Xingchen Wan Ruoxi Sun Hootan Nakhost H. Dai Julian Martin Eisenschlos Sercan Ö. Arik Tomas Pfister LRM 31 9 0 24 May 2023
Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language Models Stephen Brade Bryan Wang Maurício Sousa Sageev Oore Tovi Grossman MLLM DiffM 13 81 0 18 Apr 2023
Expressive Text-to-Image Generation with Rich Text Songwei Ge Taesung Park Jun-Yan Zhu Jia-Bin Huang DiffM 77 79 0 13 Apr 2023
A Pilot Study of Query-Free Adversarial Attack against Stable Diffusion Haomin Zhuang Yihua Zhang Sijia Liu DiffM 18 60 0 29 Mar 2023
Zeroth-Order Optimization Meets Human Feedback: Provable Learning via Ranking Oracles Zhiwei Tang Dmitry Rybin Tsung-Hui Chang ALM DiffM 31 25 0 07 Mar 2023
Collage Diffusion Vishnu Sarukkai Linden Li Arden Ma Christopher Ré Kayvon Fatahalian DiffM 10 23 0 01 Mar 2023
Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models Shawn Shan Jenna Cryan Emily Wenger Haitao Zheng Rana Hanocka Ben Y. Zhao WIGM 12 173 0 08 Feb 2023
Discovering and Mitigating Visual Biases through Keyword Explanation Younghyun Kim Sangwoo Mo Minkyu Kim Kyungmin Lee Jaeho Lee Jinwoo Shin 26 30 0 26 Jan 2023
Red-Teaming the Stable Diffusion Safety Filter Javier Rando Daniel Paleka David Lindner Lennard Heim Florian Tramèr DiffM 122 179 0 03 Oct 2022
Discovering the Hidden Vocabulary of DALLE-2 Giannis Daras A. Dimakis 115 64 0 01 Jun 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 385 4,010 0 28 Jan 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 203 1,651 0 15 Oct 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 278 3,784 0 18 Apr 2021
Gradient-based Adversarial Attacks against Text Transformers Chuan Guo Alexandre Sablayrolles Hervé Jégou Douwe Kiela SILM 98 225 0 15 Apr 2021