Blind Baselines Beat Membership Inference Attacks for Foundation Models

23 June 2024

Florian Tramèr

Papers citing "Blind Baselines Beat Membership Inference Attacks for Foundation Models"

27 / 27 papers shown

Title
Revisiting Data Auditing in Large Vision-Language Models Hongyu Zhu Sichu Liang W. Wang Boheng Li Tongxin Yuan Fangqi Li Shilin Wang Zhuosheng Zhang VLM 88 0 0 25 Apr 2025
Beyond Public Access in LLM Pre-Training Data Sruly Rosenblat Tim O'Reilly Ilan Strauss MLAU 53 0 0 24 Apr 2025
STAMP Your Content: Proving Dataset Membership via Watermarked Rephrasings Saksham Rastogi Pratyush Maini Danish Pruthi 40 0 0 18 Apr 2025
Language Models May Verbatim Complete Text They Were Not Explicitly Trained On Ken Ziyu Liu Christopher A. Choquette-Choo Matthew Jagielski Peter Kairouz Sanmi Koyejo Percy Liang Nicolas Papernot 49 0 0 21 Mar 2025
Privacy Auditing of Large Language Models Ashwinee Panda Xinyu Tang Milad Nasr Christopher A. Choquette-Choo Prateek Mittal PILM 62 5 0 09 Mar 2025
Adversarial ML Problems Are Getting Harder to Solve and to Evaluate Javier Rando Jie Zhang Nicholas Carlini F. Tramèr AAML ELM 52 3 0 04 Feb 2025
Riddle Me This! Stealthy Membership Inference for Retrieval-Augmented Generation A. Naseh Yuefeng Peng Anshuman Suri Harsh Chaudhari Alina Oprea Amir Houmansadr SILM AAML RALM 49 0 0 01 Feb 2025
Synthetic Data Can Mislead Evaluations: Membership Inference as Machine Text Detection Ali Naseh Niloofar Mireshghallah 49 0 0 20 Jan 2025
A Statistical and Multi-Perspective Revisiting of the Membership Inference Attack in Large Language Models Bowen Chen Namgi Han Yusuke Miyao 93 0 0 18 Dec 2024
Membership Inference Attacks against Large Vision-Language Models Zhan Li Yongtao Wu Yihang Chen F. Tonin Elias Abad Rocamora V. Cevher 39 4 0 05 Nov 2024
Detecting Training Data of Large Language Models via Expectation Maximization Gyuwan Kim Yang Li Evangelia Spiliopoulou Jie Ma Miguel Ballesteros William Yang Wang MIALM 90 3 2 10 Oct 2024
Fine-tuning can Help Detect Pretraining Data from Large Language Models H. Zhang Songxin Zhang Bingyi Jing Hongxin Wei 34 0 0 09 Oct 2024
Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Models Theo Putterman Derek Lim Yoav Gelberg Stefanie Jegelka Haggai Maron AI4CE 43 5 0 05 Oct 2024
Real-World Benchmarks Make Membership Inference Attacks Fail on Diffusion Models Chumeng Liang Jiaxuan You 31 0 0 04 Oct 2024
Ward: Provable RAG Dataset Inference via LLM Watermarks Nikola Jovanović Robin Staab Maximilian Baader Martin Vechev 45 1 0 04 Oct 2024
Membership Inference Attacks Cannot Prove that a Model Was Trained On Your Data Jie Zhang Debeshee Das Gautam Kamath Florian Tramèr MIALM MIACV 215 16 1 29 Sep 2024
Context-Aware Membership Inference Attacks against Pre-trained Large Language Models Hongyan Chang Ali Shahin Shamsabadi Kleomenis Katevas Hamed Haddadi Reza Shokri MIALM 53 6 0 11 Sep 2024
Nob-MIAs: Non-biased Membership Inference Attacks Assessment on Large Language Models with Ex-Post Dataset Construction Cédric Eichler Nathan Champeil Nicolas Anciaux Alexandra Bensamoun Héber H. Arcolezi José Maria De Fuentes 38 4 0 12 Aug 2024
ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods Roy Xie Junlin Wang Ruomin Huang Minxing Zhang Rong Ge Jian Pei Neil Zhenqiang Gong Bhuwan Dhingra MIALM 37 11 0 23 Jun 2024
DataComp-LM: In search of the next generation of training sets for language models Jeffrey Li Alex Fang Georgios Smyrnis Maor Ivgi Matt Jordan ... Alexandros G. Dimakis Y. Carmon Achal Dave Ludwig Schmidt Vaishaal Shankar ELM 29 79 0 17 Jun 2024
Membership Inference on Text-to-Image Diffusion Models via Conditional Likelihood Discrepancy Shengfang Zhai Huanran Chen Yinpeng Dong Jiajun Li Qingni Shen Yansong Gao Hang Su Yang Liu EGVM 46 9 0 23 May 2024
Rethinking LLM Memorization through the Lens of Adversarial Compression Avi Schwarzschild Zhili Feng Pratyush Maini Zachary Chase Lipton J. Zico Kolter 39 38 0 23 Apr 2024
PANORAMIA: Privacy Auditing of Machine Learning Models without Retraining Mishaal Kazmi H. Lautraite Alireza Akbari Mauricio Soroco Qiaoyue Tang Tao Wang Sébastien Gambs Mathias Lécuyer 24 8 0 12 Feb 2024
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 253 4,735 0 24 Feb 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 273 1,077 0 17 Feb 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 245 1,977 0 31 Dec 2020
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 267 1,798 0 14 Dec 2020