SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models

We introduce SemEval-2025 Task 4: unlearning sensitive content from Large Language Models (LLMs). The task features 3 subtasks for LLM unlearning spanning different use cases: (1) unlearn long form synthetic creative documents spanning different genres; (2) unlearn short form synthetic biographies containing personally identifiable information (PII), including fake names, phone number, SSN, email and home addresses, and (3) unlearn real documents sampled from the target model's training dataset. We received over 100 submissions from over 30 institutions and we summarize the key techniques and lessons in this paper.

@article{ramakrishna2025_2504.02883,
  title={ SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models },
  author={ Anil Ramakrishna and Yixin Wan and Xiaomeng Jin and Kai-Wei Chang and Zhiqi Bu and Bhanukiran Vinzamuri and Volkan Cevher and Mingyi Hong and Rahul Gupta },
  journal={arXiv preprint arXiv:2504.02883},
  year={ 2025 }
}