ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting

ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting

28 June 2024

Jipeng Zhang

Xiaoyu Wang

Tong Zhang

Papers citing "ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting"

13 / 13 papers shown

Title
The Best Instruction-Tuning Data are Those That Fit Dylan Zhang Qirun Dai Hao Peng ALM 113 3 0 06 Feb 2025
Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models Yulei Qin Yuncheng Yang Pengcheng Guo Gang Li Hang Shao Yuchen Shi Zihan Xu Yun Gu Ke Li Xing Sun ALM 73 11 0 31 Dec 2024
Differentially Private Bilevel Optimization Guy Kornowski 41 0 0 29 Sep 2024
WildChat: 1M ChatGPT Interaction Logs in the Wild Wenting Zhao Xiang Ren Jack Hessel Claire Cardie Yejin Choi Yuntian Deng 40 171 0 02 May 2024
LESS: Selecting Influential Data for Targeted Instruction Tuning Mengzhou Xia Sadhika Malladi Suchin Gururangan Sanjeev Arora Danqi Chen 73 180 0 06 Feb 2024
The Falcon Series of Open Language Models Ebtesam Almazrouei Hamza Alobeidli Abdulaziz Alshamsi Alessandro Cappelli Ruxandra-Aimée Cojocaru ... Quentin Malartic Daniele Mazzotta Badreddine Noune B. Pannier Guilherme Penedo AI4TS ALM 104 389 0 28 Nov 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 154 576 0 06 Apr 2023
First-order penalty methods for bilevel optimization Zhaosong Lu Sanyou Mei 53 31 0 04 Jan 2023
BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach Mao Ye B. Liu S. Wright Peter Stone Qian Liu 69 82 0 19 Sep 2022
Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning Jiahui Gao Renjie Pi Yong Lin Hang Xu Jiacheng Ye Zhiyong Wu Weizhong Zhang Xiaodan Liang Zhenguo Li Lingpeng Kong SyDa VLM 49 45 0 25 May 2022
A framework for bilevel optimization that enables stochastic and global variance reduction algorithms Mathieu Dagréou Pierre Ablin Samuel Vaiter Thomas Moreau 126 73 0 31 Jan 2022
Bilevel Programming for Hyperparameter Optimization and Meta-Learning Luca Franceschi P. Frasconi Saverio Salzo Riccardo Grazzi Massimiliano Pontil 96 714 0 13 Jun 2018
Forward and Reverse Gradient-Based Hyperparameter Optimization Luca Franceschi Michele Donini P. Frasconi Massimiliano Pontil 109 370 0 06 Mar 2017