Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion
Models: A Tutorial and Review

Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Review

18 July 2024

Masatoshi Uehara

Tommaso Biancalani

Papers citing "Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Review"

18 / 18 papers shown

Title
Reward Fine-Tuning Two-Step Diffusion Models via Learning Differentiable Latent-Space Surrogate Reward Zhiwei Jia Yuesong Nan Huixi Zhao Gengdai Liu EGVM 84 0 0 22 Nov 2024
Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design Chenyu Wang Masatoshi Uehara Yichun He Amy Wang Tommaso Biancalani Avantika Lal Tommi Jaakkola Sergey Levine Hanchen Wang Aviv Regev 48 8 0 17 Oct 2024
Cliqueformer: Model-Based Optimization with Structured Transformers J. Kuba Pieter Abbeel Sergey Levine OffRL AI4CE 47 2 0 17 Oct 2024
Adding Conditional Control to Diffusion Models with Reinforcement Learning Yulai Zhao Masatoshi Uehara Gabriele Scalia Tommaso Biancalani Sergey Levine Ehsan Hajiramezanali Ehsan Hajiramezanali AI4CE 52 3 0 17 Jun 2024
Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models Masatoshi Uehara Yulai Zhao Ehsan Hajiramezanali Gabriele Scalia Gökçen Eraslan Avantika Lal Sergey Levine Tommaso Biancalani 45 13 0 30 May 2024
Discrete Probabilistic Inference as Control in Multi-path Environments T. Deleu Padideh Nouri Nikolay Malkin Doina Precup Yoshua Bengio 106 28 0 15 Feb 2024
Dirichlet Flow Matching with Applications to DNA Sequence Design Hannes Stärk Bowen Jing Chenyu Wang Gabriele Corso Bonnie Berger Regina Barzilay Tommi Jaakkola BDL 39 45 0 08 Feb 2024
Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design Andrew Campbell Jason Yim Regina Barzilay Tom Rainforth Tommi Jaakkola AI4CE 60 96 0 07 Feb 2024
Latent Diffusion Model for DNA Sequence Generation Zehui Li Yuhao Ni Tim August B. Huygelen Akashaditya Das Guoxuan Xia Guy-Bart Stan Yiren Zhao 30 9 0 09 Oct 2023
From Denoising Diffusions to Denoising Markov Models Joe Benton Yuyang Shi Valentin De Bortoli George Deligiannidis Arnaud Doucet DiffM 66 25 0 07 Nov 2022
Diffusion Models: A Comprehensive Survey of Methods and Applications Ling Yang Zhilong Zhang Yingxia Shao Shenda Hong Runsheng Xu Yue Zhao Wentao Zhang Bin Cui Ming-Hsuan Yang DiffM MedIm 213 1,277 0 02 Sep 2022
A Continuous Time Framework for Discrete Denoising Models Andrew Campbell Joe Benton Valentin De Bortoli Tom Rainforth George Deligiannidis Arnaud Doucet DiffM 172 132 0 30 May 2022
Planning with Diffusion for Flexible Behavior Synthesis Michael Janner Yilun Du J. Tenenbaum Sergey Levine DiffM 196 381 0 20 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Riemannian Score-Based Generative Modelling Valentin De Bortoli Emile Mathieu M. Hutchinson James Thornton Yee Whye Teh Arnaud Doucet DiffM 206 163 0 06 Feb 2022
Trajectory balance: Improved credit assignment in GFlowNets Nikolay Malkin Moksh Jain Emmanuel Bengio Chen Sun Yoshua Bengio 145 165 0 31 Jan 2022
Pessimistic Model-based Offline Reinforcement Learning under Partial Coverage Masatoshi Uehara Wen Sun OffRL 89 144 0 13 Jul 2021
MCMC using Hamiltonian dynamics Radford M. Neal 130 3,260 0 09 Jun 2012