Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2505.18736
Cited By
v1
v2 (latest)
Rethinking Direct Preference Optimization in Diffusion Models
24 May 2025
Junyong Kang
Seohyun Lim
Kyungjune Baek
Hyunjung Shim
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"Rethinking Direct Preference Optimization in Diffusion Models"
33 / 33 papers shown
Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards
Computer Vision and Pattern Recognition (CVPR), 2025
Zijing Hu
Tai-wei Chang
Long Chen
Kun Kuang
Jiahui Li
Kaifeng Gao
Jun Xiao
X. Wang
Wenwu Zhu
EGVM
595
18
0
14 Mar 2025
RainbowPO: A Unified Framework for Combining Improvements in Preference Optimization
International Conference on Learning Representations (ICLR), 2024
Hanyang Zhao
Genta Indra Winata
Anirban Das
Shi-Xiong Zhang
D. Yao
Wenpin Tang
Sambit Sahu
389
17
0
05 Oct 2024
Margin-aware Preference Optimization for Aligning Diffusion Models without Reference
Jiwoo Hong
Sayak Paul
Noah Lee
Kashif Rasul
James Thorne
Jongheon Jeong
323
32
0
10 Jun 2024
SimPO: Simple Preference Optimization with a Reference-Free Reward
Neural Information Processing Systems (NeurIPS), 2024
Yu Meng
Mengzhou Xia
Danqi Chen
542
785
0
23 May 2024
Learn Your Reference Model for Real Good Alignment
Alexey Gorbatovski
Boris Shaposhnikov
Alexey Malakhov
Nikita Surnachev
Gleb Gerasimov
Ian Maksimov
Nikita Balagansky
Daniil Gavrilov
OffRL
570
46
0
15 Apr 2024
Aligning Diffusion Models by Optimizing Human Utility
Shufan Li
Konstantinos Kallidromitis
Akash Gokul
Yusuke Kato
Kazuki Kozuka
308
68
0
06 Apr 2024
ORPO: Monolithic Preference Optimization without Reference Model
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Jiwoo Hong
Noah Lee
James Thorne
OSLM
702
446
0
12 Mar 2024
A Dense Reward View on Aligning Text-to-Image Diffusion with Preference
Shentao Yang
Tianqi Chen
Mingyuan Zhou
EGVM
348
44
0
13 Feb 2024
KTO: Model Alignment as Prospect Theoretic Optimization
Kawin Ethayarajh
Winnie Xu
Niklas Muennighoff
Dan Jurafsky
Douwe Kiela
840
834
0
02 Feb 2024
Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model
Computer Vision and Pattern Recognition (CVPR), 2023
Kai Yang
Jian Tao
Jiafei Lyu
Chunjiang Ge
Jiaxin Chen
Qimai Li
Weihan Shen
Xiaolong Zhu
Xiu Li
EGVM
493
188
0
22 Nov 2023
Diffusion Model Alignment Using Direct Preference Optimization
Computer Vision and Pattern Recognition (CVPR), 2023
Bram Wallace
Meihua Dang
Rafael Rafailov
Linqi Zhou
Aaron Lou
Senthil Purushwalkam
Stefano Ermon
Caiming Xiong
Shafiq Joty
Nikhil Naik
EGVM
450
516
0
21 Nov 2023
A General Theoretical Paradigm to Understand Learning from Human Preferences
International Conference on Artificial Intelligence and Statistics (AISTATS), 2023
M. G. Azar
Mark Rowland
Bilal Piot
Daniel Guo
Daniele Calandriello
Michal Valko
Rémi Munos
615
845
0
18 Oct 2023
Aligning Text-to-Image Diffusion Models with Reward Backpropagation
Mihir Prabhudesai
Anirudh Goyal
Deepak Pathak
Katerina Fragkiadaki
468
207
0
05 Oct 2023
Directly Fine-Tuning Diffusion Models on Differentiable Rewards
International Conference on Learning Representations (ICLR), 2023
Amita Gajewar
Paul Vicol
G. Bansal
David J Fleet
272
303
0
29 Sep 2023
Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis
Xiaoshi Wu
Yiming Hao
Keqiang Sun
Yixiong Chen
Feng Zhu
Rui Zhao
Jiaming Song
294
567
0
15 Jun 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Neural Information Processing Systems (NeurIPS), 2023
Rafael Rafailov
Archit Sharma
E. Mitchell
Stefano Ermon
Christopher D. Manning
Chelsea Finn
ALM
906
6,769
0
29 May 2023
DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models
Neural Information Processing Systems (NeurIPS), 2023
Ying Fan
Olivia Watkins
Yuqing Du
Hao Liu
Moonkyung Ryu
Craig Boutilier
Pieter Abbeel
Mohammad Ghavamzadeh
Kangwook Lee
Kimin Lee
418
287
0
25 May 2023
Training Diffusion Models with Reinforcement Learning
International Conference on Learning Representations (ICLR), 2023
Kevin Black
Michael Janner
Yilun Du
Ilya Kostrikov
Sergey Levine
EGVM
590
640
0
22 May 2023
Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation
Neural Information Processing Systems (NeurIPS), 2023
Yuval Kirstain
Adam Polyak
Uriel Singer
Shahbuland Matiana
Joe Penna
Omer Levy
EGVM
977
703
0
02 May 2023
MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing
IEEE International Conference on Computer Vision (ICCV), 2023
Ming Cao
Xintao Wang
Chen Ma
Ying Shan
Xiaohu Qie
Yinqiang Zheng
DiffM
232
680
0
17 Apr 2023
ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation
Neural Information Processing Systems (NeurIPS), 2023
Jiazheng Xu
Xiao Liu
Yuchen Wu
Yuxuan Tong
Qinkai Li
Ming Ding
Jie Tang
Yuxiao Dong
570
745
0
12 Apr 2023
Efficient Diffusion Training via Min-SNR Weighting Strategy
IEEE International Conference on Computer Vision (ICCV), 2023
Tiankai Hang
Shuyang Gu
Chen Li
Jianmin Bao
Dong Chen
Han Hu
Xin Geng
B. Guo
306
221
0
16 Mar 2023
Diffusion Models Generate Images Like Painters: an Analytical Theory of Outline First, Details Later
Binxu Wang
John J. Vastola
DiffM
501
38
0
04 Mar 2023
eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers
Yogesh Balaji
Seungjun Nah
Xun Huang
Arash Vahdat
Jiaming Song
...
Timo Aila
S. Laine
Bryan Catanzaro
Tero Karras
Xuan Li
VLM
MoE
641
984
0
02 Nov 2022
Scaling Autoregressive Models for Content-Rich Text-to-Image Generation
Jiahui Yu
Yuanzhong Xu
Jing Yu Koh
Thang Luong
Gunjan Baid
...
Zarana Parekh
Xin Li
Han Zhang
Jason Baldridge
Yonghui Wu
EGVM
725
1,367
0
22 Jun 2022
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
Neural Information Processing Systems (NeurIPS), 2022
Chitwan Saharia
William Chan
Saurabh Saxena
Lala Li
Jay Whang
...
Raphael Gontijo-Lopes
Tim Salimans
Jonathan Ho
David J Fleet
Mohammad Norouzi
VLM
1.2K
7,527
0
23 May 2022
Perception Prioritized Training of Diffusion Models
Computer Vision and Pattern Recognition (CVPR), 2022
Jooyoung Choi
Jungbeom Lee
Chaehun Shin
Sungwon Kim
Hyunwoo J. Kim
Sung-Hoon Yoon
DiffM
299
327
0
01 Apr 2022
Training language models to follow instructions with human feedback
Neural Information Processing Systems (NeurIPS), 2022
Long Ouyang
Jeff Wu
Xu Jiang
Diogo Almeida
Carroll L. Wainwright
...
Amanda Askell
Peter Welinder
Paul Christiano
Jan Leike
Ryan J. Lowe
OSLM
ALM
2.1K
17,754
0
04 Mar 2022
High-Resolution Image Synthesis with Latent Diffusion Models
Computer Vision and Pattern Recognition (CVPR), 2021
Robin Rombach
A. Blattmann
Dominik Lorenz
Patrick Esser
Bjorn Ommer
DiffM
3.0K
21,220
0
20 Dec 2021
Learning Transferable Visual Models From Natural Language Supervision
International Conference on Machine Learning (ICML), 2021
Alec Radford
Jong Wook Kim
Chris Hallacy
Aditya A. Ramesh
Gabriel Goh
...
Amanda Askell
Pamela Mishkin
Jack Clark
Gretchen Krueger
Ilya Sutskever
CLIP
VLM
2.0K
41,575
0
26 Feb 2021
Score-Based Generative Modeling through Stochastic Differential Equations
International Conference on Learning Representations (ICLR), 2020
Yang Song
Jascha Narain Sohl-Dickstein
Diederik P. Kingma
Abhishek Kumar
Stefano Ermon
Ben Poole
DiffM
SyDa
2.2K
8,952
0
26 Nov 2020
Denoising Diffusion Probabilistic Models
Jonathan Ho
Ajay Jain
Pieter Abbeel
DiffM
5.1K
26,105
0
19 Jun 2020
Generative Modeling by Estimating Gradients of the Data Distribution
Neural Information Processing Systems (NeurIPS), 2019
Yang Song
Stefano Ermon
SyDa
DiffM
804
4,884
0
12 Jul 2019
1
Page 1 of 1
