Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model

22 November 2023

Papers citing "Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model"

50 / 75 papers shown

Title
Flow-GRPO: Training Flow Matching Models via Online RL Jie Liu Gongye Liu Jiajun Liang Y. Li Jiaheng Liu X. Wang Pengfei Wan Di Zhang Wanli Ouyang AI4CE 66 0 0 08 May 2025
SuperEdit: Rectifying and Facilitating Supervision for Instruction-Based Image Editing Ming Li Xin Gu Fan Chen X. Xing Longyin Wen C. L. P. Chen Sijie Zhu DiffM 71 1 0 05 May 2025
Analytic Energy-Guided Policy Optimization for Offline Reinforcement Learning Jifeng Hu Sili Huang Z. Yang Shengchao Hu Li Shen H. Chen Lichao Sun Yi-Ju Chang Dacheng Tao OffRL 51 0 0 03 May 2025
Twin Co-Adaptive Dialogue for Progressive Image Generation J. Wang Yangfan He Yan Zhong Xinyuan Song Jiayi Su ... Miao Zhang K. Li Jiaqi Chen Tianyu Shi Xueqian Wang 19 0 0 21 Apr 2025
DRAGON: Distributional Rewards Optimize Diffusion Generative Models Yatong Bai Jonah Casebeer Somayeh Sojoudi Nicholas J. Bryan DiffM VLM 39 1 0 21 Apr 2025
SUDO: Enhancing Text-to-Image Diffusion Models with Self-Supervised Direct Preference Optimization Liang Peng Boxi Wu Haoran Cheng Yibo Zhao Xiaofei He 29 0 0 20 Apr 2025
SkyReels-V2: Infinite-length Film Generative Model Guibin Chen D. Lin Jiangping Yang Chunze Lin J. Zhu ... Di Qiu Debang Li Zhengcong Fei Yang Li Yahui Zhou DiffM VGen 47 1 0 17 Apr 2025
Image-Editing Specialists: An RLAIF Approach for Diffusion Models Elior Benarous Yilun Du Heng Yang 17 0 0 17 Apr 2025
Science-T2I: Addressing Scientific Illusions in Image Synthesis Jialuo Li Wenhao Chai Xingyu Fu Haiyang Xu Saining Xie MedIm 38 0 0 17 Apr 2025
FashionDPO:Fine-tune Fashion Outfit Generation Model using Direct Preference Optimization Mingzhe Yu Yunshan Ma Lei Wu Changshuo Wang Xue Li Lei Meng EGVM 40 0 0 17 Apr 2025
Aligning Anime Video Generation with Human Feedback Bingwen Zhu Yudong Jiang Baohan Xu Siqian Yang Mingyu Yin Yidi Wu Huyang Sun Zuxuan Wu EGVM VGen 42 0 0 14 Apr 2025
Discriminator-Free Direct Preference Optimization for Video Diffusion Haoran Cheng Qide Dong Liang Peng Zhizhou Sha Weiguo Feng Jinghui Xie Zhao Song Shilei Wen Xiaofei He Boxi Wu VGen 41 0 0 11 Apr 2025
Sharpe Ratio-Guided Active Learning for Preference Optimization in RLHF Syrine Belakaria Joshua Kazdan Charles Marx Chris Cundy W. Neiswanger Sanmi Koyejo Barbara Engelhardt Stefano Ermon 29 0 0 28 Mar 2025
IPGO: Indirect Prompt Gradient Optimization on Text-to-Image Generative Models with High Data Efficiency Jianping Ye Michel Wedel Kunpeng Zhang 37 0 0 25 Mar 2025
Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcing Jaihoon Kim Taehoon Yoon Jisung Hwang Minhyuk Sung DiffM 51 1 0 25 Mar 2025
InPO: Inversion Preference Optimization with Reparametrized DDIM for Efficient Diffusion Model Alignment Y. Lu Qichao Wang H. Cao Xierui Wang Xiaoyin Xu Min Zhang 56 0 0 24 Mar 2025
When Preferences Diverge: Aligning Diffusion Models with Minority-Aware Adaptive DPO L. Zhang Chen Liu C. Xu Kai Hu Donghao Luo Chengjie Wang Yanwei Fu Yuan Yao 42 0 0 21 Mar 2025
SketchFusion: Learning Universal Sketch Features through Fusing Foundation Models Subhadeep Koley Tapas Kumar Dutta Aneeshan Sain Pinaki Nath Chowdhury A. Bhunia Yi-Zhe Song VLM 66 0 0 18 Mar 2025
EvolvingGrasp: Evolutionary Grasp Generation via Efficient Preference Alignment Yufei Zhu Yiming Zhong Zemin Yang Peishan Cong Jingyi Yu X. Zhu Y. Ma 51 1 0 18 Mar 2025
Rewards Are Enough for Fast Photo-Realistic Text-to-image Generation Yihong Luo Tianyang Hu Weijian Luo Kenji Kawaguchi Jing Tang EGVM 70 0 0 17 Mar 2025
BalancedDPO: Adaptive Multi-Metric Alignment Dipesh Tamboli Souradip Chakraborty Aditya Malusare B. Banerjee Amrit Singh Bedi Vaneet Aggarwal EGVM 65 0 0 16 Mar 2025
Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion Transformers via In-Context Reflection Shufan Li Konstantinos Kallidromitis Akash Gokul Arsh Koneru Yusuke Kato Kazuki Kozuka Aditya Grover VLM 56 1 0 15 Mar 2025
Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards Zijing Hu Fengda Zhang Long Chen Kun Kuang Jiahui Li Kaifeng Gao Jun Xiao X. Wang Wenwu Zhu EGVM 47 0 0 14 Mar 2025
Preference-Based Alignment of Discrete Diffusion Models Umberto Borso Davide Paglieri Jude Wells Tim Rocktaschel 57 1 0 11 Mar 2025
CoPL: Collaborative Preference Learning for Personalizing LLMs Youngbin Choi Seunghyuk Cho M. Lee Moonjeong Park Yesong Ko Jungseul Ok Dongwoo Kim 58 0 0 03 Mar 2025
Diffusion-Sharpening: Fine-tuning Diffusion Models with Denoising Trajectory Sharpening Ye Tian L. Yang Xinchen Zhang Yunhai Tong Mengdi Wang Bin Cui 60 1 0 17 Feb 2025
Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model Guoqing Ma Haoyang Huang K. Yan L. Chen Nan Duan ... Y. Wang Yuanwei Lu Yu-Cheng Chen Yu-Juan Luo Y. Luo DiffM VGen 145 14 0 14 Feb 2025
Direct Preference Optimization-Enhanced Multi-Guided Diffusion Model for Traffic Scenario Generation Seungjun Yu Kisung Kim Daejung Kim Haewook Han Jinhan Lee 55 0 0 14 Feb 2025
Dual Caption Preference Optimization for Diffusion Models Amir Saeidi Yiran Luo Agneet Chatterjee Shamanthak Hegde Bimsara Pathiraja Yezhou Yang Chitta Baral DiffM 51 0 0 09 Feb 2025
HuViDPO:Enhancing Video Generation through Direct Preference Optimization for Human-Centric Alignment Lifan Jiang Boxi Wu Jiahui Zhang Xiaotong Guan Shuang Chen VGen 61 1 0 02 Feb 2025
Refining Alignment Framework for Diffusion Models with Intermediate-Step Preference Ranking Jie Ren Yuhang Zhang Dongrui Liu Xiaopeng Zhang Qi Tian 68 0 0 01 Feb 2025
Improving Video Generation with Human Feedback Jie Liu Gongye Liu Jiajun Liang Ziyang Yuan Xiaokun Liu ... Pengfei Wan Di Zhang Kun Gai Yujiu Yang Wanli Ouyang VGen EGVM 53 12 0 23 Jan 2025
Direct Unlearning Optimization for Robust and Safe Text-to-Image Models Yong-Hyun Park Sangdoo Yun Jin-Hwa Kim Junho Kim Geonhui Jang Yonghyun Jeong Junghyo Jo Gayoung Lee 73 12 0 17 Jan 2025
DyMO: Training-Free Diffusion Model Alignment with Dynamic Multi-Objective Scheduling Xin Xie Dong Gong 82 1 0 01 Dec 2024
Omegance: A Single Parameter for Various Granularities in Diffusion-Based Synthesis Xinyu Hou Zongsheng Yue Xiaoming Li Chen Change Loy VGen DiffM 96 0 0 26 Nov 2024
Reward Fine-Tuning Two-Step Diffusion Models via Learning Differentiable Latent-Space Surrogate Reward Zhiwei Jia Yuesong Nan Huixi Zhao Gengdai Liu EGVM 84 0 0 22 Nov 2024
Aligning Few-Step Diffusion Models with Dense Reward Difference Learning Ziyi Zhang Li Shen Sen Zhang Deheng Ye Yong Luo Miaojing Shi Bo Du Dacheng Tao 89 0 0 18 Nov 2024
SEE-DPO: Self Entropy Enhanced Direct Preference Optimization Shivanshu Shekhar Shreyas Singh Tong Zhang 30 4 0 06 Nov 2024
Controlling Language and Diffusion Models by Transporting Activations P. Rodríguez Arno Blaas Michal Klein Luca Zappella N. Apostoloff Marco Cuturi Xavier Suau LLMSV 35 4 0 30 Oct 2024
PrefPaint: Aligning Image Inpainting Diffusion Model with Human Preference Kendong Liu Zhiyu Zhu Chuanhao Li Hui Liu H. Zeng Junhui Hou EGVM 38 2 0 29 Oct 2024
Diff-Instruct*: Towards Human-Preferred One-step Text-to-image Generative Models Weijian Luo C. Zhang Debing Zhang Zhengyang Geng 21 3 0 28 Oct 2024
Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferences Weijian Luo EGVM 36 6 0 24 Oct 2024
IterComp: Iterative Composition-Aware Feedback Learning from Model Gallery for Text-to-Image Generation Xinchen Zhang Ling Yang G. Li Yaqi Cai Jiake Xie Yong Tang Yujiu Yang Mengdi Wang Bin Cui EGVM CoGe 28 5 0 09 Oct 2024
Training-free Diffusion Model Alignment with Sampling Demons Po-Hung Yeh Kuang-Huei Lee Jun-Cheng Chen 24 4 0 08 Oct 2024
SePPO: Semi-Policy Preference Optimization for Diffusion Alignment Daoan Zhang Guangchen Lan Dong-Jun Han Wenlin Yao Xiaoman Pan ... Mingxiao Li Pengcheng Chen Yu Dong Christopher Brinton Jiebo Luo EGVM 31 4 0 07 Oct 2024
HERO: Human-Feedback Efficient Reinforcement Learning for Online Diffusion Model Finetuning Ayano Hiranaka Shang-Fu Chen Chieh-Hsin Lai Dongjun Kim Naoki Murata Takashi Shibuya Wei-Hsiang Liao Shao-Hua Sun Yuki Mitsufuji 39 1 0 07 Oct 2024
Tuning Timestep-Distilled Diffusion Model Using Pairwise Sample Optimization Zichen Miao Zhengyuan Yang Kevin Lin Ze Wang Zicheng Liu Lijuan Wang Qiang Qiu 40 3 0 04 Oct 2024
Illustrious: an Open Advanced Illustration Model Sang Hyun Park Jun Young Koh Junha Lee Joy Song Dongha Kim Hoyeon Moon Hyunju Lee Min Song VLM 18 1 0 30 Sep 2024
Generalizing Alignment Paradigm of Text-to-Image Generation with Preferences through $f$ -divergence Minimization Haoyuan Sun Bo Xia Yongzhe Chang Xueqian Wang EGVM 35 2 0 15 Sep 2024
Alignment of Diffusion Models: Fundamentals, Challenges, and Future Buhua Liu Shitong Shao Bao Li Lichen Bai Zhiqiang Xu Haoyi Xiong James Kwok Sumi Helal Zeke Xie 37 11 0 11 Sep 2024