Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think

9 October 2024

Papers citing "Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think"

45 / 45 papers shown

Title
No Other Representation Component Is Needed: Diffusion Transformers Can Provide Representation Guidance by Themselves D. Jiang Mengmeng Wang Liuzhuozheng Li Lei Zhang Haoyu Wang Wei Wei Guang Dai Yanning Zhang Jingdong Wang DiffM 30 0 0 05 May 2025
Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunities X. Zhang Jintao Guo Shanshan Zhao Minghao Fu Lunhao Duan Guo-Hua Wang Qing-Guo Chen Zhao Xu Weihua Luo Kaifu Zhang DiffM 57 0 0 05 May 2025
AlignDiT: Multimodal Aligned Diffusion Transformer for Synchronized Speech Generation J. Choi Ji-Hoon Kim Kim Sung-Bin Tae-Hyun Oh Joon Son Chung DiffM 44 0 0 29 Apr 2025
X-Fusion: Introducing New Modality to Frozen Large Language Models Sicheng Mo Thao Nguyen Xun Huang Siddharth Srinivasan Iyer Yijun Li ... Eli Shechtman Krishna Kumar Singh Yong Jae Lee Bolei Zhou Yuheng Li 68 0 0 29 Apr 2025
Enhancing Variational Autoencoders with Smooth Robust Latent Encoding Hyomin Lee Minseon Kim Sangwon Jang Jongheon Jeong S. Hwang DiffM AAML 27 0 0 24 Apr 2025
Boosting Generative Image Modeling via Joint Image-Feature Synthesis Theodoros Kouzelis Efstathios Karypidis Ioannis Kakogeorgiou Spyros Gidaris N. Komodakis DiffM 19 0 0 22 Apr 2025
Perception Encoder: The best visual embeddings are not at the output of the network Daniel Bolya Po-Yao (Bernie) Huang Peize Sun Jang Hyun Cho Andrea Madotto ... Shiyu Dong Nikhila Ravi Daniel Li Piotr Dollár Christoph Feichtenhofer ObjD VOS 98 0 0 17 Apr 2025
NormalCrafter: Learning Temporally Consistent Normals from Video Diffusion Priors Yanrui Bin Wenbo Hu Haoyuan Wang Xinya Chen Bing Wang DiffM 33 0 0 15 Apr 2025
Elucidating the Design Space of Multimodal Protein Language Models Cheng-Yen Hsieh X. Wang Daiheng Zhang Dongyu Xue Fei Ye Shujian Huang Zaixiang Zheng Quanquan Gu 16 1 0 15 Apr 2025
REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformers Xingjian Leng Jaskirat Singh Yunzhong Hou Zhenchang Xing Saining Xie Liang Zheng 27 0 0 14 Apr 2025
Efficient Generative Model Training via Embedded Representation Warmup Deyuan Liu Peng Sun Xufeng Li Tao Lin 14 0 0 14 Apr 2025
GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for Autoregressive Image Generation Tianwei Xiong Jun Hao Liew Zilong Huang Jiashi Feng Xihui Liu 21 0 0 11 Apr 2025
TARO: Timestep-Adaptive Representation Alignment with Onset-Aware Conditioning for Synchronized Video-to-Audio Synthesis Tri Ton Ji Woo Hong Chang D. Yoo VGen 19 0 0 08 Apr 2025
MergeVQ: A Unified Framework for Visual Generation and Representation with Disentangled Token Merging and Quantization Siyuan Li L. Zhang Zedong Wang Juanxi Tian Cheng Tan ... Chang Yu Qingsong Xie Haonan Lu Haoqian Wang Zhen Lei 46 0 0 01 Apr 2025
HumanDreamer: Generating Controllable Human-Motion Videos via Decoupled Generation Boyuan Wang Xiaofeng Wang Chaojun Ni Guosheng Zhao Zhiqin Yang ... Yukun Zhou Xinze Chen Guan Huang Lihong Liu Xingang Wang VGen 49 2 0 31 Mar 2025
MMGen: Unified Multi-modal Image Generation and Understanding in One Go Jiepeng Wang Zhaoqing Wang H. Pan Yuan Liu Dongdong Yu Changhu Wang Wenping Wang DiffM 71 0 0 26 Mar 2025
U-REPA: Aligning Diffusion U-Nets to ViTs Yuchuan Tian Hanting Chen Mengyu Zheng Yuchen Liang Chao Xu Yunhe Wang 49 0 0 24 Mar 2025
SimMotionEdit: Text-Based Human Motion Editing with Motion Similarity Prediction Zhengyuan Li Kai Cheng Anindita Ghosh Uttaran Bhattacharya Liangyan Gui Aniket Bera DiffM VGen 32 0 0 23 Mar 2025
Deeply Supervised Flow-Based Generative Models Inkyu Shin Chenglin Yang Liang-Chieh Chen 53 0 0 18 Mar 2025
FlowTok: Flowing Seamlessly Across Text and Image Tokens Ju He Qihang Yu Qihao Liu Liang-Chieh Chen 55 0 0 13 Mar 2025
Unified Dense Prediction of Video Diffusion Lehan Yang Lu Qi X. Li Sheng Li Varun Jampani Ming Yang MDE VOS VGen 53 0 0 12 Mar 2025
SARA: Structural and Adversarial Representation Alignment for Training-efficient Diffusion Models Hesen Chen Junyan Wang Zhiyu Tan Hao Li 43 0 0 11 Mar 2025
"Principal Components" Enable A New Language of Images Xin Wen Bingchen Zhao Ismail Elezi Jiankang Deng Xiaojuan Qi 52 0 0 11 Mar 2025
Aligning Text to Image in Diffusion Models is Easier Than You Think J. Lee Byunghee Cha Jeongsol Kim Jong Chul Ye 45 0 0 11 Mar 2025
Keeping Representation Similarity in Finetuning for Medical Image Analysis Wenqiang Zu Shenghao Xie Hao Chen Yiming Liang Lei Ma MedIm OOD 36 0 0 10 Mar 2025
Effective and Efficient Masked Image Generation Models Zebin You Jingyang Ou Xiaolu Zhang Jun Hu Jun Zhou Chongxuan Li DiffM VLM 47 1 0 10 Mar 2025
USP: Unified Self-Supervised Pretraining for Image Generation and Understanding Xiangxiang Chu Renda Li Yong Wang 53 0 0 08 Mar 2025
Interpreting CLIP with Hierarchical Sparse Autoencoders Vladimir Zaigrajew Hubert Baniecki P. Biecek 41 0 0 27 Feb 2025
Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generation Sucheng Ren Qihang Yu Ju He Xiaohui Shen Alan Yuille Liang-Chieh Chen VGen 71 5 0 27 Feb 2025
Human2Robot: Learning Robot Actions from Paired Human-Robot Videos Sicheng Xie Haidong Cao Zejia Weng Zhen Xing Shiwei Shen Jiaqi Leng Xipeng Qiu Yanwei Fu Zuxuan Wu Yu Jiang 34 0 0 23 Feb 2025
MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation Sihyun Yu Meera Hahn Dan Kondratyuk Jinwoo Shin Agrim Gupta José Lezama Irfan Essa David A. Ross Jonathan Huang DiffM VGen 63 0 0 18 Feb 2025
Diffusion Models without Classifier-free Guidance Zhicong Tang Jianmin Bao Dong Chen Baining Guo VLM 47 2 0 17 Feb 2025
EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling Theodoros Kouzelis Ioannis Kakogeorgiou Spyros Gidaris N. Komodakis DRL 62 5 0 17 Feb 2025
Understanding Representation Dynamics of Diffusion Models via Low-Dimensional Modeling Xiao Li Zekai Zhang Xiang Li Siyi Chen Zhihui Zhu Peng Wang Qing Qu DiffM 38 0 0 09 Feb 2025
AnyEnhance: A Unified Generative Model with Prompt-Guidance and Self-Critic for Voice Enhancement Junan Zhang Jing Yang Zihao Fang Y. Wang Zehua Zhang Zhuo Wang Fan Fan Z. Wu 29 2 0 26 Jan 2025
IntroStyle: Training-Free Introspective Style Attribution using Diffusion Features Anand Kumar Jiteng Mu Nuno Vasconcelos DiffM 64 0 0 19 Dec 2024
E-CAR: Efficient Continuous Autoregressive Image Generation via Multistage Modeling Zhihang Yuan Yuzhang Shang H. Zhang Tongcheng Fang Rui Xie Bingxin Xu Yan Yan Shengen Yan Guohao Dai Yu Wang DiffM 82 1 0 18 Dec 2024
SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizer H. Chen Z. Wang X. Li X. Sun Fangyi Chen Jiang Liu J. Wang Bhiksha Raj Zicheng Liu Emad Barsoum VLM 92 6 0 14 Dec 2024
[MASK] is All You Need Vincent Tao Hu Bjorn Ommer DiffM 127 2 0 09 Dec 2024
FoundHand: Large-Scale Domain-Specific Learning for Controllable Hand Image Generation Kefan Chen Chaerin Min Linguang Zhang Shreyas Hampali Cem Keskin Srinath Sridhar 70 0 0 03 Dec 2024
XQ-GAN: An Open-source Image Tokenization Framework for Autoregressive Generation X. Li Kai Qiu H. Chen Jason Kuen Jiuxiang Gu J. Wang Zhe-nan Lin Bhiksha Raj VLM 100 3 0 02 Dec 2024
COAP: Memory-Efficient Training with Correlation-Aware Gradient Projection Jinqi Xiao S. Sang Tiancheng Zhi Jing Liu Qing Yan Linjie Luo Bo Yuan Bo Yuan VLM 79 1 0 26 Nov 2024
Factorized Visual Tokenization and Generation Zechen Bai Jianxiong Gao Ziteng Gao Pichao Wang Zheng Zhang Tong He Mike Zheng Shou 56 0 0 25 Nov 2024
Optical Flow Representation Alignment Mamba Diffusion Model for Medical Video Generation Zhenbin Wang Lei Zhang Lituan Wang Minjuan Zhu Zhenwei Zhang VGen MedIm 54 1 0 03 Nov 2024
Movie Gen: A Cast of Media Foundation Models Adam Polyak Amit Zohar Andrew Brown Andros Tjandra Animesh Sinha ... Simone Parmeggiani Steve Fine Tara Fowler Vladan Petrovic Yuming Du VGen DiffM 58 160 0 17 Oct 2024