Structure and Content-Guided Video Synthesis with Diffusion Models

6 February 2023

Papers citing "Structure and Content-Guided Video Synthesis with Diffusion Models"

50 / 422 papers shown

Title
SimDA: Simple Diffusion Adapter for Efficient Video Generation Zhen Xing Qi Dai Hang-Rui Hu Zuxuan Wu Yu-Gang Jiang VGen DiffM 24 81 0 18 Aug 2023
StableVideo: Text-driven Consistency-aware Diffusion Video Editing Wenhao Chai Xun Guo Gaoang Wang Yang Lu VGen DiffM 11 147 0 18 Aug 2023
Edit Temporal-Consistent Videos with Image Diffusion Model Yuan-Zheng Wang Yong Li Xiaoya Zhang Xin Liu Anbo Dai Antoni B. Chan Zhen Cui DiffM 19 5 0 17 Aug 2023
DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectory Sheng-Siang Yin Chenfei Wu Jian Liang Jie Shi Houqiang Li Gong Ming Nan Duan VGen 12 130 0 16 Aug 2023
CoDeF: Content Deformation Fields for Temporally Consistent Video Processing Ouyang Hao Qiuyu Wang Yuxi Xiao Qingyan Bai Juntao Zhang Kecheng Zheng Xiaowei Zhou Qifeng Chen Yujun Shen DiffM VGen 35 81 0 15 Aug 2023
ModelScope Text-to-Video Technical Report Jiuniu Wang Hangjie Yuan Dayou Chen Yingya Zhang Xiang Wang Shiwei Zhang VGen DiffM 16 388 0 12 Aug 2023
DiffSynth: Latent In-Iteration Deflickering for Realistic Video Synthesis Zhongjie Duan Lizhou You Chengyu Wang Cen Chen Ziheng Wu Weining Qian Jun Huang DiffM 23 8 0 07 Aug 2023
ConceptLab: Creative Concept Generation using VLM-Guided Diffusion Prior Constraints Elad Richardson Kfir Goldberg Yuval Alaluf Daniel Cohen-Or DiffM 21 10 0 03 Aug 2023
VideoControlNet: A Motion-Guided Video-to-Video Translation Framework by Using Diffusion Model with ControlNet Zhihao Hu Dong Xu DiffM VGen 6 64 0 26 Jul 2023
InFusion: Inject and Attention Fusion for Multi Concept Zero-Shot Text-based Video Editing Anant Khandelwal DiffM VGen 23 14 0 22 Jul 2023
PreDiff: Precipitation Nowcasting with Latent Diffusion Models Zhihan Gao Xingjian Shi Boran Han Hongya Wang Xiaoyong Jin Danielle C. Maddix Yi Zhu Mu Li Bernie Wang BDL DiffM 23 53 0 19 Jul 2023
TokenFlow: Consistent Diffusion Features for Consistent Video Editing Michal Geyer Omer Bar-Tal Shai Bagon Tali Dekel VGen DiffM 18 249 0 19 Jul 2023
AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning Yuwei Guo Ceyuan Yang Anyi Rao Zhengyang Liang Yaohui Wang Yu Qiao Maneesh Agrawala Dahua Lin Bo Dai VGen 18 780 0 10 Jul 2023
SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis Dustin Podell Zion English Kyle Lacey A. Blattmann Tim Dockhorn Jonas Muller Joe Penna Robin Rombach 11 2,109 0 04 Jul 2023
Collaborative Score Distillation for Consistent Visual Synthesis Subin Kim Kyungmin Lee June Suk Choi Jongheon Jeong Kihyuk Sohn Jinwoo Shin DiffM 19 21 0 04 Jul 2023
Bidirectional Temporal Diffusion Model for Temporally Consistent Human Animation Tserendorj Adiya Jae Shin Yoon Jungeun Lee Sang-hyeon Kim Hwasup Lim DiffM 13 0 0 02 Jul 2023
DisCo: Disentangled Control for Realistic Human Dance Generation Tan Wang Linjie Li Kevin Qinghong Lin Yuanhao Zhai Chung-Ching Lin Zhengyuan Yang Hanwang Zhang Zicheng Liu Lijuan Wang VGen 13 70 0 30 Jun 2023
Diff-Foley: Synchronized Video-to-Audio Synthesis with Latent Diffusion Models Simian Luo Chuanhao Yan Chenxu Hu Hang Zhao DiffM 11 79 0 29 Jun 2023
VidEdit: Zero-Shot and Spatially Aware Text-Driven Video Editing Paul Couairon Clément Rambour Jean-Emmanuel Haugeard Nicolas Thome DiffM VGen 4 29 0 14 Jun 2023
The Age of Synthetic Realities: Challenges and Opportunities J. P. Cardenuto Jing Yang Rafael Padilha Renjie Wan Daniel Moreira Haoliang Li Shiqi Wang Fernanda A. Andaló Sébastien Marcel Anderson de Rezende Rocha DeLMO 34 29 0 09 Jun 2023
Instructed Diffuser with Temporal Condition Guidance for Offline Reinforcement Learning Jifeng Hu Yan Sun Sili Huang Siyuan Guo Hechang Chen Li Shen Lichao Sun Yi-Ju Chang Dacheng Tao DiffM OffRL 26 13 0 08 Jun 2023
HeadSculpt: Crafting 3D Head Avatars with Text Xiaoping Han Yukang Cao Kai Han Xiatian Zhu Jiankang Deng Yi-Zhe Song Tao Xiang Kwan-Yee Kenneth Wong DiffM 11 45 0 05 Jun 2023
Stable Diffusion is Unstable Chengbin Du Yanxi Li Zhongwei Qiu Chang Xu DiffM 27 17 0 05 Jun 2023
VideoComposer: Compositional Video Synthesis with Motion Controllability Xiang Wang Hangjie Yuan Shiwei Zhang Dayou Chen Jiuniu Wang Yingya Zhang Yujun Shen Deli Zhao Jingren Zhou VGen DiffM 16 315 0 03 Jun 2023
Probabilistic Adaptation of Text-to-Video Models Mengjiao Yang Yilun Du Bo Dai Dale Schuurmans J. Tenenbaum Pieter Abbeel VGen DiffM 24 23 0 02 Jun 2023
Video Colorization with Pre-trained Text-to-Image Diffusion Models Hanyuan Liu M. Xie Jinbo Xing Chengze Li T. Wong VLM DiffM 8 12 0 02 Jun 2023
Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Models Chang-rui Liu Haoning Wu Yujie Zhong Xiaoyu Zhang Yanfeng Wang Weidi Xie DiffM VLM 20 39 0 01 Jun 2023
Make-Your-Video: Customized Video Generation Using Textual and Structural Guidance Jinbo Xing Menghan Xia Yuxin Liu Yuechen Zhang Yong Zhang ... Haoxin Chen Xiaodong Cun Xintao Wang Ying Shan T. Wong VGen DiffM 28 83 0 01 Jun 2023
Control4D: Efficient 4D Portrait Editing with Text Ruizhi Shao Jingxiang Sun Cheng Peng Zerong Zheng Boyao Zhou Hongwen Zhang Yebin Liu DiffM 14 22 0 31 May 2023
SAVE: Spectral-Shift-Aware Adaptation of Image Diffusion Models for Text-driven Video Editing Nazmul Karim Umar Khalid M. Joneidi Chen Chen Nazanin Rahnavard DiffM VGen 19 5 0 30 May 2023
Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoising Fu Lee Wang Wenshuo Chen Guanglu Song Han-Jia Ye Yu Liu Hongsheng Li VGen DiffM 28 87 0 29 May 2023
InstructEdit: Improving Automatic Masks for Diffusion-based Image Editing With User Instructions Qian Wang Biao Zhang Michael Birsak Peter Wonka DiffM 28 30 0 29 May 2023
Towards Consistent Video Editing with Text-to-Image Diffusion Models Zicheng Zhang Bonan Li Xuecheng Nie Congying Han Tiande Guo Luoqi Liu DiffM 10 24 0 27 May 2023
ControlVideo: Conditional Control for One-shot Text-driven Video Editing and Beyond Min Zhao Rongzheng Wang Fan Bao Chongxuan Li Jun Zhu VGen DiffM 16 4 0 26 May 2023
Optimal Linear Subspace Search: Learning to Construct Fast and High-Quality Schedulers for Diffusion Models Zhongjie Duan Chengyu Wang Cen Chen Jun Huang Weining Qian DiffM 11 12 0 24 May 2023
Vision + Language Applications: A Survey Yutong Zhou N. Shimada VLM 18 5 0 24 May 2023
DirecT2V: Large Language Models are Frame-Level Directors for Zero-Shot Text-to-Video Generation Susung Hong Junyoung Seo Heeseong Shin Sung‐Jin Hong Seung Wook Kim DiffM VGen 18 34 0 23 May 2023
Control-A-Video: Controllable Text-to-Video Generation with Diffusion Models Weifeng Chen Yatai Ji Jie Wu Hefeng Wu Pan Xie Jiashi Li Xin Xia Xuefeng Xiao Liang Lin VGen 121 6 0 23 May 2023
VDT: General-purpose Video Diffusion Transformers via Mask Modeling Haoyu Lu Guoxing Yang Nanyi Fei Yuqi Huo Zhiwu Lu Ping Luo Mingyu Ding DiffM VGen 20 56 0 22 May 2023
ControlVideo: Training-free Controllable Text-to-Video Generation Yabo Zhang Yuxiang Wei Dongsheng Jiang Xiaopeng Zhang W. Zuo Qi Tian VGen DiffM 8 235 0 22 May 2023
Any-to-Any Generation via Composable Diffusion Zineng Tang Ziyi Yang Chenguang Zhu Michael Zeng Mohit Bansal VGen DiffM 18 169 0 19 May 2023
Graphologue: Exploring Large Language Model Responses with Interactive Diagrams Peiling Jiang Jude Rayan Steven W. Dow Haijun Xia 9 96 0 19 May 2023
Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generation Wenjing Wang Huan Yang Zixi Tuo Huiguo He Junchen Zhu Jianlong Fu Jiaying Liu DiffM VGen 40 113 0 18 May 2023
Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models Songwei Ge Seungjun Nah Guilin Liu Tyler Poon Andrew Tao Bryan Catanzaro David Jacobs Jia-Bin Huang Ming-Yu Liu Yogesh Balaji DiffM VGen 27 248 0 17 May 2023
Make-A-Protagonist: Generic Video Editing with An Ensemble of Experts Yuyang Zhao Enze Xie Lanqing Hong Zhenguo Li G. Lee DiffM VGen 15 32 0 15 May 2023
Style-A-Video: Agile Diffusion for Arbitrary Text-based Video Style Transfer Nisha Huang Yu-xin Zhang Weiming Dong DiffM VGen 17 14 0 09 May 2023
Motion-Conditioned Diffusion Model for Controllable Video Synthesis Tsai-Shien Chen C. Lin Hung-Yu Tseng Tsung-Yi Lin Ming Yang DiffM VGen 69 60 0 27 Apr 2023
Seeing is not always believing: Benchmarking Human and Model Perception of AI-Generated Images Zeyu Lu Di Huang Lei Bai Jingjing Qu Chengzhi Wu Xihui Liu Wanli Ouyang 4 51 0 25 Apr 2023
Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models A. Blattmann Robin Rombach Huan Ling Tim Dockhorn Seung Wook Kim Sanja Fidler Karsten Kreis 3DGS VGen 11 1,009 0 18 Apr 2023
Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generation Jie An Songyang Zhang Harry Yang Sonal Gupta Jia-Bin Huang Jiebo Luo Xiaoyue Yin DiffM VGen 27 106 0 17 Apr 2023