MotionCtrl: A Unified and Flexible Motion Controller for Video Generation

6 December 2023

Ping Luo

Ying Shan

DiffM

VGen

ArXiv PDF HTML

Papers citing "MotionCtrl: A Unified and Flexible Motion Controller for Video Generation"

50 / 154 papers shown

Title
CinePreGen: Camera Controllable Video Previsualization via Engine-powered Diffusion Yiran Chen Anyi Rao Xuekun Jiang Shishi Xiao Ruiqing Ma Zeyu Wang Hui Xiong Bo Dai VGen DiffM 26 1 0 30 Aug 2024
ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Model F. Liu Wenqiang Sun Hanyang Wang Yikai Wang Haowen Sun Junliang Ye Jun Zhang Yueqi Duan VGen 35 31 0 29 Aug 2024
EasyControl: Transfer ControlNet to Video Diffusion for Controllable Generation and Interpolation Cong Wang Jiaxi Gu Panwen Hu Haoyu Zhao Yuanfan Guo J. N. Han Hang Xu Xiaodan Liang VGen DiffM 26 3 0 23 Aug 2024
TrackGo: A Flexible and Efficient Method for Controllable Video Generation Haitao Zhou Chuang Wang Rui Nie Jinxiao Lin Dongdong Yu Qian Yu Changhu Wang VGen DiffM 46 14 0 21 Aug 2024
Puppet-Master: Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamics Ruining Li Chuanxia Zheng Christian Rupprecht Andrea Vedaldi DiffM VGen 36 9 0 08 Aug 2024
Reenact Anything: Semantic Video Motion Transfer Using Motion-Textual Inversion Monika Zimmermann Jacek Naruniec Christopher Schroers Markus Gross Romann M. Weber VGen DiffM 40 3 0 01 Aug 2024
Tora: Trajectory-oriented Diffusion Transformer for Video Generation Zhenghao Zhang Junchao Liao Menghao Li Zuozhuo Dai Bingxue Qiu Hao Hu Shaowei Cai Weizhi Wang VGen 42 42 0 31 Jul 2024
Faster Image2Video Generation: A Closer Look at CLIP Image Embedding's Impact on Spatio-Temporal Cross-Attentions Ashkan Taghipour Morteza Ghahremani Bennamoun Aref Miri Rekavandi Zinuo Li Hamid Laga F. Boussaïd VGen 71 2 0 27 Jul 2024
HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation Zhenzhi Wang Yixuan Li Yanhong Zeng Youqing Fang Yuwei Guo ... Jing Tan Kai Chen Tianfan Xue Bo Dai Dahua Lin VGen 3DH 38 17 0 24 Jul 2024
Streetscapes: Large-scale Consistent Street View Generation Using Autoregressive Video Diffusion Boyang Deng Richard Tucker Zhengqi Li Leonidas J. Guibas Noah Snavely Gordon Wetzstein VGen 3DGS DiffM 32 11 0 18 Jul 2024
VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control Sherwin Bahmani Ivan Skorokhodov Aliaksandr Siarohin Willi Menapace Guocheng Qian ... Chaoyang Wang Jiaxu Zou Andrea Tagliasacchi David B. Lindell Sergey Tulyakov VGen DiffM 77 42 0 17 Jul 2024
Controlling Space and Time with Diffusion Models Daniel Watson Saurabh Saxena Lala Li Andrea Tagliasacchi David J. Fleet VGen 56 27 0 10 Jul 2024
TAPVid-3D: A Benchmark for Tracking Any Point in 3D Skanda Koppula Ignacio Rocco Yi Yang Joe Heyward João Carreira Andrew Zisserman Gabriel J. Brostow Carl Doersch 39 14 0 08 Jul 2024
E.T. the Exceptional Trajectories: Text-to-camera-trajectory generation with character awareness Robin Courant Nicolas Dufour Xi Wang Marc Christie Vicky Kalogeiton VGen 31 4 0 01 Jul 2024
Text-Animator: Controllable Visual Text Video Generation Lin Liu Quande Liu Shengju Qian Yuan Zhou Wengang Zhou Houqiang Li Lingxi Xie Qi Tian VGen 33 1 0 25 Jun 2024
MotionBooth: Motion-Aware Customized Text-to-Video Generation Jianzong Wu Xiangtai Li Yanhong Zeng J. J. Zhang Qianyu Zhou Yining Li Yunhai Tong Kai Chen DiffM VGen 70 40 0 25 Jun 2024
FreeTraj: Tuning-Free Trajectory Control in Video Diffusion Models Haonan Qiu Zhaoxi Chen Zhouxia Wang Yingqing He Menghan Xia Ziwei Liu VGen DiffM 34 17 0 24 Jun 2024
Identifying and Solving Conditional Image Leakage in Image-to-Video Diffusion Model Min Zhao Hongzhou Zhu Chendong Xiang Kaiwen Zheng Chongxuan Li Jun Zhu 61 8 0 22 Jun 2024
Image Conductor: Precision Control for Interactive Video Synthesis Yaowei Li Xintao Wang Zhaoyang Zhang Zhouxia Wang Ziyang Yuan Liangbin Xie Yuexian Zou Ying Shan VGen 42 23 0 21 Jun 2024
Training-free Camera Control for Video Generation Chen Hou Guoqiang Wei VGen DiffM 70 29 0 14 Jun 2024
WonderWorld: Interactive 3D Scene Generation from a Single Image Hong-Xing Yu Haoyi Duan Charles Herrmann William T. Freeman Jiajun Wu 3DGS VGen 41 37 0 13 Jun 2024
Vivid-ZOO: Multi-View Video Generation with Diffusion Model Bing Li Cheng Zheng Wenxuan Zhu Jinjie Mai Biao Zhang Peter Wonka Bernard Ghanem 40 16 0 12 Jun 2024
4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models Heng Yu Chaoyang Wang Peiye Zhuang Willi Menapace Aliaksandr Siarohin Junli Cao László A. Jeni Sergey Tulyakov Hsin-Ying Lee VGen 43 23 0 11 Jun 2024
ProcessPainter: Learn Painting Process from Sequence Data Yiren Song Shijie Huang Chen Yao Xiaojun Ye Hai Ci Jiaming Liu Yuxuan Zhang Mike Zheng Shou DiffM 26 6 0 10 Jun 2024
MotionClone: Training-Free Motion Cloning for Controllable Video Generation Pengyang Ling Jiazi Bu Pan Zhang Xiaoyi Dong Yuhang Zang Tong Wu H. Chen Jiaqi Wang Yi Jin VGen DiffM 26 34 0 08 Jun 2024
CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generation Dejia Xu Weili Nie Chao Liu Sifei Liu Jan Kautz Zhangyang Wang Arash Vahdat DiffM VGen 71 52 0 04 Jun 2024
Bootstrap3D: Improving 3D Content Creation with Synthetic Data Zeyi Sun Tong Wu Pan Zhang Yuhang Zang Xiao-wen Dong Yuanjun Xiong Dahua Lin Jiaqi Wang 34 0 0 31 May 2024
VividDream: Generating 3D Scene with Ambient Dynamics Yao-Chih Lee Yi-Ting Chen Andrew Wang Ting-Hsuan Liao Brandon Y. Feng Jia-Bin Huang VGen 32 10 0 30 May 2024
MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model Muyao Niu Xiaodong Cun Xintao Wang Yong Zhang Ying Shan Yinqiang Zheng DiffM VGen 33 30 0 30 May 2024
Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Control Zhengfei Kuang Shengqu Cai Hao He Yinghao Xu Hongsheng Li Leonidas J. Guibas Gordon Wetzstein VGen DiffM 35 30 0 27 May 2024
Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability Shenyuan Gao Jiazhi Yang Li Chen Kashyap Chitta Yihang Qiu Andreas Geiger Jun Zhang Hongyang Li 60 75 0 27 May 2024
NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizer Meng You Zhiyu Zhu Hui Liu Junhui Hou VGen DiffM 23 23 0 24 May 2024
Video Diffusion Models are Training-free Motion Interpreter and Controller Zeqi Xiao Yifan Zhou Shuai Yang Xingang Pan VGen 57 17 0 23 May 2024
CAT3D: Create Anything in 3D with Multi-View Diffusion Models Ruiqi Gao Aleksander Holynski Philipp Henzler Arthur Brussee Ricardo Martín Brualla Pratul P. Srinivasan Jonathan T. Barron Ben Poole 29 149 0 16 May 2024
Motion Inversion for Video Customization Luozhou Wang Guibao Shen Yixun Liang Xin Tao Pengfei Wan Di Zhang Yijun Li Yingcong Chen VGen DiffM 34 7 0 29 Mar 2024
TC4D: Trajectory-Conditioned Text-to-4D Generation Sherwin Bahmani Xian Liu Yifan Wang Ivan Skorokhodov Victor Rong ... Jeong Joon Park Sergey Tulyakov Gordon Wetzstein Andrea Tagliasacchi David B. Lindell 94 34 0 26 Mar 2024
DragAPart: Learning a Part-Level Motion Prior for Articulated Objects Ruining Li Chuanxia Zheng Christian Rupprecht Andrea Vedaldi DiffM 32 17 0 22 Mar 2024
Explorative Inbetweening of Time and Space Haiwen Feng Zheng Ding Zhihao Xia Simon Niklaus Victoria Fernandez-Abrevaya Michael J. Black Xuaner Zhang DiffM VGen 29 5 0 21 Mar 2024
DragAnything: Motion Control for Anything using Entity Representation Wejia Wu Zhuang Li Yuchao Gu Rui Zhao Yefei He David Junhao Zhang Mike Zheng Shou Yan Li Tingting Gao Di Zhang VGen 71 51 0 12 Mar 2024
Action Reimagined: Text-to-Pose Video Editing for Dynamic Human Actions Lan Wang Vishnu Naresh Boddeti Sernam Lim VGen DiffM 29 0 0 11 Mar 2024
UniEdit: A Unified Tuning-Free Framework for Video Motion and Appearance Editing Jianhong Bai Tianyu He Yuchi Wang Junliang Guo Haoji Hu Zuozhu Liu Jiang Bian VGen 16 26 0 20 Feb 2024
Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motion Shiyuan Yang Liang Hou Haibin Huang Chongyang Ma Pengfei Wan Di Zhang Xiaodong Chen Jing Liao VGen DiffM 64 77 0 05 Feb 2024
Boximator: Generating Rich and Controllable Motions for Video Synthesis Jiawei Wang Yuchen Zhang Jiaxin Zou Yan Zeng Guoqiang Wei Liping Yuan Hang Li DiffM VGen 21 42 0 02 Feb 2024
Motion-Zero: Zero-Shot Moving Object Control Framework for Diffusion-Based Video Generation Changgu Chen Junwei Shu Lianggangxu Chen Gaoqi He Changbo Wang VGen 17 13 0 18 Jan 2024
TrailBlazer: Trajectory Control for Diffusion-Based Video Generation W. Ma J. P. Lewis W. Kleijn DiffM VGen 11 34 0 31 Dec 2023
PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding Zhen Li Mingdeng Cao Xintao Wang Zhongang Qi Ming-Ming Cheng Ying Shan DiffM 39 187 0 07 Dec 2023
TrackDiffusion: Tracklet-Conditioned Video Generation via Diffusion Models Pengxiang Li Kai Chen Zhili Liu Ruiyuan Gao Lanqing Hong Guo Zhou Hua Yao Dit-Yan Yeung Huchuan Lu Xu Jia VGen DiffM 20 0 0 01 Dec 2023
Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets A. Blattmann Tim Dockhorn Sumith Kulal Daniel Mendelevitch Maciej Kilian ... Zion English Vikram S. Voleti Adam Letts Varun Jampani Robin Rombach VGen 150 985 0 25 Nov 2023
LAMP: Learn A Motion Pattern for Few-Shot-Based Video Generation Ruiqi Wu Liangyu Chen Tong Yang Chunle Guo Chongyi Li Xiangyu Zhang DiffM VGen 86 52 0 16 Oct 2023
A Survey on Video Diffusion Models Zhen Xing Qijun Feng Haoran Chen Qi Dai Hang-Rui Hu Hang Xu Zuxuan Wu Yu-Gang Jiang EGVM VGen 52 112 0 16 Oct 2023