MotionCtrl: A Unified and Flexible Motion Controller for Video Generation

6 December 2023

Ping Luo

Ying Shan

DiffM

VGen

ArXiv PDF HTML

Papers citing "MotionCtrl: A Unified and Flexible Motion Controller for Video Generation"

50 / 154 papers shown

Title
TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Models Mark YU Wenbo Hu Jinbo Xing Ying Shan VGen 85 3 0 07 Mar 2025
FuseChat-3.0: Preference Optimization Meets Heterogeneous Model Fusion Ziyi Yang Fanqi Wan Longguang Zhong Canbin Huang Guosheng Liang Xiaojun Quan MoMe 90 0 0 06 Mar 2025
GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control Xuanchi Ren Tianchang Shen Jiahui Huang Huan Ling Yifan Lu Merlin Nimier-David Thomas Muller Alexander Keller Sanja Fidler Jun Gao DiffM VGen 74 8 0 05 Mar 2025
Learning to Animate Images from A Few Videos to Portray Delicate Human Actions Haoxin Li Yingchen Yu Qilong Wu Hanwang Zhang Boyang Li Song Bai 3DH VGen 75 0 0 01 Mar 2025
C-Drag: Chain-of-Thought Driven Motion Controller for Video Generation Yuhao Li Mirana Claire Angel Salman Khan Yu Zhu Jinqiu Sun Yanning Zhang F. Khan VGen 46 0 0 27 Feb 2025
MotionMatcher: Motion Customization of Text-to-Video Diffusion Models via Motion Feature Matching Yen-Siang Wu Chi-Pin Huang Fu-En Yang Yu-Jie Wang DiffM VGen 54 1 0 18 Feb 2025
MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generation Jinbo Xing Long Mai Cusuh Ham Jiahui Huang Aniruddha Mahapatra Chi-Wing Fu T. Wong Feng Liu DiffM VGen 116 2 0 06 Feb 2025
Consistent Video Colorization via Palette Guidance Han Wang Yuang Zhang Yuhong Zhang Lingxiao Lu Li-Na Song DiffM VGen 86 0 0 31 Jan 2025
PreciseCam: Precise Camera Control for Text-to-Image Generation Edurne Bernal-Berdun Ana Serrano B. Masiá Matheus Gadelha Yannick Hold-Geoffroy Xin Sun Diego F. F. Gutierrez DiffM VGen 45 0 0 22 Jan 2025
GS-DiT: Advancing Video Generation with Pseudo 4D Gaussian Fields through Efficient Dense 3D Point Tracking Weikang Bian Zhaoyang Huang Xiaoyu Shi Yijin Li Fu-Yun Wang Hongsheng Li 3DGS VGen DiffM 34 3 0 05 Jan 2025
AKiRa: Augmentation Kit on Rays for optical video generation Xi Wang Robin Courant Marc Christie Vicky Kalogeiton VGen 98 3 0 31 Dec 2024
Generative Inbetweening through Frame-wise Conditions-Driven Video Generation Tianyi Zhu Dongwei Ren Qilong Wang Xiaohe Wu W. Zuo VGen 67 1 0 16 Dec 2024
Can video generation replace cinematographers? Research on the cinematic language of generated video X. Li Kai WU Siyi Yang YiZhan Qu Guohua. Zhang ... Mingliang Xiong Hao Deng Qingwen Liu Gang Li Bin He VGen DiffM 90 1 0 16 Dec 2024
InterDyn: Controllable Interactive Dynamics with Video Diffusion Models Rick Akkerman Haiwen Feng M. Black Dimitrios Tzionas Victoria Fernandez-Abrevaya VGen AI4CE 100 3 0 16 Dec 2024
Wonderland: Navigating 3D Scenes from a Single Image Hanwen Liang Junli Cao Vidit Goel Guocheng Qian Sergei Korolev Demetri Terzopoulos Konstantinos N. Plataniotis Sergey Tulyakov Jian Ren VGen 125 11 0 16 Dec 2024
GenLit: Reformulating Single-Image Relighting as Video Generation Shrisha Bharadwaj Haiwen Feng Victoria Fernandez-Abrevaya Michael J. Black VGen 75 2 0 15 Dec 2024
Video Diffusion Transformers are In-Context Learners Zhengcong Fei Di Qiu Changqian Yu Debang Li Mingyuan Fan VGen DiffM 127 2 0 14 Dec 2024
LiftImage3D: Lifting Any Single Image to 3D Gaussians with Video Generation Priors Yabo Chen Chen-Ning Yang Jiemin Fang Xiaopeng Zhang Lingxi Xie Wei-Ming Shen Wenrui Dai Hongkai Xiong Q. Tian 3DGS 94 2 0 12 Dec 2024
Mojito: Motion Trajectory and Intensity Control for Video Generation Xuehai He Shuohang Wang Jianwei Yang Xiaoxia Wu Y. Wang Kuan-Chieh Jackson Wang Z. Zhan Olatunji Ruwase Yelong Shen X. Wang VGen 83 1 0 12 Dec 2024
You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scale Baorui Ma Huachen Gao Haoge Deng Zhengxiong Luo Tiejun Huang Lulu Tang Xinlong Wang DiffM VGen 114 14 0 09 Dec 2024
4Real-Video: Learning Generalizable Photo-Realistic 4D Video Diffusion Chaoyang Wang Peiye Zhuang Tuan Duc Ngo Willi Menapace Aliaksandr Siarohin Michael Vasilkovsky Ivan Skorokhodov Sergey Tulyakov Peter Wonka Hsin-Ying Lee DiffM VGen 90 3 0 05 Dec 2024
InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Models Yifan Lu Xuanchi Ren Jiawei Yang Tianchang Shen Zhangjie Wu ... Y. Wang Siheng Chen Mike Chen Sanja Fidler Jiahui Huang VGen 98 5 0 05 Dec 2024
World-consistent Video Diffusion with Explicit 3D Modeling Qihang Zhang Shuangfei Zhai Miguel Angel Bautista Kevin Miao Alexander Toshev J. Susskind Jiatao Gu VGen 75 8 0 02 Dec 2024
CPA: Camera-pose-awareness Diffusion Transformer for Video Generation Yuelei Wang Jian Zhang Pengtao Jiang H. Zhang Jinwei Chen Bo Li VGen DiffM 105 4 0 02 Dec 2024
AerialGo: Walking-through City View Generation from Aerial Perspectives Fuqiang Zhao Yijing Guo Siyuan Yang Xi Chen Luo Wang Lan Xu Y. Zhang Yujiao Shi Jingyi Yu 68 0 0 29 Nov 2024
MVGenMaster: Scaling Multi-View Generation from Any Image via 3D Priors Enhanced Diffusion Model Chenjie Cao Chaohui Yu Shang Liu Fan Wang Xiangyang Xue Yanwei Fu 87 1 0 25 Nov 2024
Novel View Extrapolation with Video Diffusion Priors Kunhao Liu Ling Shao Shijian Lu VGen 75 3 0 21 Nov 2024
PhysFlow: Unleashing the Potential of Multi-modal Foundation Models and Video Diffusion for 4D Dynamic Physical Scene Simulation Zhuoman Liu Weicai Ye Yan Luximon Pengfei Wan Di Zhang VGen AI4CE 113 2 0 21 Nov 2024
OnlyFlow: Optical Flow based Motion Conditioning for Video Diffusion Models Mathis Koroglu Hugo Caselles-Dupré Guillaume Jeanneret Sanmiguel Matthieu Cord VGen DiffM 20 1 0 15 Nov 2024
EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Video Generation Xiaofeng Wang Kang Zhao F. Liu Jiayu Wang Guosheng Zhao Xiaoyi Bao Zheng Hua Zhu Yingya Zhang Xingang Wang VGen 56 6 0 13 Nov 2024
Artificial Intelligence for Biomedical Video Generation Linyuan Li Jianing Qiu Anujit Saha Lin Li Poyuan Li Mengxian He Ziyu Guo Wu Yuan VGen 55 1 0 12 Nov 2024
I2VControl-Camera: Precise Video Camera Control with Adjustable Motion Strength Wanquan Feng Jiawei Liu Pengqi Tu Tianhao Qi Mingzhen Sun Tianxiang Ma Songtao Zhao Siyu Zhou Qian He VGen 47 7 0 10 Nov 2024
ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuning David Junhao Zhang Roni Paiss Shiran Zada Nikhil Karnad David E. Jacobs Yael Pritch Inbar Mosseri Mike Zheng Shou Neal Wadhwa Nataniel Ruiz DiffM VGen 66 15 0 07 Nov 2024
DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion Wenqiang Sun Shuo Chen F. Liu Zilong Chen Yueqi Duan Jun Zhang Yikai Wang VGen 46 31 0 07 Nov 2024
MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views Yuedong Chen Chuanxia Zheng Haofei Xu Bohan Zhuang Andrea Vedaldi Tat-Jen Cham Jianfei Cai 3DGS 53 14 0 07 Nov 2024
SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generation Koichi Namekata Sherwin Bahmani Ziyi Wu Yash Kant Igor Gilitschenski David B. Lindell VGen 57 13 0 07 Nov 2024
GenXD: Generating Any 3D and 4D Scenes Yuyang Zhao Chung-Ching Lin Kevin Qinghong Lin Zhiwen Yan Linjie Li Z. Yang Jianfeng Wang G. Lee Lijuan Wang VGen 43 14 0 04 Nov 2024
Enhancing Motion in Text-to-Video Generation with Decomposed Encoding and Conditioning Penghui Ruan Pichao Wang Divya Saxena Jiannong Cao Yuhui Shi DiffM VGen 31 0 0 31 Oct 2024
Framer: Interactive Frame Interpolation Wen Wang Qiuyu Wang Kecheng Zheng Hao Ouyang Zhekai Chen Biao Gong Hao Chen Yujun Shen Chunhua Shen VGen 61 4 0 24 Oct 2024
VistaDream: Sampling multiview consistent images for single-view scene reconstruction Haiping Wang Yuan-Bin Liu Ziwei Liu Wenping Wang Z. Dong Bisheng Yang 45 11 0 22 Oct 2024
DreamVideo-2: Zero-Shot Subject-Driven Video Customization with Precise Motion Control Yujie Wei Shiwei Zhang Hangjie Yuan Xiang Wang Haonan Qiu ... F. Liu Zhizhong Huang Jiaxin Ye Yingya Zhang Hongming Shan DiffM VGen 69 14 0 17 Oct 2024
Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning Aakanksha Arash Ahmadian Seraphina Goldfarb-Tarrant B. Ermiş Marzieh Fadaee Sara Hooker MoMe 61 11 0 14 Oct 2024
Enhancing JEPAs with Spatial Conditioning: Robust and Efficient Representation Learning Etai Littwin Vimal Thilak Anand Gopalakrishnan 37 8 0 14 Oct 2024
Compressing Scene Dynamics: A Generative Approach Shanzhi Yin Zihan Zhang Bolin Chen Shiqi Wang Yan Ye VGen 29 0 0 13 Oct 2024
ByTheWay: Boost Your Text-to-Video Generation Model to Higher Quality in a Training-free Way Jiazi Bu Pengyang Ling Pan Zhang Tong Wu Xiaoyi Dong Yuhang Zang Yuhang Cao Dahua Lin Jiaqi Wang DiffM VGen 28 0 0 08 Oct 2024
ChatCam: Empowering Camera Control through Conversational AI Xinhang Liu Yu-Wing Tai Chi-Keung Tang VGen 31 2 0 25 Sep 2024
Multi-Modal Generative AI: Multi-modal LLM, Diffusion and Beyond Hong Chen Xin Wang Yuwei Zhou Bin Huang Yipeng Zhang Wei Feng Houlun Chen Zeyang Zhang Siao Tang Wenwu Zhu DiffM 44 7 0 23 Sep 2024
SpaceBlender: Creating Context-Rich Collaborative Spaces Through Generative 3D Scene Blending Nels Numan Shwetha Rajaram Balasaravanan Thoravi Kumaravel Nicolai Marquardt A. D. Wilson 26 1 0 20 Sep 2024
MyGo: Consistent and Controllable Multi-View Driving Video Generation with Camera Control Yining Yao Xi Guo C. Ding Wei Yu Wu VGen 29 2 0 10 Sep 2024
ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis Wangbo Yu Jinbo Xing Li Yuan Wenbo Hu Xiaoyu Li Zhipeng Huang Xiangjun Gao T. Wong Ying Shan Yonghong Tian VGen DiffM 42 77 0 03 Sep 2024