Towards Accurate Generative Models of Video: A New Metric & Challenges

3 December 2018

Thomas Unterthiner

Sjoerd van Steenkiste

Papers citing "Towards Accurate Generative Models of Video: A New Metric & Challenges"

50 / 132 papers shown

Title
PAHA: Parts-Aware Audio-Driven Human Animation with Diffusion Model Y.B. Wang S.Z. Zhou J.F. Wu T. Hu J.N. Zhang Z. Li Yanzhe Liu DiffM VGen 53 0 0 06 May 2025
Learning 3D Persistent Embodied World Models Siyuan Zhou Yilun Du Yuncong Yang Lei Han Peihao Chen Dit-Yan Yeung Chuang Gan VGen 47 0 0 05 May 2025
ReVision: High-Quality, Low-Cost Video Generation with Explicit 3D Physics Modeling for Complex Motion and Interaction Qihao Liu Ju He Qihang Yu Liang-Chieh Chen Alan Yuille DiffM VGen 78 0 0 30 Apr 2025
Direct Motion Models for Assessing Generated Videos Kelsey R. Allen Carl Doersch Guangyao Zhou Mohammed Suhail Danny Driess ... Thomas Kipf Mehdi S. M. Sajjadi Kevin P. Murphy João Carreira Sjoerd van Steenkiste EGVM DiffM VGen 74 0 0 30 Apr 2025
MagicPortrait: Temporally Consistent Face Reenactment with 3D Geometric Guidance Mengting Wei Yante Li Tuomas Varanka Yan Jiang Licai Sun DiffM VGen 62 0 0 30 Apr 2025
DiVE: Efficient Multi-View Driving Scenes Generation Based on Video Diffusion Transformer Junpeng Jiang Gangyi Hong Miao Zhang Hengtong Hu Kun Zhan Rui Shao Liqiang Nie VGen 51 0 0 28 Apr 2025
IM-Portrait: Learning 3D-aware Video Diffusion for Photorealistic Talking Heads from Monocular Videos Yuan Li Ziqian Bai Feitong Tan Zhaopeng Cui S. Fanello Yinda Zhang DiffM VGen 52 0 0 27 Apr 2025
NoiseController: Towards Consistent Multi-view Video Generation via Noise Decomposition and Collaboration Haotian Dong X. Wang D. Lin Yipeng Wu Qin Chen R. Liu Kairui Yang Ping Li Qing-Wu Guo VGen 42 0 0 25 Apr 2025
Solving New Tasks by Adapting Internet Video Knowledge Calvin Luo Zilai Zeng Yilun Du Chen Sun 21 0 0 21 Apr 2025
Vivid4D: Improving 4D Reconstruction from Monocular Video by Video Inpainting Jiaxin Huang Sheng Miao BangBnag Yang Yuewen Ma Yiyi Liao VGen MDE 27 0 0 15 Apr 2025
VideoPanda: Video Panoramic Diffusion with Multi-view Attention Kevin Xie Amirmojtaba Sabour Jiahui Huang Despoina Paschalidou G. Klár Umar Iqbal Sanja Fidler Xiaohui Zeng VGen MDE 34 0 0 15 Apr 2025
Seaweed-7B: Cost-Effective Training of Video Generation Foundation Model Team Seawead Ceyuan Yang Zhijie Lin Yang Zhao Shanchuan Lin ... Zuquan Song Zhenheng Yang Jiashi Feng Jianchao Yang Lu Jiang DiffM 81 1 0 11 Apr 2025
DyDiT++: Dynamic Diffusion Transformers for Efficient Visual Generation Wangbo Zhao Yizeng Han Jiasheng Tang Kai Wang Hao Luo Yibing Song Gao Huang Fan Wang Yang You 66 0 0 09 Apr 2025
Video-Bench: Human-Aligned Video Generation Benchmark Hui Han Siyuan Li Jiaqi Chen Yiwen Yuan Yuling Wu ... Y. Li J. Zhang Chi Zhang Li Li Yongxin Ni EGVM VGen 68 0 0 07 Apr 2025
Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generation Fa-Ting Hong Zunnan Xu Zixiang Zhou Jun Zhou Xiu Li Qin Lin Qinglin Lu D. Xu DiffM VGen 54 2 0 03 Apr 2025
CoGen: 3D Consistent Video Generation via Adaptive Conditioning for Autonomous Driving Yishen Ji Ziyue Zhu Zhenxin Zhu Kaixin Xiong Ming Lu Zhiqi Li Lijun Zhou Haiyang Sun Bing Wang Tong Lu VGen 53 1 0 28 Mar 2025
AdaWorld: Learning Adaptable World Models with Latent Actions Shenyuan Gao Siyuan Zhou Yilun Du Jun Zhang Chuang Gan VGen 57 3 0 24 Mar 2025
Ultrasound Image-to-Video Synthesis via Latent Dynamic Diffusion Models Tingxiu Chen Yilei Shi Zixuan Zheng Bingcong Yan Jingliang Hu Xiao Xiang Zhu Lichao Mou VGen MedIm 49 3 0 19 Mar 2025
Reangle-A-Video: 4D Video Generation as Video-to-Video Translation Hyeonho Jeong Suhyeon Lee Jong Chul Ye VGen 125 0 0 12 Mar 2025
VACT: A Video Automatic Causal Testing System and a Benchmark Haotong Yang Qingyuan Zheng Yunjian Gao Yongkun Yang Yangbo He Zhouchen Lin Muhan Zhang VGen CML 59 0 0 08 Mar 2025
Get In Video: Add Anything You Want to the Video Shaobin Zhuang Zhipeng Huang Binxin Yang Ying Zhang Fangyikang Wang Canmiao Fu Chong Sun Zheng-Jun Zha Chen Li Y. Wang DiffM VGen 49 0 0 08 Mar 2025
Learning to Animate Images from A Few Videos to Portray Delicate Human Actions Haoxin Li Yingchen Yu Qilong Wu Hanwang Zhang Boyang Li Song Bai 3DH VGen 120 0 0 01 Mar 2025
Raccoon: Multi-stage Diffusion Training with Coarse-to-Fine Curating Videos Zhiyu Tan Junyan Wang Hao Yang Luozheng Qin Hesen Chen Qiang-feng Zhou Hao Li VGen 64 0 0 28 Feb 2025
X-Dancer: Expressive Music to Human Dance Video Generation Zeyuan Chen Hongyi Xu Guoxian Song You Xie Chenxu Zhang X. Chen Chao Wang Di Chang Linjie Luo VGen 39 0 0 24 Feb 2025
HumanGif: Single-View Human Diffusion with Generative Prior Shoukang Hu Takuya Narihira Kazumi Fukuda Ryosuke Sawata Takashi Shibuya Yuki Mitsufuji 90 1 0 24 Feb 2025
MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation Sihyun Yu Meera Hahn Dan Kondratyuk Jinwoo Shin Agrim Gupta José Lezama Irfan Essa David A. Ross Jonathan Huang DiffM VGen 72 0 0 18 Feb 2025
SayAnything: Audio-Driven Lip Synchronization with Conditional Video Diffusion Junxian Ma Shiwen Wang Jian Yang Junyi Hu Jian Liang Guosheng Lin Jingbo Chen Kai Li Yu Meng DiffM VGen 61 3 0 17 Feb 2025
Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance Li Hu Guangyuan Wang Zhen Shen Xin Gao Dechao Meng Lian Zhuo Peng Zhang Bang Zhang Liefeng Bo DiffM VGen 93 8 0 10 Feb 2025
History-Guided Video Diffusion Kiwhan Song Boyuan Chen Max Simchowitz Yilun Du Russ Tedrake Vincent Sitzmann VGen 109 7 0 10 Feb 2025
VFX Creator: Animated Visual Effect Generation with Controllable Diffusion Transformer Xinyu Liu Ailing Zeng Wei Xue Harry Yang Wenhan Luo Qifeng Liu Yike Guo VGen 151 0 0 09 Feb 2025
MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generation Haibo Tong Zhaoyang Wang Z. Chen Haonian Ji Shi Qiu ... Peng Xia Mingyu Ding Rafael Rafailov Chelsea Finn Huaxiu Yao EGVM VGen 95 2 0 03 Feb 2025
Improving Tropical Cyclone Forecasting With Video Diffusion Models Zhibo Ren Pritthijit Nath Pancham Shukla 38 0 0 27 Jan 2025
MEt3R: Measuring Multi-View Consistency in Generated Images Mohammad Asim Christopher Wewer Thomas Wimmer Bernt Schiele J. E. Lenssen EGVM 3DGS VGen 46 7 0 10 Jan 2025
VideoAuteur: Towards Long Narrative Video Generation Junfei Xiao Feng Cheng Lu Qi Liangke Gui Jiepeng Cen Zhibei Ma Alan L. Yuille Lu Jiang VGen 56 2 0 10 Jan 2025
DriveEditor: A Unified 3D Information-Guided Framework for Controllable Object Editing in Driving Scenes Yiyuan Liang Zhiying Yan Liqun Chen Jiahuan Zhou Luxin Yan Sheng Zhong Xu Zou DiffM VGen 30 0 0 31 Dec 2024
VidTwin: Video VAE with Decoupled Structure and Dynamics Yuchi Wang Junliang Guo Xinyi Xie Tianyu He Xu Sun Jiang Bian DRL VGen 73 3 0 23 Dec 2024
Label-Efficient Data Augmentation with Video Diffusion Models for Guidewire Segmentation in Cardiac Fluoroscopy Shaoyan Pan Yikang Liu Lin Zhao Eric Z. Chen Xiao Chen Terrence Chen Shanhui Sun VGen MedIm 90 0 0 20 Dec 2024
Parallelized Autoregressive Visual Generation Y. Wang Shuhuai Ren Zhijie Lin Yujin Han Haoyuan Guo Zhenheng Yang Difan Zou Jiashi Feng Xihui Liu VGen 84 11 0 19 Dec 2024
AniDoc: Animation Creation Made Easier Yihao Meng Hao Ouyang Hanlin Wang Qiuyu Wang Wen Wang Ka Leong Cheng Zhiheng Liu Yujun Shen Huamin Qu DiffM VGen 104 5 0 18 Dec 2024
Can video generation replace cinematographers? Research on the cinematic language of generated video X. Li Kai WU Siyi Yang YiZhan Qu Guohua. Zhang ... Mingliang Xiong Hao Deng Qingwen Liu Gang Li Bin He VGen DiffM 90 1 0 16 Dec 2024
Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer Jiahao Cui Hui Li Yun Zhan Hanlin Shang K. Cheng Yuqi Ma Shan Mu Hang Zhou Jingdong Wang Siyu Zhu ViT VGen 99 6 0 01 Dec 2024
Human-Activity AGV Quality Assessment: A Benchmark Dataset and an Objective Evaluation Metric Zhichao Zhang Wei Sun Xinyue Li Yunhao Li Qihang Ge ... Zhongpeng Ji Fengyu Sun Shangling Jui Xiongkuo Min Guangtao Zhai EGVM 117 1 0 25 Nov 2024
SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generation Koichi Namekata Sherwin Bahmani Ziyi Wu Yash Kant Igor Gilitschenski David B. Lindell VGen 57 13 0 07 Nov 2024
TPC: Test-time Procrustes Calibration for Diffusion-based Human Image Animation Sunjae Yoon Gwanhyeong Koo Younghwan Lee Chang-Dong Yoo VGen 72 3 0 31 Oct 2024
Restructuring Vector Quantization with the Rotation Trick Christopher Fifty Ronald G. Junkins Dennis Duan Aniketh Iger Jerry W. Liu Ehsan Amid Sebastian Thrun Christopher Ré LLMSV 43 11 0 08 Oct 2024
ECHOPulse: ECG controlled echocardio-grams video generation Yiwei Li Sekeun Kim Zihao Wu Hanqi Jiang Yi Pan ... Sifan Song Yucheng Shi Tianming Liu Quanzheng Li Xiang Li VGen 24 1 0 04 Oct 2024
Loong: Generating Minute-level Long Videos with Autoregressive Language Models Yuqing Wang Tianwei Xiong Daquan Zhou Zhijie Lin Yang Zhao Bingyi Kang Jiashi Feng Xihui Liu VGen 46 23 0 03 Oct 2024
Replace Anyone in Videos Xiang Wang Shiwei Zhang Haonan Qiu Ruihang Chu Zekun Li Y. Zhang Changxin Gao Yuehuan Wang Chunhua Shen Nong Sang VGen DiffM 64 1 0 30 Sep 2024
High Quality Human Image Animation using Regional Supervision and Motion Blur Condition Zhongcong Xu Chaoyue Song Guoxian Song Jianfeng Zhang Jun Hao Liew ... You Xie Linjie Luo Guosheng Lin Jiashi Feng Mike Zheng Shou DiffM 3DH VGen 25 3 0 29 Sep 2024
A Simple but Strong Baseline for Sounding Video Generation: Effective Adaptation of Audio and Video Diffusion Models for Joint Generation Masato Ishii Akio Hayakawa Takashi Shibuya Yuki Mitsufuji VGen DiffM 63 4 0 26 Sep 2024