Phenaki: Variable Length Video Generation From Open Domain Textual Description

5 October 2022

Ruben Villegas

Mohammad Babaeizadeh

Pieter-Jan Kindermans

Papers citing "Phenaki: Variable Length Video Generation From Open Domain Textual Description"

50 / 287 papers shown

Title
LidarDM: Generative LiDAR Simulation in a Generated World Vlas Zyrianov Henry Che Zhijian Liu Shenlong Wang VGen 25 20 0 03 Apr 2024
Video Interpolation with Diffusion Models Siddhant Jain Daniel Watson Eric Tabellion Aleksander Holyñski Ben Poole Janne Kontkanen SupR VGen DiffM 30 32 0 01 Apr 2024
Frame by Familiar Frame: Understanding Replication in Video Diffusion Models Aimon Rahman Malsha V. Perera Vishal M. Patel VGen 40 7 0 28 Mar 2024
BAMM: Bidirectional Autoregressive Motion Model Ekkasit Pinyoanuntapong Muhammad Usama Saleem Pu Wang Minwoo Lee Srijan Das C. L. P. Chen VGen 27 22 0 28 Mar 2024
A Survey on Long Video Generation: Challenges, Methods, and Prospects Chengxuan Li Di Huang Zeyu Lu Yang Xiao Qingqi Pei Lei Bai EGVM 34 19 0 25 Mar 2024
Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition Sihyun Yu Weili Nie De-An Huang Boyi Li Jinwoo Shin A. Anandkumar VGen DiffM 24 15 0 21 Mar 2024
StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text Roberto Henschel Levon Khachatryan Daniil Hayrapetyan Hayk Poghosyan Vahram Tadevosyan Zhangyang Wang Shant Navasardyan Humphrey Shi DiffM VGen 91 77 0 21 Mar 2024
Mora: Enabling Generalist Video Generation via A Multi-Agent Framework Zhengqing Yuan Ruoxi Chen Zhaoxu Li Haolong Jia Lifang He Chi Wang Lichao Sun VGen 53 27 0 20 Mar 2024
VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis Enric Corona Andrei Zanfir Eduard Gabriel Bazavan Nikos Kolotouros Thiemo Alldieck C. Sminchisescu VGen DiffM 35 26 0 13 Mar 2024
Scaling Instructable Agents Across Many Simulated Worlds Sima Team Maria Abi Raad Arun Ahuja Catarina Barros F. Besse ... Daan Wierstra Duncan Williams Nathaniel Wong Sarah York Nick Young LM&Ro 107 38 0 13 Mar 2024
AesopAgent: Agent-driven Evolutionary System on Story-to-Video Production Jiuniu Wang Zehua Du Yuyuan Zhao Bo Yuan Kexiang Wang ... Yihen Lu Gengliang Li Junlong Gao Xin Tu Zhenyu Guo LLMAG VGen 28 7 0 12 Mar 2024
DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation Guosheng Zhao Xiaofeng Wang Zheng Zhu Xinze Chen Guan Huang Xiaoyi Bao Xingang Wang VGen 33 14 0 11 Mar 2024
Sora as an AGI World Model? A Complete Survey on Text-to-Video Generation Joseph Cho Fachrina Dewi Puspitasari Sheng Zheng Jingyao Zheng Lik-Hang Lee Tae-Ho Kim Choong Seon Hong Chaoning Zhang EGVM VGen 36 40 0 08 Mar 2024
UniCtrl: Improving the Spatiotemporal Consistency of Text-to-Video Diffusion Models via Training-Free Unified Attention Control Xuweiyi Chen Tian Xia Sihan Xu VGen DiffM 32 7 0 04 Mar 2024
Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models Yixin Liu Kai Zhang Yuan Li Zhiling Yan Chujie Gao ... Yue Huang Hanchi Sun Jianfeng Gao Lifang He Lichao Sun VLM VGen EGVM 68 254 0 27 Feb 2024
Genie: Generative Interactive Environments Jake Bruce Michael Dennis Ashley D. Edwards Jack Parker-Holder Yuge Shi ... Konrad Zolna Jeff Clune Nando de Freitas Satinder Singh Tim Rocktaschel VGen VLM 61 142 0 23 Feb 2024
Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesis Willi Menapace Aliaksandr Siarohin Ivan Skorokhodov Ekaterina Deyneka Tsai-Shien Chen ... Yuwei Fang A. Stoliar Elisa Ricci Jian Ren Sergey Tulyakov VGen 38 56 0 22 Feb 2024
Customize-A-Video: One-Shot Motion Customization of Text-to-Video Diffusion Models Yixuan Ren Yang Zhou Jimei Yang Jing Shi Difan Liu Feng Liu Mingi Kwon Abhinav Shrivastava DiffM VGen 94 34 0 22 Feb 2024
MobileSpeech: A Fast and High-Fidelity Framework for Mobile Zero-Shot Text-to-Speech Shengpeng Ji Ziyue Jiang Hanting Wang Jia-li Zuo Zhou Zhao 32 9 0 14 Feb 2024
Magic-Me: Identity-Specific Video Customized Diffusion Ze Ma Daquan Zhou Chun-Hsiao Yeh Xue-She Wang Xiuyu Li Huanrui Yang Zhen Dong Kurt Keutzer Jiashi Feng VGen DiffM 27 31 0 14 Feb 2024
World Model on Million-Length Video And Language With Blockwise RingAttention Hao Liu Wilson Yan Matei A. Zaharia Pieter Abbeel VGen 29 57 0 13 Feb 2024
Rolling Diffusion Models David Ruhe Jonathan Heek Tim Salimans Emiel Hoogeboom DiffM 23 32 0 12 Feb 2024
Animated Stickers: Bringing Stickers to Life with Video Diffusion David Yan Winnie Zhang Luxin Zhang Anmol Kalia Dingkang Wang ... Guan Pang Ali K. Thabet Peter Vajda Amy Bearman Licheng Yu VGen DiffM 46 2 0 08 Feb 2024
Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization Yang Jin Zhicheng Sun Kun Xu Kun Xu Liwei Chen ... Yuliang Liu Di Zhang Yang Song Kun Gai Yadong Mu VGen 47 42 0 05 Feb 2024
InteractiveVideo: User-Centric Controllable Video Generation with Synergistic Multimodal Instructions Yiyuan Zhang Yuhao Kang Zhixin Zhang Xiaohan Ding Sanyuan Zhao Xiangyu Yue VGen 52 4 0 05 Feb 2024
Boximator: Generating Rich and Controllable Motions for Video Synthesis Jiawei Wang Yuchen Zhang Jiaxin Zou Yan Zeng Guoqiang Wei Liping Yuan Hang Li DiffM VGen 27 43 0 02 Feb 2024
Lumiere: A Space-Time Diffusion Model for Video Generation Omer Bar-Tal Hila Chefer Omer Tov Charles Herrmann Roni Paiss ... T. Michaeli Oliver Wang Deqing Sun Tali Dekel Inbar Mosseri VGen 104 215 0 23 Jan 2024
ActAnywhere: Subject-Aware Video Background Generation Boxiao Pan Zhan Xu Chun-Hao Paul Huang Krishna Kumar Singh Yang Zhou Leonidas J. Guibas Jimei Yang VGen DiffM 24 3 0 19 Jan 2024
WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens Xiaofeng Wang Zheng Zhu Guan Huang Boyuan Wang Xinze Chen Jiwen Lu VGen 27 32 0 18 Jan 2024
CustomVideo: Customizing Text-to-Video Generation with Multiple Subjects Zhao Wang Aoxue Li Lingting Zhu Yong Guo Qi Dou Zhenguo Li VGen DiffM 22 40 0 18 Jan 2024
Vlogger: Make Your Dream A Vlog Shaobin Zhuang Kunchang Li Xinyuan Chen Yaohui Wang Ziwei Liu Yu Qiao Yali Wang VGen DiffM 17 34 0 17 Jan 2024
VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models Haoxin Chen Yong Zhang Xiaodong Cun Menghan Xia Xintao Wang Chao-Liang Weng Ying Shan VGen DiffM 115 274 0 17 Jan 2024
Collaboratively Self-supervised Video Representation Learning for Action Recognition Jie M. Zhang Zhifan Wan Lanqing Hu Stephen Lin Shuzhe Wu Shiguang Shan TTA 56 0 0 15 Jan 2024
RAVEN: Rethinking Adversarial Video Generation with Efficient Tri-plane Networks Partha Ghosh Soubhik Sanyal Cordelia Schmid Bernhard Scholkopf VGen 22 1 0 11 Jan 2024
FlashVideo: A Framework for Swift Inference in Text-to-Video Generation Bin Lei Le Chen Caiwen Ding VGen 20 1 0 30 Dec 2023
4DGen: Grounded 4D Content Generation with Spatial-temporal Consistency Yuyang Yin Dejia Xu Zhangyang Wang Yao-Min Zhao Yunchao Wei 3DGS 47 72 0 28 Dec 2023
Generative AI Beyond LLMs: System Implications of Multi-Modal Generation Alicia Golden Samuel Hsia Fei Sun Bilge Acun Basil Hosmer ... Zachary DeVito Jeff Johnson Gu-Yeon Wei David Brooks Carole-Jean Wu VLM DiffM 22 8 0 22 Dec 2023
VideoPoet: A Large Language Model for Zero-Shot Video Generation Dan Kondratyuk Lijun Yu Xiuye Gu José Lezama Jonathan Huang ... Irfan Essa Huisheng Wang David A. Ross Bryan Seybold Lu Jiang VGen 15 237 0 21 Dec 2023
InstructVideo: Instructing Video Diffusion Models with Human Feedback Hangjie Yuan Shiwei Zhang Xiang Wang Yujie Wei Tao Feng Yining Pan Yingya Zhang Ziwei Liu Samuel Albanie Dong Ni VGen 21 41 0 19 Dec 2023
MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers Haoyu Ma Shahin Mahdizadehaghdam Bichen Wu Zhipeng Fan Yuchao Gu Wenliang Zhao Lior Shapira Xiaohui Xie DiffM VGen 12 4 0 19 Dec 2023
VideoLCM: Video Latent Consistency Model Xiang Wang Shiwei Zhang Han Zhang Yu Liu Yingya Zhang Changxin Gao Nong Sang VGen DiffM 17 48 0 14 Dec 2023
Foundation Models in Robotics: Applications, Challenges, and the Future Roya Firoozi Johnathan Tucker Stephen Tian Anirudha Majumdar Jiankai Sun ... Brian Ichter Danny Driess Jiajun Wu Cewu Lu Mac Schwager LM&Ro AI4CE LRM VLM 35 137 0 13 Dec 2023
FreeInit: Bridging Initialization Gap in Video Diffusion Models Tianxing Wu Chenyang Si Yuming Jiang Ziqi Huang Ziwei Liu DiffM VGen 30 45 0 12 Dec 2023
Photorealistic Video Generation with Diffusion Models Agrim Gupta Lijun Yu Kihyuk Sohn Xiuye Gu Meera Hahn Fei-Fei Li Irfan Essa Lu Jiang José Lezama VGen 39 174 0 11 Dec 2023
Diffusion for Natural Image Matting Yihan Hu Yiheng Lin Wei Wang Yao-Min Zhao Yunchao Wei Humphrey Shi 18 7 0 10 Dec 2023
Generating Illustrated Instructions Sachit Menon Ishan Misra Rohit Girdhar DiffM 24 4 0 07 Dec 2023
MEVG: Multi-event Video Generation with Text-to-Video Models Gyeongrok Oh Jaehwan Jeong Sieun Kim Wonmin Byeon Jinkyu Kim Sungwoong Kim Sangpil Kim VGen DiffM 33 20 0 07 Dec 2023
AVID: Any-Length Video Inpainting with Diffusion Model Zhixing Zhang Bichen Wu Xiaoyan Wang Yaqiao Luo Luxin Zhang Yinan Zhao Peter Vajda Dimitris N. Metaxas Licheng Yu VGen DiffM 34 33 0 06 Dec 2023
Fine-grained Controllable Video Generation via Object Appearance and Context Hsin-Ping Huang Yu-Chuan Su Deqing Sun Lu Jiang Xuhui Jia Yukun Zhu Ming-Hsuan Yang DiffM VGen 13 13 0 05 Dec 2023
GIVT: Generative Infinite-Vocabulary Transformers Michael Tschannen Cian Eastwood Fabian Mentzer 10 34 0 04 Dec 2023