Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language

28 October 2021

Mingyu Ding

Chuang Gan

Papers citing "Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language"

50 / 55 papers shown

Title
Neuro-Symbolic Concepts Jiayuan Mao Joshua B. Tenenbaum Jiajun Wu NAI 19 0 0 09 May 2025
When Counterfactual Reasoning Fails: Chaos and Real-World Complexity Yahya Aalaila Gerrit Großmann Sumantrak Mukherjee Jonas Wahl Sebastian Vollmer CML LRM 47 0 0 31 Mar 2025
Prof. Robot: Differentiable Robot Rendering Without Static and Self-Collisions Quanyuan Ruan Jiabao Lei Wenhao Yuan Y. Zhang Dekun Lu Guiliang Liu Kui Jia 59 0 0 14 Mar 2025
Learning to Reason Iteratively and Parallelly for Complex Visual Reasoning Scenarios Shantanu Jaiswal Debaditya Roy Basura Fernando Cheston Tan ReLM LRM 66 2 0 20 Nov 2024
SPARTAN: A Sparse Transformer Learning Local Causation Anson Lei Bernhard Schölkopf Ingmar Posner 30 2 0 11 Nov 2024
Improving Viewpoint-Independent Object-Centric Representations through Active Viewpoint Selection Yinxuan Huang Chengmin Gao Bin Li Xiangyang Xue OCL 28 0 0 01 Nov 2024
Multi-granularity Contrastive Cross-modal Collaborative Generation for End-to-End Long-term Video Question Answering Ting Yu Kunhao Fu Jian Zhang Qingming Huang Jun Yu 25 2 0 12 Oct 2024
Vision-Language Models Assisted Unsupervised Video Anomaly Detection Yalong Jiang Liquan Mao 16 0 0 21 Sep 2024
An Investigation on The Position Encoding in Vision-Based Dynamics Prediction Jiageng Zhu Hanchen Xie Jiazhi Li Mahyar Khayatkhoei Wael AbdAlmageed 21 1 0 27 Aug 2024
Robo-GS: A Physics Consistent Spatial-Temporal Model for Robotic Arm with Hybrid Representation Haozhe Lou Yurong Liu Yike Pan Yiran Geng Jianteng Chen ... Lin Wang Hengzhen Feng Lu Shi Liyi Luo Yongliang Shi AI4CE 44 15 0 27 Aug 2024
One-shot Video Imitation via Parameterized Symbolic Abstraction Graphs Jianren Wang Kangni Liu Dingkun Guo Xian Zhou Christopher G Atkeson 20 0 0 22 Aug 2024
Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Models Amir Mohammad Karimi Mamaghan Samuele Papa Karl Henrik Johansson Stefan Bauer Andrea Dittadi OCL 35 5 0 22 Jul 2024
AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation Kaifeng Zhang Baoyu Li Kris Hauser Yunzhu Li AI4CE 28 14 0 10 Jul 2024
A Review of Differentiable Simulators Rhys Newbury Jack Collins Kerry He Jiahe Pan Ingmar Posner David Howard Akansel Cosgun AI4CE 31 9 0 08 Jul 2024
Guiding Video Prediction with Explicit Procedural Knowledge Patrick Takenaka Johannes Maucher Marco F. Huber 37 1 0 26 Jun 2024
Compositional 4D Dynamic Scenes Understanding with Physics Priors for Video Question Answering Xingrui Wang Wufei Ma Angtian Wang Shuo Chen Adam Kortylewski Alan L. Yuille 29 3 0 02 Jun 2024
Transformers and Slot Encoding for Sample Efficient Physical World Modelling Francesco Petri Luigi Asprino Aldo Gangemi OCL ViT 27 0 0 30 May 2024
STAR: A Benchmark for Situated Reasoning in Real-World Videos Bo Wu Shoubin Yu Zhenfang Chen Joshua B Tenenbaum Chuang Gan 31 176 0 15 May 2024
Human Motion Prediction under Unexpected Perturbation Jiangbei Yue Baiyi Li Julien Pettré Armin Seyfried He-Nan Wang 33 2 0 23 Mar 2024
Reasoning-Enhanced Object-Centric Learning for Videos Jian Li Pu Ren Yang Liu Hao-Lun Sun OCL LRM 33 2 0 22 Mar 2024
PhyGrasp: Generalizing Robotic Grasping with Physics-informed Large Multimodal Models Dingkun Guo Yuqi Xiang Shuqi Zhao Xinghao Zhu Masayoshi Tomizuka Mingyu Ding Wei Zhan 16 9 0 26 Feb 2024
ContPhy: Continuum Physical Concept Learning and Reasoning from Videos Zhicheng Zheng Xin Yan Zhenfang Chen Jingzhou Wang Qin Zhi Eddie Lim Joshua B. Tenenbaum Chuang Gan LRM 25 6 0 09 Feb 2024
STAIR: Spatial-Temporal Reasoning with Auditable Intermediate Results for Video Question Answering Yueqian Wang Yuxuan Wang Kai Chen Dongyan Zhao 25 2 0 08 Jan 2024
Commonsense for Zero-Shot Natural Language Video Localization Meghana Holla Ismini Lourentzou 21 2 0 29 Dec 2023
A Survey of Reasoning with Foundation Models Jiankai Sun Chuanyang Zheng E. Xie Zhengying Liu Ruihang Chu ... Xipeng Qiu Yi-Chen Guo Hui Xiong Qun Liu Zhenguo Li ReLM LRM AI4CE 19 74 0 17 Dec 2023
Benchmarks for Physical Reasoning AI Andrew Melnik Robin Schiewer Moritz Lange Andrei Muresanu Mozhgan Saeidi Animesh Garg Helge J. Ritter 16 8 0 17 Dec 2023
ACQUIRED: A Dataset for Answering Counterfactual Questions In Real-Life Videos Te-Lin Wu Zi-Yi Dou Qingyuan Hu Yu Hou Nischal Reddy Chandra Marjorie Freedman R. Weischedel Nanyun Peng 18 5 0 02 Nov 2023
CLEVRER-Humans: Describing Physical and Causal Events the Human Way Jiayuan Mao Xuelin Yang Xikun Zhang Noah D. Goodman Jiajun Wu NAI 9 22 0 05 Oct 2023
X-VoE: Measuring eXplanatory Violation of Expectation in Physical Events Bo Dai Linge Wang Baoxiong Jia Zeyu Zhang Song-Chun Zhu Chi Zhang Yixin Zhu 34 1 0 21 Aug 2023
Advancing Visual Grounding with Scene Knowledge: Benchmark and Method Zhihong Chen Ruifei Zhang Yibing Song Xiang Wan Guanbin Li 17 15 0 21 Jul 2023
Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Text Zhun Yang Adam Ishay Joohyung Lee LRM ELM 19 50 0 15 Jul 2023
Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties H. Tung Mingyu Ding Zhenfang Chen Daniel M. Bear Chuang Gan J. Tenenbaum Daniel L. K. Yamins Judy Fan Kevin A. Smith 58 13 0 27 Jun 2023
Physics-Informed Computer Vision: A Review and Perspectives C. Banerjee Kien Nguyen Clinton Fookes G. Karniadakis PINN AI4CE 25 27 0 29 May 2023
SlotDiffusion: Object-Centric Generative Modeling with Diffusion Models Ziyi Wu Jingyu Hu Wuyue Lu Igor Gilitschenski Animesh Garg DiffM OCL 23 44 0 18 May 2023
A Critical View of Vision-Based Long-Term Dynamics Prediction Under Environment Misalignment Hanchen Xie Jiageng Zhu Mahyar Khayatkhoei Jiazhi Li Mohamed E. Hussein Wael AbdAlmgaeed 22 3 0 12 May 2023
Embodied Concept Learner: Self-supervised Learning of Concepts and Mapping through Instruction Following Mingyu Ding Yan Xu Zhenfang Chen David D. Cox Ping Luo J. Tenenbaum Chuang Gan LM&Ro 41 20 0 07 Apr 2023
Visual Dependency Transformers: Dependency Tree Emerges from Reversed Attention Mingyu Ding Yikang Shen Lijie Fan Zhenfang Chen Z. Chen Ping Luo J. Tenenbaum Chuang Gan ViT 59 14 0 06 Apr 2023
Intrinsic Physical Concepts Discovery with Object-Centric Predictive Models Qu Tang Xiangyu Zhu Zhen Lei Zhaoxiang Zhang OCL 42 7 0 03 Mar 2023
Integrating Earth Observation Data into Causal Inference: Challenges and Opportunities Connor Jerzak Fredrik D. Johansson Adel Daoud CML 28 11 0 30 Jan 2023
See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoning Zhenfang Chen Qinhong Zhou Yikang Shen Yining Hong Hao Zhang Chuang Gan LRM VLM 26 35 0 12 Jan 2023
Masked Motion Encoding for Self-Supervised Video Representation Learning Xinyu Sun Peihao Chen Liang-Chieh Chen Chan Li Thomas H. Li Mingkui Tan Chuang Gan 27 28 0 12 Oct 2022
SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models Ziyi Wu Nikita Dvornik Klaus Greff Thomas Kipf Animesh Garg OCL BDL 59 87 0 12 Oct 2022
On the Learning Mechanisms in Physical Reasoning Shiqian Li Ke Wu Chi Zhang Yixin Zhu AI4CE 34 13 0 05 Oct 2022
Image-based Treatment Effect Heterogeneity Connor Jerzak Fredrik D. Johansson Adel Daoud 16 20 0 13 Jun 2022
Estimating Causal Effects Under Image Confounding Bias with an Application to Poverty in Africa Connor Jerzak Fredrik D. Johansson Adel Daoud CML 9 5 0 13 Jun 2022
Fixing Malfunctional Objects With Learned Physical Simulation and Functional Prediction Yining Hong Kaichun Mo L. Yi Leonidas J. Guibas Antonio Torralba J. Tenenbaum Chuang Gan 15 5 0 05 May 2022
ComPhy: Compositional Physical Reasoning of Objects and Events from Videos Zhenfang Chen Kexin Yi Yunzhu Li Mingyu Ding Antonio Torralba J. Tenenbaum Chuang Gan CoGe OCL 15 51 0 02 May 2022
A World-Self Model Towards Understanding Intelligence Yutao Yue 8 2 0 25 Mar 2022
Inferring Articulated Rigid Body Dynamics from RGBD Video Eric Heiden Ziang Liu Vibhav Vineet Erwin Coumans Gaurav Sukhatme PINN AI4CE 10 11 0 20 Mar 2022
Video Question Answering: Datasets, Algorithms and Challenges Yaoyao Zhong Junbin Xiao Wei Ji Yicong Li Wei Deng Tat-Seng Chua 10 83 0 02 Mar 2022