v1v2 (latest)

PlaTe: Visually-Grounded Planning with Transformers in Procedural Tasks

10 September 2021

De-An Huang

Papers citing "PlaTe: Visually-Grounded Planning with Transformers in Procedural Tasks"

45 / 45 papers shown

Title
WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planning Delong Chen Willy Chung Yejin Bang Ziwei Ji Pascale Fung VGen LM&Ro 74 0 0 04 Jun 2025
Neuro Symbolic Knowledge Reasoning for Procedural Video Question Answering Thanh-Son Nguyen Hong Yang Tzeh Yuan Neoh Hao Zhang Ee Yeo Keat Basura Fernando NAI 101 0 0 19 Mar 2025
CLAD: Constrained Latent Action Diffusion for Vision-Language Procedure Planning Lei Shi Andreas Bulling DiffM 97 2 0 09 Mar 2025
Learning Human Skill Generators at Key-Step Levels Yilu Wu Chenhui Zhu Shuai Wang Hanlin Wang Jing Wang Zhaoxiang Zhang Limin Wang VGen 212 0 0 12 Feb 2025
SUTrack: Towards Simple and Unified Single Object Tracking Xin Chen Ben Kang Wanting Geng Jiawen Zhu Yebin Liu Dong Wang Huchuan Lu VOT ViT 108 1 0 26 Dec 2024
Advances in Transformers for Robotic Applications: A Review Nikunj Sanghai Nik Bear Brown AI4CE 148 0 0 13 Dec 2024
Human Action Anticipation: A Survey Bolin Lai Sam Toyer Tushar Nagarajan Rohit Girdhar S. Zha James M. Rehg Kris Kitani Kristen Grauman Ruta Desai Miao Liu AI4TS 80 1 0 17 Oct 2024
Propose, Assess, Search: Harnessing LLMs for Goal-Oriented Planning in Instructional Videos Md. Mohaiminul Islam Tushar Nagarajan Huiyu Wang Fu-Jen Chu Kris Kitani Gedas Bertasius Xitong Yang 72 4 0 30 Sep 2024
Hierarchical Hybrid Learning for Long-Horizon Contact-Rich Robotic Assembly Jiankai Sun Aidan Curtis Yang You Yan Xu Michael Koehle Leonidas Guibas Sachin Chitta Mac Schwager Hui Li 118 2 0 24 Sep 2024
Open-Event Procedure Planning in Instructional Videos Yilu Wu Hanlin Wang Jing Wang Limin Wang 93 1 0 06 Jul 2024
KOROL: Learning Visualizable Object Feature with Koopman Operator Rollout for Manipulation Hongyi Chen Abulikemu Abuduweili Aviral Agrawal Yunhai Han Harish Ravichandar Changliu Liu Jeffrey Ichnowski 117 6 0 29 Jun 2024
RAP: Retrieval-Augmented Planner for Adaptive Procedure Planning in Instructional Videos Ali Zare Yulei Niu Hammad A. Ayyubi Shih-Fu Chang 87 1 0 27 Mar 2024
Visual Action Planning with Multiple Heterogeneous Agents M. Lippi Michael C. Welle Marco Moletta Alessandro Marino Andrea Gasparri Danica Kragic 45 0 0 25 Mar 2024
ActionDiffusion: An Action-aware Diffusion Model for Procedure Planning in Instructional Videos Lei Shi Paul-Christian Bürkner Andreas Bulling DiffM VGen 83 4 0 13 Mar 2024
$$\text{R}^2$-Bench: Benchmarking the Robustness of Referring Perception Models under Perturbations$ $\text{R}^2$ -Bench: Benchmarking the Robustness of Referring Perception Models under Perturbations Xiang Li Kai Qiu Jinglu Wang Xiaohao Xu Rita Singh Kashu Yamazaki Hao Chen Xiaonan Huang Bhiksha Raj VOS 91 2 0 07 Mar 2024
Why Not Use Your Textbook? Knowledge-Enhanced Procedure Planning of Instructional Videos Kumaranage Ravindu Yasas Nagasinghe Honglu Zhou Malitha Gunawardhana Martin Renqiang Min Daniel Harari Muhammad Haris Khan 105 7 0 05 Mar 2024
SCHEMA: State CHangEs MAtter for Procedure Planning in Instructional Videos Yulei Niu Wenliang Guo Long Chen Xudong Lin Shih-Fu Chang 93 12 0 03 Mar 2024
CI w/o TN: Context Injection without Task Name for Procedure Planning Xinjie Li 73 0 0 23 Feb 2024
Foundation Models in Robotics: Applications, Challenges, and the Future Roya Firoozi Johnathan Tucker Stephen Tian Anirudha Majumdar Jiankai Sun ... Brian Ichter Danny Driess Jiajun Wu Cewu Lu Mac Schwager LM&Ro AI4CE LRM VLM 108 161 0 13 Dec 2023
Adaptive Online Replanning with Diffusion Models Siyuan Zhou Yilun Du Shun Zhang Mengdi Xu Yikang Shen Wei Xiao Dit-Yan Yeung Chuang Gan 90 24 0 14 Oct 2023
GePSAn: Generative Procedure Step Anticipation in Cooking Videos M. A. Abdelsalam Samrudhdhi B. Rangrej Isma Hadji Nikita Dvornik Konstantinos G. Derpanis Afsaneh Fazly AI4TS 61 7 0 12 Oct 2023
Learning Concept-Based Causal Transition and Symbolic Reasoning for Visual Planning Yilue Qian Peiyu Yu Ying Nian Wu Yao Su Wei Wang Lifeng Fan LRM 101 4 0 05 Oct 2023
How Physics and Background Attributes Impact Video Transformers in Robotic Manipulation: A Case Study on Planar Pushing Shutong Jin Ruiyu Wang Muhammad Zahid Florian T. Pokorny 68 2 0 03 Oct 2023
Skip-Plan: Procedure Planning in Instructional Videos via Condensed Action Space Learning Zhiheng Li Wenjia Geng Muheng Li Lei Chen Yansong Tang Jiwen Lu Jie Zhou 74 10 0 01 Oct 2023
Masked Diffusion with Task-awareness for Procedure Planning in Instructional Videos Fen Fang Yun Liu Ali Koksal Qianli Xu Joo-Hwee Lim VGen DiffM 76 6 0 14 Sep 2023
Event-Guided Procedure Planning from Instructional Videos with Text Supervision Ante Wang Kun-Li Channing Lin Jiachen Du Jingke Meng Wei-Shi Zheng 67 16 0 17 Aug 2023
AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos? Qi Zhao Shijie Wang Ce Zhang Changcheng Fu Minh Quan Do Nakul Agarwal Kwonjoon Lee Chen Sun LM&Ro 132 51 0 31 Jul 2023
Pretrained Language Models as Visual Planners for Human Assistance Dhruvesh Patel H. Eghbalzadeh Nitin Kamra Michael L. Iuzzolino Unnat Jain Ruta Desai LM&Ro 87 25 0 17 Apr 2023
Procedure-Aware Pretraining for Instructional Video Understanding Honglu Zhou Roberto Martín-Martín Mubbasir Kapadia Silvio Savarese Juan Carlos Niebles 123 40 0 31 Mar 2023
Planning with Sequence Models through Iterative Energy Minimization Hongyi Chen Yilun Du Yiye Chen J. Tenenbaum Patricio A. Vela 65 6 0 28 Mar 2023
PDPP: Projected Diffusion for Procedure Planning in Instructional Videos Hanlin Wang Yilu Wu Sheng Guo Limin Wang VGen DiffM 169 31 0 26 Mar 2023
Large AI Models in Health Informatics: Applications, Challenges, and the Future Jianing Qiu Lin Li Jiankai Sun Jiachuan Peng Peilun Shi ... Bo Xiao Wu Yuan Ningli Wang Dong Xu Benny Lo AI4MH LM&MA 114 140 0 21 Mar 2023
Foundation Models for Decision Making: Problems, Methods, and Opportunities Sherry Yang Ofir Nachum Yilun Du Jason W. Wei Pieter Abbeel Dale Schuurmans LM&Ro OffRL LRM AI4CE 184 172 0 07 Mar 2023
Action Dynamics Task Graphs for Learning Plannable Representations of Procedural Tasks Weichao Mao Ruta Desai Michael L. Iuzzolino Nitin Kamra 79 5 0 11 Jan 2023
A Survey on Transformers in Reinforcement Learning Wenzhe Li Hao Luo Zichuan Lin Chongjie Zhang Zongqing Lu Deheng Ye OffRL MU AI4CE 119 58 0 08 Jan 2023
On Transforming Reinforcement Learning by Transformer: The Development Trajectory Shengchao Hu Li Shen Ya Zhang Yixin Chen Dacheng Tao OffRL 148 30 0 29 Dec 2022
SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models Ziyi Wu Nikita Dvornik Klaus Greff Thomas Kipf Animesh Garg OCL BDL 140 95 0 12 Oct 2022
EgoTaskQA: Understanding Human Tasks in Egocentric Videos Baoxiong Jia Ting Lei Song-Chun Zhu Siyuan Huang EgoV 89 65 0 08 Oct 2022
See, Plan, Predict: Language-guided Cognitive Planning with Video Prediction Maria Attarian Advaya Gupta Ziyi Zhou Wei Yu Igor Gilitschenski Animesh Garg LM&Ro 76 8 0 07 Oct 2022
NeRF-Loc: Transformer-Based Object Localization Within Neural Radiance Fields Jiankai Sun Yan Xu Mingyu Ding Hongwei Yi Chen Wang Jingdong Wang Liangjun Zhang Mac Schwager 100 12 0 24 Sep 2022
Inner Monologue: Embodied Reasoning through Planning with Language Models Wenlong Huang F. Xia Ted Xiao Harris Chan Jacky Liang ... Tomas Jackson Linda Luu Sergey Levine Karol Hausman Brian Ichter LLMAG LM&Ro LRM 190 926 0 12 Jul 2022
Transformers are Adaptable Task Planners Vidhi Jain Yixin Lin Eric Undersander Yonatan Bisk Akshara Rai 113 24 0 06 Jul 2022
P3IV: Probabilistic Procedure Planning from Instructional Videos with Weak Supervision Henghui Zhao Isma Hadji Nikita Dvornik Konstantinos G. Derpanis Richard P. Wildes Allan D. Jepson 96 46 0 04 May 2022
LocATe: End-to-end Localization of Actions in 3D with Transformers Jiankai Sun Bolei Zhou Michael J. Black Arjun Chandrasekaran 143 8 0 21 Mar 2022
A Survey of Embodied AI: From Simulators to Research Tasks Jiafei Duan Samson Yu Tangyao Li Huaiyu Zhu Cheston Tan LM&Ro 118 296 0 08 Mar 2021