Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videos

31 March 2021

Papers citing "Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videos"

38 / 38 papers shown

Title
Pixel Motion as Universal Representation for Robot Control Kanchana Ranasinghe Xiang Li Cristina Mata J. Park Michael S. Ryoo VGen 27 0 0 12 May 2025
Chain-of-Modality: Learning Manipulation Programs from Multimodal Human Videos with Vision-Language-Models Chen Wang Fei Xia Wenhao Yu Tingnan Zhang Ruohan Zhang Ce Liu Li Fei-Fei Jie Tan Jacky Liang 31 0 0 17 Apr 2025
VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation Hanzhi Chen Boyang Sun Anran Zhang Marc Pollefeys Stefan Leutenegger LM&Ro 65 0 0 10 Mar 2025
Human2Robot: Learning Robot Actions from Paired Human-Robot Videos Sicheng Xie Haidong Cao Zejia Weng Zhen Xing Shiwei Shen Jiaqi Leng Xipeng Qiu Yanwei Fu Zuxuan Wu Yu Jiang 49 0 0 23 Feb 2025
CityWalker: Learning Embodied Urban Navigation from Web-Scale Videos Xinhao Liu J. Li Yichen Jiang Niranjan Sujay Z. Yang Juexiao Zhang John Abanes Jing Zhang Chen Feng 112 1 0 26 Nov 2024
Grounding Video Models to Actions through Goal Conditioned Exploration Yunhao Luo Yilun Du LM&Ro VGen 79 1 0 11 Nov 2024
VideoAgent: Self-Improving Video Generation Achint Soni Sreyas Venkataraman Abhranil Chandra Sebastian Fischmeister Percy Liang Bo Dai Sherry Yang LM&Ro VGen 52 7 0 14 Oct 2024
Learning Wheelchair Tennis Navigation from Broadcast Videos with Domain Knowledge Transfer and Diffusion Motion Planning Zixuan Wu Z. Zaidi Adithya Patil Qingyu Xiao Matthew C. Gombolay 67 0 0 29 Sep 2024
DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Control Zichen Jeff Cui Hengkai Pan Aadhithya Iyer Siddhant Haldar Lerrel Pinto VGen 24 10 0 18 Sep 2024
One-Shot Imitation under Mismatched Execution K. Kedia Prithwish Dan Sanjiban Choudhury Maximus Adrian Pace Sanjiban Choudhury 35 2 0 10 Sep 2024
Leveraging Unlabeled Data Sharing through Kernel Function Approximation in Offline Reinforcement Learning Yen-Ru Lai Fu-Chieh Chang Pei-Yuan Wu OffRL 64 1 0 22 Aug 2024
Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts Yanting Yang Minghao Chen Qibo Qiu Jiahao Wu Wenxiao Wang Binbin Lin Ziyu Guan Xiaofei He LM&Ro 37 2 0 20 Jul 2024
R+X: Retrieval and Execution from Everyday Human Videos Georgios Papagiannis Norman Di Palo Pietro Vitiello Edward Johns 51 15 0 17 Jul 2024
Dreamitate: Real-World Visuomotor Policy Learning via Video Generation Junbang Liang Ruoshi Liu Ege Ozguroglu Sruthi Sudhakar Achal Dave P. Tokmakov Shuran Song Carl Vondrick VGen 40 22 0 24 Jun 2024
RILe: Reinforced Imitation Learning Mert Albaba Sammy Christen Christoph Gebhardt Thomas Langarek Otmar Hilliges Otmar Hilliges 42 1 0 12 Jun 2024
Video-Language Critic: Transferable Reward Functions for Language-Conditioned Robotics Minttu Alakuijala Reginald McLean Isaac Woungang Nariman Farsad Samuel Kaski Pekka Marttinen Kai Yuan LM&Ro 34 0 0 30 May 2024
VICtoR: Learning Hierarchical Vision-Instruction Correlation Rewards for Long-horizon Manipulation Kuo-Han Hung Pang-Chi Lo Jia-Fong Yeh Han-Yuan Hsu Yi-Ting Chen Winston H. Hsu 28 0 0 26 May 2024
Human Demonstrations are Generalizable Knowledge for Robots Te Cui Guangyan Chen Tianxing Zhou Zicai Peng Mengxiao Hu Haoyang Lu Haizhou Li Meiling Wang Yi Yang Yufeng Yue LM&Ro 29 6 0 05 Dec 2023
AirExo: Low-Cost Exoskeletons for Learning Whole-Arm Manipulation in the Wild Hongjie Fang Haoshu Fang Yiming Wang Jieji Ren Jing Chen Ruo Zhang Weiming Wang Cewu Lu 32 46 0 26 Sep 2023
Language Reward Modulation for Pretraining Reinforcement Learning Ademi Adeniji Amber Xie Carmelo Sferrazza Younggyo Seo Stephen James Pieter Abbeel 32 26 0 23 Aug 2023
Structured World Models from Human Videos Russell Mendonca Shikhar Bahl Deepak Pathak LM&Ro 30 85 0 21 Aug 2023
Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning Jialong Wu Haoyu Ma Chao Deng Mingsheng Long OffRL 26 24 0 29 May 2023
Affordances from Human Videos as a Versatile Representation for Robotics Shikhar Bahl Russell Mendonca Lili Chen Unnat Jain Deepak Pathak 35 160 0 17 Apr 2023
Reward Design with Language Models Minae Kwon Sang Michael Xie Kalesha Bullard Dorsa Sadigh LM&Ro 25 198 0 27 Feb 2023
SIRL: Similarity-based Implicit Representation Learning Andreea Bobu Yi Liu Rohin Shah Daniel S. Brown Anca Dragan SSL DRL 19 17 0 02 Jan 2023
Learning Reward Functions for Robotic Manipulation by Observing Humans Minttu Alakuijala Gabriel Dulac-Arnold Julien Mairal Jean Ponce Cordelia Schmid OffRL 19 26 0 16 Nov 2022
Learning Object Manipulation Skills from Video via Approximate Differentiable Physics Vladimir Petrik M. N. Qureshi Josef Sivic Makarand Tapaswi 26 9 0 03 Aug 2022
Human-to-Robot Imitation in the Wild Shikhar Bahl Abhi Gupta Deepak Pathak 9 163 0 19 Jul 2022
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge Linxi Fan Guanzhi Wang Yunfan Jiang Ajay Mandlekar Yuncong Yang Haoyi Zhu Andrew Tang De-An Huang Yuke Zhu Anima Anandkumar LM&Ro 42 347 0 17 Jun 2022
A Generalist Agent Scott E. Reed Konrad Zolna Emilio Parisotto Sergio Gomez Colmenarejo Alexander Novikov ... Yutian Chen R. Hadsell Oriol Vinyals Mahyar Bordbar Nando de Freitas LM&Ro LLMAG AI4CE 54 783 0 12 May 2022
Reinforcement Learning with Action-Free Pre-Training from Videos Younggyo Seo Kimin Lee Stephen James Pieter Abbeel SSL OnRL 16 116 0 25 Mar 2022
Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformers A. Bucker Luis F. C. Figueredo Sami Haddadin Ashish Kapoor Shuang Ma Rogerio Bonatti LM&Ro 16 49 0 25 Mar 2022
Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks Bohan Wu Suraj Nair Li Fei-Fei Chelsea Finn OffRL LM&Ro 38 24 0 21 Sep 2021
Conservative Data Sharing for Multi-Task Offline Reinforcement Learning Tianhe Yu Aviral Kumar Yevgen Chebotar Karol Hausman Sergey Levine Chelsea Finn OffRL 19 78 0 16 Sep 2021
Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation Suraj Nair E. Mitchell Kevin Chen Brian Ichter Silvio Savarese Chelsea Finn LM&Ro OffRL 16 154 0 02 Sep 2021
Learning by Watching: Physical Imitation of Manipulation Skills from Human Videos Haoyu Xiong Quanzhou Li Yun-Chun Chen Homanga Bharadhwaj Samarth Sinha Animesh Garg SSL 119 93 0 18 Jan 2021
Model-Based Inverse Reinforcement Learning from Visual Demonstrations Neha Das Sarah Bechtle Todor Davchev Dinesh Jayaraman Akshara Rai Franziska Meier 111 82 0 18 Oct 2020
Scalable Multi-Task Imitation Learning with Autonomous Improvement Avi Singh Eric Jang A. Irpan Daniel Kappler Murtaza Dalal Sergey Levine Mohi Khansari Chelsea Finn 45 35 0 25 Feb 2020