WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models

25 January 2024

Hongliang He

Wenlin Yao

Kaixin Ma

Wenhao Yu

Dong Yu

Papers citing "WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models"

33 / 33 papers shown

Title
WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch Zimu Lu Y. Yang Houxing Ren Haotian Hou Han Xiao Ke Wang Weikang Shi Aojun Zhou Mingjie Zhan H. Li LLMAG 41 0 0 06 May 2025
Self-Generated In-Context Examples Improve LLM Agents for Sequential Decision-Making Tasks Vishnu Sarukkai Zhiqiang Xie Kayvon Fatahalian LLMAG 68 0 0 01 May 2025
Visual Test-time Scaling for GUI Agent Grounding Tiange Luo Lajanugen Logeswaran Justin Johnson Honglak Lee 48 0 0 01 May 2025
WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Model Tianqing Fang H. M. Zhang Z. Zhang Kaixin Ma W. Yu Haitao Mi Dong Yu LLMAG KELM 104 0 0 23 Apr 2025
WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks Ivan Evtimov Arman Zharmagambetov Aaron Grattafiori Chuan Guo Kamalika Chaudhuri AAML 33 0 0 22 Apr 2025
Breaking the Data Barrier -- Building GUI Agents Through Task Generalization Junlei Zhang Zichen Ding Chang Ma Zijie Chen Qiushi Sun Zhenzhong Lan Junxian He 39 0 0 14 Apr 2025
An Illusion of Progress? Assessing the Current State of Web Agents Tianci Xue Weijian Qi Tianneng Shi Chan Hee Song Boyu Gou D. Song Huan Sun Yu Su LLMAG ELM 82 4 1 02 Apr 2025
Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks Lutfi Eren Erdogan Nicholas Lee Sehoon Kim Suhong Moon Hiroki Furuta Gopala Anumanchipalli K. K. Amir Gholami LLMAG LM&Ro AIFin 76 2 0 12 Mar 2025
AppAgentX: Evolving GUI Agents as Proficient Smartphone Users Wenjia Jiang Yangyang Zhuang Chenxi Song Xu Yang Chi Zhang Chi Zhang LLMAG 79 1 0 04 Mar 2025
Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agents Vardaan Pahuja Yadong Lu Corby Rosset Boyu Gou Arindam Mitra Spencer Whitehead Yu Su Ahmed Awadallah LLMAG LM&Ro Presented at ResearchTrend Connect \| LLMAG on 14 Mar 2025 145 3 1 20 Feb 2025
Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks Ang Li Yin Zhou Vethavikashini Chithrra Raghuram Tom Goldstein Micah Goldblum AAML 61 7 0 12 Feb 2025
WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning Zehan Qi Xiao-Chang Liu Iat Long Iong Hanyu Lai X. Sun ... Shuntian Yao Tianjie Zhang Wei Xu J. Tang Yuxiao Dong 93 14 0 28 Jan 2025
Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks Zhenhailong Wang Haiyang Xu Junyang Wang Xi Zhang Ming Yan J. Zhang Fei Huang Heng Ji 41 9 0 20 Jan 2025
Cognitive Kernel: An Open-source Agent System towards Generalist Autopilots H. Zhang Xiaoman Pan Hongwei Wang Kaixin Ma W. Yu Dong Yu LLMAG 52 3 0 03 Jan 2025
The BrowserGym Ecosystem for Web Agent Research Thibault Le Sellier De Chezelles Maxime Gasse Alexandre Lacoste Alexandre Drouin Massimo Caccia ... Siva Reddy Quentin Cappart Graham Neubig Ruslan Salakhutdinov Nicolas Chapados LLMAG 96 9 0 06 Dec 2024
Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks Adam Fourney Gagan Bansal Hussein Mozannar Cheng Tan Eduardo Salinas ... Victor C. Dibia Ahmed Hassan Awadallah Ece Kamar Rafah Hosn Saleema Amershi AI4CE LRM LLMAG 38 34 0 07 Nov 2024
Ferret-UI 2: Mastering Universal User Interface Understanding Across Platforms Zhangheng Li Keen You H. Zhang Di Feng Harsh Agrawal Xiujun Li Mohana Prasad Sathya Moorthy Jeff Nichols Y. Yang Zhe Gan MLLM 44 18 0 24 Oct 2024
Large Language Models Empowered Personalized Web Agents Hongru Cai Yongqi Li W. Wang Fengbin Zhu Xiaoyu Shen Wenjie Li Tat-Seng Chua LLMAG 41 12 0 22 Oct 2024
Beyond Browsing: API-Based Web Agents Yueqi Song Frank F. Xu Shuyan Zhou Graham Neubig 39 13 0 21 Oct 2024
From Grounding to Planning: Benchmarking Bottlenecks in Web Agents Segev Shlomov Ben wiesel Aviad Sela Ido Levy Liane Galanti Roy Abitbol LLMAG 30 3 0 03 Sep 2024
The Compressor-Retriever Architecture for Language Model OS Yuan Yang Siheng Xiong Ehsan Shareghi Faramarz Fekri RALM KELM 23 1 0 02 Sep 2024
Teams of LLM Agents can Exploit Zero-Day Vulnerabilities Richard Fang Antony Kellermann Akul Gupta Qiusi Zhan Richard Fang R. Bindu Daniel Kang LLMAG 33 27 0 02 Jun 2024
AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents Christopher Rawles Sarah Clinckemaillie Yifan Chang Jonathan Waltz Gabrielle Lau ... Daniel Toyama Robert Berry Divya Tyamagundlu Timothy Lillicrap Oriana Riva LLMAG 57 44 0 23 May 2024
MMInA: Benchmarking Multihop Multimodal Internet Agents Ziniu Zhang Shulin Tian Liangyu Chen Ziwei Liu LLMAG LM&Ro 27 13 0 15 Apr 2024
Tur[k]ingBench: A Challenge Benchmark for Web Agents Kevin Xu Yeganeh Kordi Kate Sanders Yizhong Wang Adam Byerly Kate Sanders Adam Byerly Jingyu Zhang Benjamin Van Durme Daniel Khashabi LLMAG 62 6 0 18 Mar 2024
WebLINX: Real-World Website Navigation with Multi-Turn Dialogue Xing Han Lù Zdeněk Kasner Siva Reddy 22 59 0 08 Feb 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 120 106 0 08 Feb 2024
SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents Kanzhi Cheng Qiushi Sun Yougang Chu Fangzhi Xu Yantao Li Jianbing Zhang Zhiyong Wu LLMAG 162 137 0 17 Jan 2024
Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding Kenton Lee Mandar Joshi Iulia Turc Hexiang Hu Fangyu Liu Julian Martin Eisenschlos Urvashi Khandelwal Peter Shaw Ming-Wei Chang Kristina Toutanova CLIP VLM 148 259 0 07 Oct 2022
ReAct: Synergizing Reasoning and Acting in Language Models Shunyu Yao Jeffrey Zhao Dian Yu Nan Du Izhak Shafran Karthik Narasimhan Yuan Cao LLMAG ReLM LRM 223 2,413 0 06 Oct 2022
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 207 1,089 0 20 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoning Jack Hessel Jena D. Hwang J. Park Rowan Zellers Chandra Bhagavatula Anna Rohrbach Kate Saenko Yejin Choi ReLM 139 48 0 10 Feb 2022