Title
TongUI: Building Generalized GUI Agents by Learning from Multimodal Web Tutorials Bofei Zhang Zirui Shang Zhi Gao Wang Zhang Rui Xie Xiaojian Ma Tao Yuan Xinxiao Wu Song-Chun Zhu Qing Li LLMAG 35 1 0 17 Apr 2025
Breaking the Data Barrier -- Building GUI Agents Through Task Generalization Junlei Zhang Zichen Ding Chang Ma Zijie Chen Qiushi Sun Zhenzhong Lan Junxian He 39 0 0 14 Apr 2025
GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents Run Luo Lu Wang Wanwei He Xiaobo Xia LLMAG 47 5 0 14 Apr 2025
Navi-plus: Managing Ambiguous GUI Navigation Tasks with Follow-up Ziming Cheng Zhiyuan Huang Junting Pan Zhaohui Hou Mingjie Zhan 38 0 0 31 Mar 2025
GUI-Xplore: Empowering Generalizable GUI Agents with One Exploration Yuchen Sun Shanhui Zhao Tao Yu Hao Wen Samith Va Mengwei Xu Yuanchun Li Chongyang Zhang LLMAG 62 0 0 22 Mar 2025
DeskVision: Large Scale Desktop Region Captioning for Advanced GUI Agents Yibin Xu Liang Yang Hao Chen Hua Wang Zhi Chen Yaohua Tang 3DV 56 0 0 14 Mar 2025
SpiritSight Agent: Advanced GUI Agent with One Look Zhiyuan Huang Ziming Cheng Junting Pan Zhaohui Hou Mingjie Zhan LLMAG 91 2 0 05 Mar 2025
Position: Standard Benchmarks Fail -- LLM Agents Present Overlooked Risks for Financial Applications Zichen Chen Jiaao Chen Jianda Chen Misha Sra ELM 34 1 0 21 Feb 2025
Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agents Vardaan Pahuja Yadong Lu Corby Rosset Boyu Gou Arindam Mitra Spencer Whitehead Yu Su Ahmed Awadallah LLMAG LM&Ro Presented at ResearchTrend Connect \| LLMAG on 14 Mar 2025 145 3 1 20 Feb 2025
Magma: A Foundation Model for Multimodal AI Agents Jianwei Yang Reuben Tan Qianhui Wu Ruijie Zheng Baolin Peng ... Seonghyeon Ye Joel Jang Yuquan Deng Lars Liden Jianfeng Gao VLM AI4TS 102 8 0 18 Feb 2025
InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflection Y. Liu Pengxiang Li Zishu Wei C. Xie Xueyu Hu Xinchen Xu Shengyu Zhang Xiaotian Han Hongxia Yang Fei Wu LLMAG LRM 53 11 0 08 Jan 2025
Aria-UI: Visual Grounding for GUI Instructions Yuhao Yang Yue Wang Dongxu Li Ziyang Luo Bei Chen C. Huang Junnan Li LM&Ro LLMAG 103 14 0 20 Dec 2024
Attention-driven GUI Grounding: Leveraging Pretrained Multimodal Large Language Models without Fine-Tuning Hai-Ming Xu Qi Chen Lei Wang Lingqiao Liu 62 1 0 14 Dec 2024
ShowUI: One Vision-Language-Action Model for GUI Visual Agent Kevin Qinghong Lin Linjie Li Difei Gao Z. Yang Shiwei Wu Zechen Bai Weixian Lei Lijuan Wang Mike Zheng Shou LLMAG 72 13 0 26 Nov 2024
OS-ATLAS: A Foundation Action Model for Generalist GUI Agents Zhiyong Wu Zhenyu Wu Fangzhi Xu Yian Wang Qiushi Sun ... Kanzhi Cheng Zichen Ding L. Chen Paul Pu Liang Yu Qiao 37 31 0 30 Oct 2024
MMDocBench: Benchmarking Large Vision-Language Models for Fine-Grained Visual Document Understanding Fengbin Zhu Ziyang Liu Xiang Yao Ng Haohui Wu W. Wang Fuli Feng Chao Wang Huanbo Luan Tat-Seng Chua VLM 32 3 0 25 Oct 2024
OSCAR: Operating System Control via State-Aware Reasoning and Re-Planning Xiaoqiang Wang Bang Liu LLMAG LM&Ro LRM 31 6 0 24 Oct 2024
VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents S. Yu C. Tang Bokai Xu Junbo Cui Junhao Ran ... Zhenghao Liu Shuo Wang Xu Han Zhiyuan Liu Maosong Sun VLM 31 21 0 14 Oct 2024
ClickAgent: Enhancing UI Location Capabilities of Autonomous Agents Jakub Hoscilowicz Bartosz Maj Bartosz Kozakiewicz Oleksii Tymoshchuk Artur Janicki LLMAG 47 5 0 09 Oct 2024
TinyClick: Single-Turn Agent for Empowering GUI Automation Pawel Pawlowski Krystian Zawistowski Wojciech Lapacz Marcin Skorupa Adam Wiacek Sebastien Postansque Jakub Hoscilowicz MLLM LLMAG LRM 35 6 0 09 Oct 2024
Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents Boyu Gou Ruohan Wang Boyuan Zheng Yanan Xie Cheng Chang Yiheng Shu Huan Sun Yu Su LM&Ro LLMAG 71 48 0 07 Oct 2024
Inferring Alt-text For UI Icons With Large Language Models During App Development Sabrina Haque Christoph Csallner VLM 31 0 0 26 Sep 2024
CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents Tianqi Xu Linyao Chen Dai-Jie Wu Yanjun Chen Zecheng Zhang ... Shilong Liu Bochen Qian Philip H. S. Torr Bernard Ghanem G. Li 38 14 0 01 Jul 2024
ScreenAI: A Vision-Language Model for UI and Infographics Understanding Gilles Baechler Srinivas Sunkara Maria Wang Fedir Zubach Hassan Mansoor Vincent Etter Victor Carbune Jason Lin Jindong Chen Abhanshu Sharma 112 47 0 07 Feb 2024
SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents Kanzhi Cheng Qiushi Sun Yougang Chu Fangzhi Xu Yantao Li Jianbing Zhang Zhiyong Wu LLMAG 162 137 0 17 Jan 2024
LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model Yichen Zhu Minjie Zhu Ning Liu Zhicai Ou Xiaofeng Mou Jian Tang 63 89 0 04 Jan 2024
V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs Penghao Wu Saining Xie LRM 49 120 0 21 Dec 2023
CogAgent: A Visual Language Model for GUI Agents Wenyi Hong Weihan Wang Qingsong Lv Jiazheng Xu Wenmeng Yu ... Juanzi Li Bin Xu Yuxiao Dong Ming Ding Jie Tang MLLM 137 310 0 14 Dec 2023
RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback M. Steyvers Yuan Yao Haoye Zhang Taiwen He Yifeng Han ... Xinyue Hu Zhiyuan Liu Hai-Tao Zheng Maosong Sun Tat-Seng Chua MLLM VLM 130 176 0 01 Dec 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 244 4,186 0 30 Jan 2023