Learning to Denoise Raw Mobile UI Layouts for Improving Datasets at Scale

11 January 2022

Papers citing "Learning to Denoise Raw Mobile UI Layouts for Improving Datasets at Scale"

37 / 37 papers shown

Title
PixelWeb: The First Web GUI Dataset with Pixel-Wise Labels Qi Yang Weichen Bi Haiyang Shen Y. Guo Yun Ma 32 0 0 23 Apr 2025
RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins Yao Mu Tianxing Chen Z. Chen Shijia Peng Zhiqian Lan ... Mingkun Xu Lunkai Lin Zhiqiang Xie Mingyu Ding Ping Luo 28 1 0 17 Apr 2025
AutoGameUI: Constructing High-Fidelity Game UIs via Multimodal Learning and Interactive Web-Based Tool Zhongliang Tang Mengchen Tan Fei Xia Qingrong Cheng Hao Jiang Y. Zhang 31 0 0 06 Nov 2024
WebRPG: Automatic Web Rendering Parameters Generation for Visual Presentation Zirui Shao Feiyu Gao Hangdi Xing Zepeng Zhu Zhi Yu Jiajun Bu Qi Zheng Cong Yao 28 2 0 22 Jul 2024
UICrit: Enhancing Automated Design Evaluation with a UICritique Dataset Peitong Duan Chin-yi Chen Gang Li Bjoern Hartmann Yang Li 53 7 0 11 Jul 2024
CoLay: Controllable Layout Generation through Multi-conditional Latent Diffusion Chin-Yi Cheng Ruiqi Gao Forrest Huang Yang Li DiffM 36 2 0 18 May 2024
Leveraging Human Revisions for Improving Text-to-Layout Models Amber Xie Chin-Yi Cheng Forrest Huang Yang Li 35 1 0 16 May 2024
MUD: Towards a Large-Scale and Noise-Filtered UI Dataset for Modern Style UI Modeling Sidong Feng Suyu Ma Han Wang David Kong Chunyang Chen 34 9 0 11 May 2024
Graph4GUI: Graph Neural Networks for Representing Graphical User Interfaces Yue Jiang Changkong Zhou Vikas Garg Antti Oulasvirta 25 7 0 21 Apr 2024
AI Assistance for UX: A Literature Review Through Human-Centered AI Yuwen Lu Yuewen Yang Qinyi Zhao Chengzhi Zhang Toby Jia-Jun Li 16 16 0 08 Feb 2024
ScreenAI: A Vision-Language Model for UI and Infographics Understanding Gilles Baechler Srinivas Sunkara Maria Wang Fedir Zubach Hassan Mansoor Vincent Etter Victor Carbune Jason Lin Jindong Chen Abhanshu Sharma 115 47 0 07 Feb 2024
GPTVoiceTasker: LLM-Powered Virtual Assistant for Smartphone Minh Duc Vu Han Wang Zhuang Li Jieshan Chen Shengdong Zhao Zhenchang Xing Chunyang Chen 26 3 0 25 Jan 2024
UI Layout Generation with LLMs Guided by UI Grammar Yuwen Lu Ziang Tong Qinyi Zhao Chengzhi Zhang Toby Jia-Jun Li 33 11 0 24 Oct 2023
Automatic Macro Mining from Interaction Traces at Scale Forrest Huang Gang Li Tao Li Yang Li 14 8 0 10 Oct 2023
Pairwise GUI Dataset Construction Between Android Phones and Tablets Han Hu Haolan Zhan Yujin Huang Di Liu 22 3 0 07 Oct 2023
Towards Automated Accessibility Report Generation for Mobile Apps Amanda Swearngin Jason Wu Xiaoyi Zhang Esteban Gomez Jen Coughenour ... Bhavya Garg Greg Hughes Adriana Hilliard Jeffrey P. Bigham Jeffrey Nichols 21 10 0 29 Sep 2023
EGFE: End-to-end Grouping of Fragmented Elements in UI Designs with Multimodal Learning Liuqing Chen Yunnong Chen Shuhong Xiao Yaxuan Song Lingyun Sun Yankun Zhen Tingting Zhou Yan-fang Chang 41 4 0 18 Sep 2023
Never-ending Learning of User Interfaces Jason Wu Rebecca Krosnick E. Schoop Amanda Swearngin Jeffrey P. Bigham Jeffrey Nichols VLM HAI 11 15 0 17 Aug 2023
Unveiling the Tricks: Automated Detection of Dark Patterns in Mobile Applications Jieshan Chen Jiamou Sun Sidong Feng Zhenchang Xing Qinghua Lu Xiwei Xu Chunyang Chen 19 25 0 11 Aug 2023
Video2Action: Reducing Human Interactions in Action Annotation of App Tutorial Videos Sidong Feng Chunyang Chen Zhenchang Xing 24 11 0 07 Aug 2023
A Pairwise Dataset for GUI Conversion and Retrieval between Android Phones and Tablets Han Hu Haolan Zhan Yujin Huang Di Liu 20 0 0 25 Jul 2023
GUILGET: GUI Layout GEneration with Transformer Andrey Sobolevsky Guillaume-Alexandre Bilodeau Jinghui Cheng Jin L. C. Guo 17 4 0 18 Apr 2023
WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semantics Jason Wu Siyan Wang Siman Shen Yi-Hao Peng Jeffrey Nichols Jeffrey P. Bigham 16 68 0 30 Jan 2023
PLay: Parametrically Conditioned Layout Generation using Latent Diffusion Ching-Yi Cheng Forrest Huang Gang Li Yang Li DiffM 19 29 0 27 Jan 2023
LayoutDETR: Detection Transformer Is a Good Multimodal Layout Designer Ning Yu Chia-Chih Chen Zeyuan Chen Rui Meng Ganglu Wu P. Josel Juan Carlos Niebles Caiming Xiong Ran Xu ViT DiffM 24 6 0 19 Dec 2022
UI Layers Group Detector: Grouping UI Layers via Text Fusion and Box Attention Shuhong Xiao Tingting Zhou Yunnong Chen Dengming Zhang Liuqing Chen Lingyun Sun Shiyu Yue 19 4 0 07 Dec 2022
Towards Better Semantic Understanding of Mobile Interfaces Srinivas Sunkara Maria Wang Lijuan Liu Gilles Baechler Yu-Chung Hsiao Jindong Chen Chen Abhanshu Sharma James Stout 21 21 0 06 Oct 2022
MUG: Interactive Multimodal Grounding on User Interfaces Tao Li Gang Li Jingjie Zheng Purple Wang Yang Li LLMAG 33 8 0 29 Sep 2022
Spotlight: Mobile UI Understanding using Vision-Language Models with a Focus Gang Li Yang Li 22 67 0 29 Sep 2022
Enabling Conversational Interaction with Mobile UI using Large Language Models Bryan Wang Gang Li Yang Li 175 132 0 18 Sep 2022
ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots Yu-Chung Hsiao Fedir Zubach Maria Wang Jindong Chen Victor Carbune Jason Lin Maria Wang Yun Zhu Jindong Chen RALM 157 25 0 16 Sep 2022
UI Layers Merger: Merging UI layers via Visual Learning and Boundary Prior Yunnong Chen Yankun Zhen Chuhao Shi Jiazhi Li Liuqing Chen Z. Li Lingyun Sun Tingting Zhou Yan-fang Chang 29 4 0 18 Jun 2022
SCENIC: A JAX Library for Computer Vision Research and Beyond Mostafa Dehghani A. Gritsenko Anurag Arnab Matthias Minderer Yi Tay 46 68 0 18 Oct 2021
Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots Jason Wu Xiaoyi Zhang Jeffrey Nichols Jeffrey P. Bigham 3DV 163 71 0 17 Sep 2021
Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning Bryan Wang Gang Li Xin Zhou Zhourong Chen Tovi Grossman Yang Li 164 152 0 07 Aug 2021
Understanding Mobile GUI: from Pixel-Words to Screen-Sentences Jingwen Fu Xiaoyi Zhang Yuwang Wang Wenjun Zeng Sam Yang Grayson Hilliard 21 14 0 25 May 2021
Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels Xiaoyi Zhang Lilian de Greef Amanda Swearngin Samuel White Kyle I. Murray ... Jeffrey Nichols Jason Wu Chris Fleizach Aaron Everitt Jeffrey P. Bigham 188 167 0 13 Jan 2021