Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks

17 June 2022

Papers citing "Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks"

50 / 327 papers shown

Title
Inquire, Interact, and Integrate: A Proactive Agent Collaborative Framework for Zero-Shot Multimodal Medical Reasoning Zishan Gu Fenglin Liu Changchang Yin Ping Zhang LRM LM&MA 38 0 0 19 May 2024
Libra: Building Decoupled Vision System on Large Language Models Yifan Xu Xiaoshan Yang Y. Song Changsheng Xu MLLM VLM 31 6 0 16 May 2024
UniCorn: A Unified Contrastive Learning Approach for Multi-view Molecular Representation Learning Shikun Feng Yuyan Ni Minghao Li Yanwen Huang Zhiming Ma Wei-Ying Ma Yanyan Lan SSL 41 7 0 15 May 2024
DocRes: A Generalist Model Toward Unifying Document Image Restoration Tasks Jiaxin Zhang Dezhi Peng Chongyu Liu Peirong Zhang Lianwen Jin VLM 30 12 0 07 May 2024
One-Stage Open-Vocabulary Temporal Action Detection Leveraging Temporal Multi-scale and Action Label Features Trung Thanh Nguyen Yasutomo Kawanishi Takahiro Komamizu Ichiro Ide VLM 24 3 0 30 Apr 2024
UniFS: Universal Few-shot Instance Perception with Point Representations Sheng Jin Ruijie Yao Lumin Xu Wentao Liu Chao Qian Ji Wu Ping Luo 40 2 0 30 Apr 2024
Chameleon: A Data-Efficient Generalist for Dense Visual Prediction in the Wild Donggyun Kim Seongwoong Cho Semin Kim Chong Luo Seunghoon Hong VLM 31 2 0 29 Apr 2024
What Makes Multimodal In-Context Learning Work? Folco Bertini Baldassini Mustafa Shukor Matthieu Cord Laure Soulier Benjamin Piwowarski 32 18 0 24 Apr 2024
In-Context Translation: Towards Unifying Image Recognition, Processing, and Generation Han Xue Qianru Sun Li-Na Song Wenjun Zhang Zhiwu Huang MLLM 36 0 0 15 Apr 2024
A Survey on Multimodal Wearable Sensor-based Human Action Recognition Jianyuan Ni Hao Tang Syed Tousiful Haque Yan Yan A. Ngu 66 5 0 14 Apr 2024
Connecting NeRFs, Images, and Text Francesco Ballerini Pierluigi Zama Ramirez Roberto Mirabella Samuele Salti Luigi Di Stefano 37 4 0 11 Apr 2024
GLID: Pre-training a Generalist Encoder-Decoder Vision Model Jihao Liu Jinliang Zheng Yu Liu Hongsheng Li VLM 19 3 0 11 Apr 2024
BRAVE: Broadening the visual encoding of vision-language models Ouguzhan Fatih Kar A. Tonioni Petra Poklukar Achin Kulshrestha Amir Zamir Federico Tombari MLLM VLM 42 25 0 10 Apr 2024
Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction Keyu Tian Yi-Xin Jiang Zehuan Yuan Bingyue Peng Liwei Wang VGen 25 248 0 03 Apr 2024
LocCa: Visual Pretraining with Location-aware Captioners Bo Wan Michael Tschannen Yongqin Xian Filip Pavetić Ibrahim M. Alabdulmohsin Xiao Wang André Susano Pinto Andreas Steiner Lucas Beyer Xiao-Qi Zhai VLM 40 5 0 28 Mar 2024
Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observation Zhitong Xiong Yi Wang Fahong Zhang Adam J. Stewart Joelle Hanna Damian Borth Ioannis Papoutsis B. L. Saux Gustau Camps-Valls Xiao Xiang Zhu AI4CE 69 12 0 22 Mar 2024
Lexicon-Level Contrastive Visual-Grounding Improves Language Modeling Chengxu Zhuang Evelina Fedorenko Jacob Andreas 24 2 0 21 Mar 2024
What if...?: Thinking Counterfactual Keywords Helps to Mitigate Hallucination in Large Multi-modal Models Junho Kim Yeonju Kim Yonghyun Ro LRM MLLM 29 4 0 20 Mar 2024
SC-Tune: Unleashing Self-Consistent Referential Comprehension in Large Vision Language Models Tongtian Yue Jie Cheng Longteng Guo Xingyuan Dai Zijia Zhao Xingjian He Gang Xiong Yisheng Lv Jing Liu 36 9 0 20 Mar 2024
A Versatile Framework for Multi-scene Person Re-identification Wei-Shi Zheng Junkai Yan Yi-Xing Peng VLM 32 5 0 17 Mar 2024
3D-VLA: A 3D Vision-Language-Action Generative World Model Haoyu Zhen Xiaowen Qiu Peihao Chen Jincheng Yang Xin Yan Yilun Du Yining Hong Chuang Gan LM&Ro VGen PINN 34 81 0 14 Mar 2024
GiT: Towards Generalist Vision Transformer through Universal Language Interface Haiyang Wang Hao Tang Li Jiang Shaoshuai Shi Muhammad Ferjad Naeem Hongsheng Li Bernt Schiele Liwei Wang VLM 27 10 0 14 Mar 2024
Explore In-Context Segmentation via Latent Diffusion Models Chaoyang Wang Xiangtai Li Henghui Ding Lu Qi Jiangning Zhang Yunhai Tong Chen Change Loy Shuicheng Yan DiffM 63 6 0 14 Mar 2024
Masked AutoDecoder is Effective Multi-Task Vision Generalist Han Qiu Jiaxing Huang Peng Gao Lewei Lu Xiaoqin Zhang Shijian Lu 35 3 0 12 Mar 2024
Lumen: Unleashing Versatile Vision-Centric Capabilities of Large Multimodal Models Yang Jiao Shaoxiang Chen Zequn Jie Jing Chen Lin Ma Yueping Jiang MLLM 29 18 0 12 Mar 2024
Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Prompts Jiawen Zhu Guansong Pang VLM 53 34 0 11 Mar 2024
Unlocking the Potential of Multimodal Unified Discrete Representation through Training-Free Codebook Optimization and Hierarchical Alignment Hai Huang Yan Xia Shengpeng Ji Shulei Wang Hanting Wang Jieming Zhu Zhenhua Dong Zhou Zhao 22 6 0 08 Mar 2024
Enhancing Vision-Language Pre-training with Rich Supervisions Yuan Gao Kunyu Shi Pengkai Zhu Edouard Belval Oren Nuriel Srikar Appalaraju Shabnam Ghadar Vijay Mahadevan Zhuowen Tu Stefano Soatto VLM CLIP 62 12 0 05 Mar 2024
NiNformer: A Network in Network Transformer with Token Mixing Generated Gating Function Abdullah Nazhat Abdullah Tarkan Aydin 23 0 0 04 Mar 2024
Non-autoregressive Sequence-to-Sequence Vision-Language Models Kunyu Shi Qi Dong Luis Goncalves Zhuowen Tu Stefano Soatto VLM 35 3 0 04 Mar 2024
GROUNDHOG: Grounding Large Language Models to Holistic Segmentation Yichi Zhang Ziqiao Ma Xiaofeng Gao Suhaila Shakiah Qiaozi Gao Joyce Chai MLLM VLM 30 38 0 26 Feb 2024
Where Do We Go from Here? Multi-scale Allocentric Relational Inference from Natural Spatial Descriptions Tzuf Paz-Argaman Sayali Kulkarni John Palowitch Jason Baldridge Reut Tsarfaty 19 3 0 26 Feb 2024
PIN: Positional Insert Unlocks Object Localisation Abilities in VLMs Michael Dorkenwald Nimrod Barazani Cees G. M. Snoek Yuki M. Asano VLM MLLM 25 12 0 13 Feb 2024
Real-World Robot Applications of Foundation Models: A Review Kento Kawaharazuka T. Matsushima Andrew Gambardella Jiaxian Guo Chris Paxton Andy Zeng OffRL VLM LM&Ro 41 45 0 08 Feb 2024
Data-efficient Large Vision Models through Sequential Autoregression Jianyuan Guo Zhiwei Hao Chengcheng Wang Yehui Tang Han Wu Han Hu Kai Han Chang Xu VLM 13 10 0 07 Feb 2024
Large Language Models for Time Series: A Survey Xiyuan Zhang Ranak Roy Chowdhury Rajesh K. Gupta Jingbo Shang AI4TS 77 53 0 02 Feb 2024
Enhancing Multimodal Large Language Models with Vision Detection Models: An Empirical Study Qirui Jiao Daoyuan Chen Yilun Huang Yaliang Li Ying Shen 15 12 0 31 Jan 2024
Q&A Prompts: Discovering Rich Visual Clues through Mining Question-Answer Prompts for VQA requiring Diverse World Knowledge Haibi Wang Weifeng Ge LRM 17 3 0 19 Jan 2024
OMG-Seg: Is One Model Good Enough For All Segmentation? Xiangtai Li Haobo Yuan Wei Li Henghui Ding Size Wu Wenwei Zhang Yining Li Kai Chen Chen Change Loy VLM MLLM ViT 69 48 0 18 Jan 2024
A Simple Latent Diffusion Approach for Panoptic Segmentation and Mask Inpainting Wouter Van Gansbeke Bert De Brabandere DiffM 22 11 0 18 Jan 2024
AntEval: Evaluation of Social Interaction Competencies in LLM-Driven Agents Yuanzhi Liang Linchao Zhu Yi Yang LLMAG 16 0 0 12 Jan 2024
CaMML: Context-Aware Multimodal Learner for Large Models Yixin Chen Shuai Zhang Boran Han Tong He Bo Li VLM 16 4 0 06 Jan 2024
Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmers Aleksandar Stanić Sergi Caelles Michael Tschannen LRM VLM 23 9 0 03 Jan 2024
Masked Modeling for Self-supervised Representation Learning on Vision and Beyond Siyuan Li Luyuan Zhang Zedong Wang Di Wu Lirong Wu ... Jun-Xiong Xia Cheng Tan Yang Liu Baigui Sun Stan Z. Li SSL 29 13 0 31 Dec 2023
Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action Jiasen Lu Christopher Clark Sangho Lee Zichen Zhang Savya Khosla Ryan Marten Derek Hoiem Aniruddha Kembhavi VLM MLLM 27 143 0 28 Dec 2023
UniRef++: Segment Every Reference Object in Spatial and Temporal Spaces Jiannan Wu Yi-Xin Jiang Bin Yan Huchuan Lu Zehuan Yuan Ping Luo VOS 24 17 0 25 Dec 2023
Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model Shraman Pramanick Guangxing Han Rui Hou Sayan Nag Ser-Nam Lim Nicolas Ballas Qifan Wang Rama Chellappa Amjad Almahairi VLM MLLM 38 29 0 19 Dec 2023
SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models Lee Hyun Kim Sung-Bin Seungju Han Youngjae Yu Tae-Hyun Oh 25 13 0 15 Dec 2023
VL-GPT: A Generative Pre-trained Transformer for Vision and Language Understanding and Generation Jinguo Zhu Xiaohan Ding Yixiao Ge Yuying Ge Sijie Zhao Hengshuang Zhao Xiaohua Wang Ying Shan ViT VLM 11 32 0 14 Dec 2023
General Object Foundation Model for Images and Videos at Scale Junfeng Wu Yi-Xin Jiang Qihao Liu Zehuan Yuan Xiang Bai Song Bai VOS VLM 25 38 0 14 Dec 2023