Co-Reinforcement Learning for Unified Multimodal Understanding and Generation

23 May 2025

Papers citing "Co-Reinforcement Learning for Unified Multimodal Understanding and Generation"

50 / 82 papers shown

Title
T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT D. Jiang Ziyu Guo Renrui Zhang Zhuofan Zong Hao Li Le Zhuo Shilin Yan Pheng-Ann Heng Haoyang Li LRM 103 14 0 01 May 2025
Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models Guanghao Zhou Panjia Qiu Chong Chen Jiadong Wang Zheming Yang Jian Xu Minghui Qiu OffRL LRM 84 4 0 30 Apr 2025
SimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RL Junke Wang Zhi Tian Xinyu Wang Xinyu Zhang Weilin Huang Zuxuan Wu Yu Jiang VGen 116 13 0 15 Apr 2025
InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models Jinguo Zhu Weiyun Wang Zhe Chen Ziwei Liu Shenglong Ye ... Dahua Lin Yu Qiao Jifeng Dai Wenhai Wang Wei Wang MLLM VLM 98 56 1 14 Apr 2025
VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning Xinhao Li Ziang Yan Desen Meng Lu Dong Xiangyu Zeng Yinan He Yun Wang Yu Qiao Yi Wang Limin Wang VLM AI4TS LRM 72 18 0 09 Apr 2025
SmolVLM: Redefining small and efficient multimodal models Andres Marafioti Orr Zohar Miquel Farré Merve Noyan Elie Bakouch ... Hugo Larcher Mathieu Morlon Lewis Tunstall Leandro von Werra Thomas Wolf VLM 63 11 0 07 Apr 2025
UniToken: Harmonizing Multimodal Understanding and Generation through Unified Visual Encoding Yang Jiao Haibo Qiu Zequn Jie Tian Jin Jingjing Chen Lin Ma Yu Jiang 62 5 0 06 Apr 2025
ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinement Runhui Huang Chunwei Wang Junwei Yang Guansong Lu Yunlong Yuan ... Lu Hou Wei Zhang Lanqing Hong Hengshuang Zhao Hang Xu MLLM 109 5 0 02 Apr 2025
Harmonizing Visual Representations for Unified Multimodal Understanding and Generation Size Wu Wentao Zhang Lumin Xu Sheng Jin Zhonghua Wu Qingyi Tao Wentao Liu Wei Li Chen Change Loy VGen 345 5 0 27 Mar 2025
Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning Yufei Zhan Yousong Zhu Shurong Zheng Hongyin Zhao Fan Yang Ming Tang Jinqiao Wang VLM 81 12 0 23 Mar 2025
DAPO: An Open-Source LLM Reinforcement Learning System at Scale Qiying Yu Zheng Zhang Ruofei Zhu Yufeng Yuan Xiaochen Zuo ... Ya Zhang Lin Yan Mu Qiao Yonghui Wu Mingxuan Wang OffRL LRM 100 131 0 18 Mar 2025
Unified Autoregressive Visual Generation and Understanding with Continuous Tokens Lijie Fan Luming Tang Siyang Qin Tianhong Li Xuan S. Yang ... Tao Zhu Michael Rubinstein Michalis Raptis Deqing Sun Radu Soricut 69 6 0 17 Mar 2025
R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization Jingyi Zhang Jiaxing Huang Huanjin Yao Shunyu Liu Xikun Zhang Shijian Lu Dacheng Tao LRM 110 45 0 17 Mar 2025
Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning Huilin Deng Ding Zou Rui Ma Hongchen Luo Yang Cao Yu Kang LRM VLM 80 15 0 10 Mar 2025
WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation Yuwei Niu Munan Ning Mengren Zheng Weiyang Jin Bin Lin ... Jiaqi Liao Chaoran Feng Kunpeng Ning Bin Zhu Li Yuan EGVM 98 23 0 10 Mar 2025
LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL Yingzhe Peng Gongrui Zhang Miaosen Zhang Zhiyuan You Jie Liu Qipeng Zhu Kai Yang Xingzhong Xu Xin Geng Xu Yang LRM ReLM 129 52 0 10 Mar 2025
Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models Wenxuan Huang Bohan Jia Zijie Zhai Shaosheng Cao Zheyu Ye Fei Zhao Zhe Xu Yao Hu Shaohui Lin MU OffRL LRM MLLM ReLM VLM 90 85 0 09 Mar 2025
Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement Yuqi Liu Bohao Peng Zhisheng Zhong Zihao Yue Fanbin Lu Bei Yu Jiaya Jia LRM VLM 98 29 0 09 Mar 2025
Unified Reward Model for Multimodal Understanding and Generation Yibin Wang Yuhang Zang Hao Li Cheng Jin Jiadong Wang EGVM 109 9 0 07 Mar 2025
Visual-RFT: Visual Reinforcement Fine-Tuning Ziyu Liu Zeyi Sun Yuhang Zang Xiaoyi Dong Yuhang Cao Haodong Duan Dahua Lin Jiaqi Wang ObjD VLM LRM 107 76 0 03 Mar 2025
Qwen2.5-VL Technical Report S. Bai Keqin Chen Xuejing Liu Jialin Wang Wenbin Ge ... Zesen Cheng Hang Zhang Zhibo Yang Haiyang Xu Junyang Lin VLM 142 430 0 20 Feb 2025
HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and Generation L. Yang Xinchen Zhang Ye Tian Chenming Shang Minghao Xu Wentao Zhang Tengjiao Wang 114 2 0 17 Feb 2025
Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling Xiaokang Chen Zhiyu Wu Xingchao Liu Zizheng Pan Wen Liu Zhenda Xie X. Yu Chong Ruan AI4TS 62 126 0 29 Jan 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI Daya Guo Dejian Yang Haowei Zhang Junxiao Song ... Shiyu Wang S. Yu Shunfeng Zhou Shuting Pan S.S. Li ReLM VLM OffRL AI4TS LRM 210 1,503 0 22 Jan 2025
Kimi k1.5: Scaling Reinforcement Learning with LLMs Kimi Team Angang Du Bofei Gao Bowei Xing Changjiu Jiang ... Zihao Huang Ziyao Xu Zhiyong Yang Zonghan Yang Zongyu Lin OffRL ALM AI4TS VLM LRM 176 250 0 22 Jan 2025
Unveiling the Mystery of Weight in Large Foundation Models: Gaussian Distribution Never Fades Chongjie Si Jingjing Jiang Wei Shen 31 3 0 18 Jan 2025
SynerGen-VL: Towards Synergistic Image Understanding and Generation with Vision Experts and Token Folding Hao Li Changyao Tian Jie Shao X. Zhu Zhaokai Wang ... Wenhan Dou Xiaogang Wang Hongsheng Li Lewei Lu Jifeng Dai MLLM 104 12 0 12 Dec 2024
ILLUME: Illuminating Your LLMs to See, Draw, and Self-Enhance Chunwei Wang Guansong Lu Junwei Yang Runhui Huang Jiawei Han Lu Hou Wei Zhang Hang Xu MLLM 88 12 0 09 Dec 2024
TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation Liao Qu Huichao Zhang Yiheng Liu Xinyu Wang Yi Jiang Yiming Gao Hu Ye Daniel K. Du Zehuan Yuan Xinglong Wu 97 30 0 04 Dec 2024
Orthus: Autoregressive Interleaved Image-Text Generation with Modality-Specific Heads Siqi Kou Jiachun Jin Chang Liu Ye Ma Jian Jia Quan Chen Peng Jiang Zhijie Deng Zhijie Deng DiffM VGen VLM 172 9 0 28 Nov 2024
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization Weiyun Wang Zhe Chen Wenhai Wang Yue Cao Yangzhou Liu ... Jinguo Zhu X. Zhu Lewei Lu Yu Qiao Jifeng Dai LRM 87 65 1 15 Nov 2024
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation Chengyue Wu Xiaokang Chen Z. F. Wu Yiyang Ma Xingchao Liu ... Wen Liu Zhenda Xie Xingkai Yu Chong Ruan Ping Luo AI4TS 93 89 0 17 Oct 2024
VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignment Lei Li Zhihui Xie Mukai Li Shunian Chen Peiyi Wang L. Chen Yazheng Yang Benyou Wang Dianbo Sui Qiang Liu VLM ALM 48 21 0 12 Oct 2024
Emu3: Next-Token Prediction is All You Need Xinlong Wang Xiaosong Zhang Zhengxiong Luo Quan-Sen Sun Yufeng Cui ... Xi Yang Jingjing Liu Yonghua Lin Tiejun Huang Zhongyuan Wang MLLM 72 183 0 27 Sep 2024
Show-o: One Single Transformer to Unify Multimodal Understanding and Generation Jinheng Xie Weijia Mao Zechen Bai David Junhao Zhang Weihao Wang Kevin Qinghong Lin Yuchao Gu Zhijie Chen Zhenheng Yang Mike Zheng Shou 81 188 0 22 Aug 2024
Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model Chunting Zhou Lili Yu Arun Babu Kushal Tirumala Michihiro Yasunaga Leonid Shamis Jacob Kahn Xuezhe Ma Luke Zettlemoyer Omer Levy DiffM 74 165 0 20 Aug 2024
VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models Haodong Duan Junming Yang Junming Yang Xinyu Fang Lin Chen ... Yuhang Zang Pan Zhang Jiaqi Wang Dahua Lin Kai Chen LM&MA VLM 92 142 0 16 Jul 2024
LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts Yijia Xiao Edward Sun Tianyu Liu Wei Wang LRM 39 32 0 06 Jul 2024
We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning? Runqi Qiao Qiuna Tan Guanting Dong Minhui Wu Chong Sun ... Yida Xu Muxi Diao Zhimin Bao Chen Li Honggang Zhang VLM LRM 67 41 0 01 Jul 2024
mDPO: Conditional Preference Optimization for Multimodal Large Language Models Fei Wang Wenxuan Zhou James Y. Huang Nan Xu Sheng Zhang Hoifung Poon Muhao Chen 84 23 0 17 Jun 2024
Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation Peize Sun Yi Jiang Shoufa Chen Shilong Zhang Bingyue Peng Ping Luo Zehuan Yuan VLM 94 253 0 10 Jun 2024
Ovis: Structural Embedding Alignment for Multimodal Large Language Model Shiyin Lu Yang Li Qing-Guo Chen Zhao Xu Weihua Luo Kaifu Zhang Han-Jia Ye VLM MLLM 85 43 0 31 May 2024
M $^3$ CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought Qiguang Chen Libo Qin Jin Zhang Zhi Chen Xiao Xu Wanxiang Che LRM 78 47 0 26 May 2024
Reinforcement learning Florentin Wörgötter 55 2,569 0 16 May 2024
Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning Yuexiang Zhai Hao Bai Zipeng Lin Jiayi Pan Shengbang Tong ... Alane Suhr Saining Xie Yann LeCun Yi-An Ma Sergey Levine LLMAG LRM 74 72 0 16 May 2024
SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generation Yuying Ge Sijie Zhao Jinguo Zhu Yixiao Ge Kun Yi Lin Song Chen Li Xiaohan Ding Ying Shan VLM 88 120 0 22 Apr 2024
Are We on the Right Way for Evaluating Large Vision-Language Models? Lin Chen Jinsong Li Xiao-wen Dong Pan Zhang Yuhang Zang ... Haodong Duan Jiaqi Wang Yu Qiao Dahua Lin Feng Zhao VLM 94 245 0 29 Mar 2024
MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems? Renrui Zhang Dongzhi Jiang Yichi Zhang Haokun Lin Ziyu Guo ... Aojun Zhou Pan Lu Kai-Wei Chang Peng Gao Hongsheng Li 45 205 0 21 Mar 2024
Strengthening Multimodal Large Language Model with Bootstrapped Preference Optimization Renjie Pi Tianyang Han Wei Xiong Jipeng Zhang Runtao Liu Rui Pan Tong Zhang MLLM 95 37 0 13 Mar 2024
ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment Xiwei Hu Rui Wang Yixiao Fang Bin-Bin Fu Pei Cheng Gang Yu VLM 71 74 0 08 Mar 2024