Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models

14 November 2023

Yunfei Chu

Jin Xu

Xiaohuan Zhou

Qian Yang

Shiliang Zhang

Zhijie Yan

Chang Zhou

Jingren Zhou

AuLLM

ArXiv PDF HTML

Papers citing "Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models"

50 / 211 papers shown

Title
Bridging Ears and Eyes: Analyzing Audio and Visual Large Language Models to Humans in Visible Sound Recognition and Reducing Their Sensory Gap via Cross-Modal Distillation Xilin Jiang Junkai Wu Vishal B. Choudhari N. Mesgarani VLM 25 0 0 11 May 2025
LLaMA-Omni2: LLM-based Real-time Spoken Chatbot with Autoregressive Streaming Speech Synthesis Qingkai Fang Yan Zhou Shoutao Guo Shaolei Zhang Yang Feng AuLLM 51 0 0 05 May 2025
Voila: Voice-Language Foundation Models for Real-Time Autonomous Interaction and Voice Role-Play Yemin Shi Yu Shu Siwei Dong Guangyi Liu Jaward Sesay Jingwen Li Zhiting Hu AuLLM VLM 43 0 0 05 May 2025
Enhancing Non-Core Language Instruction-Following in Speech LLMs via Semi-Implicit Cross-Lingual CoT Reasoning Hongfei Xue Yufeng Tang Hexin Liu Jun Zhang Xuelong Geng Lei Xie LRM 50 0 0 29 Apr 2025
Kimi-Audio Technical Report KimiTeam Ding Ding Zeqian Ju Yichong Leng S. Liu ... Z. Yang Aoxiong Yin Ruibin Yuan Y. Zhang Zaida Zhou AuLLM VLM 105 1 0 25 Apr 2025
A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigms Chengkai Huang Hongtao Huang Tong Yu Kaige Xie Junda Wu Shuai Zhang Julian McAuley Dietmar Jannach Lina Yao LRM AI4CE 22 0 0 23 Apr 2025
SimulS2S-LLM: Unlocking Simultaneous Inference of Speech LLMs for Speech-to-Speech Translation Keqi Deng Wenxi Chen Xie Chen P. Woodland 43 0 0 22 Apr 2025
Transformation of audio embeddings into interpretable, concept-based representations Alice Zhang Edison Thomaz Lie Lu 27 0 0 18 Apr 2025
A Survey on Cross-Modal Interaction Between Music and Multimodal Data Sifei Li Mining Tan Feier Shen Minyan Luo Zijiao Yin Fan Tang W. Dong Changsheng Xu 57 0 0 17 Apr 2025
Reimagining Urban Science: Scaling Causal Inference with Large Language Models Yutong Xia Ao Qu Yunhan Zheng Yihong Tang Dingyi Zhuang ... Cathy Wu R. Zimmermann Lijun Sun Roger Zimmermann Jinhua Zhao AI4CE 53 0 0 15 Apr 2025
SIFT-50M: A Large-Scale Multilingual Dataset for Speech Instruction Fine-Tuning Prabhat Pandey R. Swaminathan K V Vijay Girish Arunasish Sen Jian Xie Grant P. Strimel Andreas Schwarz 38 0 0 12 Apr 2025
PACT: Pruning and Clustering-Based Token Reduction for Faster Visual Language Models M. Dhouib Davide Buscaldi Sonia Vanier A. Shabou VLM 36 0 0 11 Apr 2025
On The Landscape of Spoken Language Models: A Comprehensive Survey Siddhant Arora Kai-Wei Chang Chung-Ming Chien Yifan Peng Haibin Wu Yossi Adi Emmanuel Dupoux Hung-yi Lee Karen Livescu Shinji Watanabe 39 1 0 11 Apr 2025
TASTE: Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling Liang-Hsuan Tseng Yi-Chang Chen Kuan-Yi Lee Da-shan Shiu Hung-yi Lee AuLLM 52 0 0 09 Apr 2025
Scaling Analysis of Interleaved Speech-Text Language Models Gallil Maimon Michael Hassid Amit Roth Yossi Adi AuLLM 40 0 0 03 Apr 2025
Are you really listening? Boosting Perceptual Awareness in Music-QA Benchmarks Yongyi Zang Sean O'Brien Taylor Berg-Kirkpatrick Julian McAuley Zachary Novack AuLLM 79 0 0 01 Apr 2025
SpeechDialogueFactory: Generating High-Quality Speech Dialogue Data to Accelerate Your Speech-LLM Development Minghan Wang Ye Bai Y. Wang Thuy-Trang Vu Ehsan Shareghi Gholamreza Haffari 43 0 0 31 Mar 2025
RLDBF: Enhancing LLMs Via Reinforcement Learning With DataBase FeedBack Weichen Dai Zijie Dai Zhijie Huang Yixuan Pan Xinhe Li Xi Li Yi Zhou Ji Qi Wu Jiang 24 0 0 28 Mar 2025
Make Some Noise: Towards LLM audio reasoning and generation using sound tokens Shivam Mehta Nebojsa Jojic Hannes Gamper 31 0 0 28 Mar 2025
ChatAnyone: Stylized Real-time Portrait Video Generation with Hierarchical Motion Diffusion Model Jinwei Qi Chaonan Ji Sheng Xu Peng Zhang Bang Zhang Liefeng Bo DiffM VGen 45 1 0 27 Mar 2025
OmniVox: Zero-Shot Emotion Recognition with Omni-LLMs John Murzaku Owen Rambow AuLLM 46 0 0 27 Mar 2025
FinAudio: A Benchmark for Audio Large Language Models in Financial Applications Yupeng Cao Haohang Li Yangyang Yu Shashidhar Reddy Javaji Yueru He ... Xiao-Yang Liu K. P. Subbalakshmi Meikang Qiu Sophia Ananiadou J. Nie AuLLM 69 0 0 26 Mar 2025
Qwen2.5-Omni Technical Report Jin Xu Zhifang Guo Jinzheng He Hangrui Hu Ting He ... K. Dang Bin Zhang X. Wang Yunfei Chu Junyang Lin VGen AuLLM 86 12 0 26 Mar 2025
QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions Siyin Wang Wenyi Yu Xianzhao Chen Xiaohai Tian J. Zhang Lu Lu Yu Tsao Junichi Yamagishi Y. Wang Chao Zhang AuLLM 76 0 0 26 Mar 2025
Dolphin: A Large-Scale Automatic Speech Recognition Model for Eastern Languages Yangyang Meng Jinpeng Li Guodong Lin Yu Pu G. Wang Hu Du Zhiming Shao Yukai Huang Ke Li Wei-Qiang Zhang ObjD 93 0 0 26 Mar 2025
Large Language Models Meet Contrastive Learning: Zero-Shot Emotion Recognition Across Languages Heqing Zou Fengmao Lv Desheng Zheng E. Chng D. Rajan 24 0 0 25 Mar 2025
Solla: Towards a Speech-Oriented LLM That Hears Acoustic Context Junyi Ao Dekun Chen Xiaohai Tian Wenjie Feng J. Zhang Lu Lu Y. Wang Haizhou Li Zhizheng Wu AuLLM 61 0 0 19 Mar 2025
Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey Y. Wang Shengqiong Wu Y. Zhang William Yang Wang Ziwei Liu Jiebo Luo Hao Fei LRM 78 7 0 16 Mar 2025
MMS-LLaMA: Efficient LLM-based Audio-Visual Speech Recognition with Minimal Multimodal Speech Tokens Jeong Hun Yeo Hyeongseop Rha Se Jin Park Y. Ro 46 0 0 14 Mar 2025
Adaptive Inner Speech-Text Alignment for LLM-based Speech Translation Henglyu Liu Andong Chen Kehai Chen X. Bai M. Zhong Yuan Qiu Min Zhang 37 0 0 13 Mar 2025
Mellow: a small audio language model for reasoning Soham Deshmukh Satvik Dixit Rita Singh Bhiksha Raj AuLLM ReLM LRM 75 1 0 11 Mar 2025
GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images Xiang Lan Feng Wu Kai He Qinghao Zhao Shenda Hong Mengling Feng AI4TS 52 2 0 08 Mar 2025
S2S-Arena, Evaluating Speech2Speech Protocols on Instruction Following with Paralinguistic Information Feng Jiang Zhiyu Lin Fan Bu Yuhao Du Benyou Wang H. Li AuLLM ELM 88 0 0 07 Mar 2025
Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities Sreyan Ghosh Zhifeng Kong Sonal Kumar S. Sakshi Jaehyeon Kim Wei Ping Rafael Valle Dinesh Manocha Bryan Catanzaro MLLM AuLLM LRM 49 4 0 06 Mar 2025
Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models Zhifei Xie Mingbao Lin Z. Liu Pengcheng Wu Shuicheng Yan Chunyan Miao AuLLM OffRL LRM 76 5 0 04 Mar 2025
InSerter: Speech Instruction Following with Unsupervised Interleaved Pre-training Dingdong Wang Jin Xu Ruihang Chu Zhifang Guo X. Wang Jincenzi Wu Dongchao Yang Shengpeng Ji Junyang Lin AuLLM 83 0 0 04 Mar 2025
A Hypernetwork-Based Approach to KAN Representation of Audio Signals Patryk Marszałek Maciej Rut Piotr Kawa P. Syga 31 0 0 04 Mar 2025
MindBridge: Scalable and Cross-Model Knowledge Editing via Memory-Augmented Modality Shuaike Li Kai Zhang Q. Liu Enhong Chen KELM 73 1 0 04 Mar 2025
Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics Siddhant Arora Zhiyun Lu Chung-Cheng Chiu Ruoming Pang Shinji Watanabe 43 2 0 03 Mar 2025
AAD-LLM: Neural Attention-Driven Auditory Scene Understanding Xilin Jiang Sukru Samet Dindar Vishal B. Choudhari Stephan Bickel A. Mehta Guy M McKhann A. Flinker D. Friedman N. Mesgarani 32 1 0 24 Feb 2025
Retrieval-Augmented Speech Recognition Approach for Domain Challenges Peng Shen Xugang Lu Hisashi Kawai RALM 60 0 0 24 Feb 2025
Baichuan-Audio: A Unified Framework for End-to-End Speech Interaction Tianpeng Li J. Liu Tao Zhang Yuanbo Fang Da Pan ... Guosheng Dong Jianhua Xu Haoze Sun Zenan Zhou Weipeng Chen AuLLM 53 3 0 24 Feb 2025
Audio-FLAN: A Preliminary Release Liumeng Xue Ziya Zhou J. Pan Z. Li Shuai Fan ... Haohe Liu Emmanouil Benetos Ge Zhang Yike Guo Wei Xue MLLM AuLLM CLIP VLM 57 1 0 23 Feb 2025
Chain-of-Description: What I can understand, I can put into words J. Guo Daimeng Wei Z. Li Hengchao Shang Yuanchang Luo Hao Yang 45 0 0 22 Feb 2025
EAGER-LLM: Enhancing Large Language Models as Recommenders through Exogenous Behavior-Semantic Integration Minjie Hong Yan Xia Z. Wang Jieming Zhu Ye Wang ... Xiaoda Yang Quanyu Dai Zhenhua Dong Zhimeng Zhang Zhou Zhao 52 1 0 21 Feb 2025
Mind the Gap! Static and Interactive Evaluations of Large Audio Models Minzhi Li William B. Held Michael Joseph Ryan Kunat Pipatanakul Potsawee Manakul Hao Zhu Diyi Yang AuLLM ALM 56 0 0 21 Feb 2025
Slamming: Training a Speech Language Model on One GPU in a Day Gallil Maimon Avishai Elmakies Yossi Adi 38 3 0 19 Feb 2025
Soundwave: Less is More for Speech-Text Alignment in LLMs Y. Zhang Zhiheng Liu Fan Bu Ruiyu Zhang Benyou Wang H. Li AuLLM SyDa VLM 98 0 0 18 Feb 2025
Survey on AI-Generated Media Detection: From Non-MLLM to MLLM Yueying Zou Peipei Li Zekun Li Huaibo Huang Xing Cui Xuannan Liu Chenghanyu Zhang Ran He DeLMO 111 1 0 07 Feb 2025
"I am bad": Interpreting Stealthy, Universal and Robust Audio Jailbreaks in Audio-Language Models Isha Gupta David Khachaturov Robert D. Mullins AAML AuLLM 60 1 0 02 Feb 2025