Title
LLaMA-Omni2: LLM-based Real-time Spoken Chatbot with Autoregressive Streaming Speech Synthesis Qingkai Fang Yan Zhou Shoutao Guo Shaolei Zhang Yang Feng AuLLM 51 0 0 05 May 2025
SIFT-50M: A Large-Scale Multilingual Dataset for Speech Instruction Fine-Tuning Prabhat Pandey R. Swaminathan K V Vijay Girish Arunasish Sen Jian Xie Grant P. Strimel Andreas Schwarz 40 0 0 12 Apr 2025
Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey Y. Wang Shengqiong Wu Y. Zhang William Yang Wang Ziwei Liu Jiebo Luo Hao Fei LRM 82 7 0 16 Mar 2025
Universal Speech Token Learning via Low-Bitrate Neural Codec and Pretrained Representations Xue Jiang Xiulian Peng Yuan Zhang Yan-Heng Lu SSL 79 0 0 15 Mar 2025
Low-Rank and Sparse Model Merging for Multi-Lingual Speech Recognition and Translation Qiuming Zhao Guangzhi Sun Chao Zhang Mingxing Xu Thomas Fang Zheng MoMe VLM 61 0 0 24 Feb 2025
DuplexMamba: Enhancing Real-time Speech Conversations with Duplex and Streaming Capabilities Xiangyu Lu Wang Xu Haoyu Wang Hongyun Zhou Haiyan Zhao Conghui Zhu T. Zhao M. Yang Mamba AuLLM 61 0 0 16 Feb 2025
FireRedASR: Open-Source Industrial-Grade Mandarin Speech Recognition Models from Encoder-Decoder to LLM Integration Kai-Tuo Xu Feng-Long Xie Xu Tang Yao Hu 59 4 0 24 Jan 2025
A Comprehensive Survey of Foundation Models in Medicine Wasif Khan Seowung Leem Kyle B. See Joshua K. Wong Shaoting Zhang R. Fang AI4CE LM&MA VLM 95 16 0 17 Jan 2025
AMPS: ASR with Multimodal Paraphrase Supervision Amruta Parulekar Abhishek Gupta Sameep Chattopadhyay P. Jyothi 75 0 0 27 Nov 2024
VoiceTextBlender: Augmenting Large Language Models with Speech Capabilities via Single-Stage Joint Speech-Text Supervised Fine-Tuning Yifan Peng Krishna C. Puvvada Zhehuai Chen Piotr .Zelasko He Huang Kunal Dhawan Ke Hu Shinji Watanabe Jagadeesh Balam Boris Ginsburg 47 2 0 23 Oct 2024
Continuous Speech Tokenizer in Text To Speech Yixing Li Ruobing Xie X. Sun Yu Cheng Zhanhui Kang AuLLM CLL 45 2 0 22 Oct 2024
Ichigo: Mixed-Modal Early-Fusion Realtime Voice Assistant Alan Dao Dinh Bach Vu Huy Hoang Ha AuLLM VLM 59 3 0 20 Oct 2024
Thought2Text: Text Generation from EEG Signal using Large Language Models (LLMs) Abhijit Mishra Shreya Shukla Jose Torres Jacek Gwizdka Shounak Roychowdhury 41 4 0 10 Oct 2024
Recent Advances in Speech Language Models: A Survey Wenqian Cui Dianzhi Yu Xiaoqi Jiao Ziqiao Meng Guangyan Zhang Qichao Wang Yiwen Guo Irwin King AuLLM 59 14 0 01 Oct 2024
MIO: A Foundation Model on Multimodal Tokens Zekun Wang King Zhu Chunpu Xu Wangchunshu Zhou Jiaheng Liu ... Yuanxing Zhang Ge Zhang Ke Xu Jie Fu Wenhao Huang MLLM AuLLM 46 11 0 26 Sep 2024
Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation Siyin Wang Wenyi Yu Yudong Yang Changli Tang Yixuan Li ... Jun Zhang Guangzhi Sun Lu Lu Yuxuan Wang Chao Zhang AuLLM LM&MA 65 5 0 25 Sep 2024
Salmon: A Suite for Acoustic Language Model Evaluation Gallil Maimon Amit Roth Yossi Adi ELM AuLLM 49 5 0 11 Sep 2024
MoWE-Audio: Multitask AudioLLMs with Mixture of Weak Encoders W. Zhang Shuo Sun Bin Wang Xunlong Zou Zhuohan Liu Yingxu He Geyu Lin Nancy F. Chen A. Aw AuLLM 65 1 0 10 Sep 2024
AudioBench: A Universal Benchmark for Audio Large Language Models Bin Wang Xunlong Zou Geyu Lin S. Zhuohan Liu Wenyu Zhang Zhengyuan Liu AiTi Aw Nancy F. Chen AuLLM ELM LM&MA 85 17 0 23 Jun 2024
Large Language Models for Dysfluency Detection in Stuttered Speech Dominik Wagner Sebastian P. Bayerl Ilja Baumann K. Riedhammer Elmar Nöth Tobias Bocklet 35 3 0 16 Jun 2024
How Should We Extract Discrete Audio Tokens from Self-Supervised Models? Pooneh Mousavi J. Duret Salah Zaiem Luca Della Libera Artem Ploujnikov Cem Subakan Mirco Ravanelli 26 9 0 15 Jun 2024
Can Large Language Models Understand Spatial Audio? Changli Tang Wenyi Yu Guangzhi Sun Xianzhao Chen Tian Tan ... Jun Zhang Lu Lu Zejun Ma Yuxuan Wang Chao Zhang 44 4 0 12 Jun 2024
PolySpeech: Exploring Unified Multitask Speech Models for Competitiveness with Single-task Models Runyan Yang Huibao Yang Xiqing Zhang Tiantian Ye Ying Liu Yingying Gao Shilei Zhang Chao Deng Junlan Feng 26 0 0 12 Jun 2024
The Interspeech 2024 Challenge on Speech Processing Using Discrete Units Xuankai Chang Jiatong Shi Jinchuan Tian Yuning Wu Yuxun Tang Yihan Wu Shinji Watanabe Yossi Adi Xie Chen Qin Jin 40 15 0 11 Jun 2024
Discrete Multimodal Transformers with a Pretrained Large Language Model for Mixed-Supervision Speech Processing V. Trinh Rosy Southwell Yiwen Guan Xinlu He Zhiyong Wang Jacob Whitehill OffRL 30 2 0 04 Jun 2024
SeamlessExpressiveLM: Speech Language Model for Expressive Speech-to-Speech Translation with Chain-of-Thought Hongyu Gong Bandhav Veluri 36 0 0 30 May 2024
SpeechVerse: A Large-scale Generalizable Audio Language Model Nilaksh Das Saket Dingliwal S. Ronanki Rohit Paturi David Huang ... Monica Sunkara S. Srinivasan Kyu J. Han Katrin Kirchhoff Katrin Kirchhoff 39 37 0 14 May 2024
MAD Speech: Measures of Acoustic Diversity of Speech Matthieu Futeral A. Agostinelli Marco Tagliasacchi Neil Zeghidour Eugene Kharitonov 46 1 0 16 Apr 2024
WavLLM: Towards Robust and Adaptive Speech Large Language Model Shujie Hu Long Zhou Shujie Liu Sanyuan Chen Hongkun Hao ... Xunying Liu Jinyu Li S. Sivasankaran Linquan Liu Furu Wei AuLLM 21 42 0 31 Mar 2024
InkSight: Offline-to-Online Handwriting Conversion by Learning to Read and Write B. Mitrevski Arina Rak Julian Schnitzler Chengkun Li Andrii Maksai Jesse Berent C. Musat DiffM 13 0 0 08 Feb 2024
ADriver-I: A General World Model for Autonomous Driving Fan Jia Weixin Mao Yingfei Liu Yucheng Zhao Yuqing Wen Chi Zhang Xiangyu Zhang Tiancai Wang 16 63 0 22 Nov 2023
SALMONN: Towards Generic Hearing Abilities for Large Language Models Changli Tang Wenyi Yu Guangzhi Sun Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Chao Zhang LM&MA AuLLM 28 195 0 20 Oct 2023
SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translation Zhehuai Chen He Huang A. Andrusenko Oleksii Hrinchuk Krishna C. Puvvada Jason Chun Lok Li Subhankar Ghosh Jagadeesh Balam Boris Ginsburg LRM 16 48 0 13 Oct 2023
Fine-grained Audio-Visual Joint Representations for Multimodal Large Language Models Guangzhi Sun Wenyi Yu Changli Tang Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Chao Zhang 23 12 0 09 Oct 2023
Discrete Audio Representation as an Alternative to Mel-Spectrograms for Speaker and Speech Recognition Krishna C. Puvvada Nithin Rao Koluguri Kunal Dhawan Jagadeesh Balam Boris Ginsburg 8 12 0 19 Sep 2023
Multiple Representation Transfer from Large Language Models to End-to-End ASR Systems Takuma Udagawa Masayuki Suzuki Gakuto Kurata Masayasu Muraoka G. Saon 15 2 0 07 Sep 2023
Spoken Language Intelligence of Large Language Models for Language Learning Linkai Peng Baorian Nuchged Yingming Gao ELM 54 3 0 28 Aug 2023
SpeechX: Neural Codec Language Model as a Versatile Speech Transformer Xiaofei Wang Manthan Thakker Zhuo Chen Naoyuki Kanda Sefik Emre Eskimez Sanyuan Chen M. Tang Shujie Liu Jinyu Li Takuya Yoshioka 13 79 0 14 Aug 2023
JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Models Peike Li Bo-Yu Chen Yao Yao Yikai Wang Allen Wang Alex Jinpeng Wang MGen VLM DiffM 46 36 0 09 Aug 2023
Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages Yu Zhang Wei Han James Qin Yongqiang Wang Ankur Bapna ... Pedro J. Moreno Chung-Cheng Chiu J. Schalkwyk Franccoise Beaufays Yonghui Wu VLM 77 249 0 02 Mar 2023
FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech Alexis Conneau Min Ma Simran Khanuja Yu Zhang Vera Axelrod Siddharth Dalmia Jason Riesa Clara E. Rivera Ankur Bapna VLM 78 281 0 25 May 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
Direct Simultaneous Speech-to-Speech Translation with Variational Monotonic Multihead Attention Xutai Ma Hongyu Gong Danni Liu Ann Lee Yun Tang Peng-Jen Chen Wei-Ning Hsu P. Koehn J. Pino 46 8 0 15 Oct 2021
Generative Spoken Language Modeling from Raw Audio Kushal Lakhotia Evgeny Kharitonov Wei-Ning Hsu Yossi Adi Adam Polyak ... Tu Nguyen Jade Copet Alexei Baevski A. Mohamed Emmanuel Dupoux AuLLM 174 336 0 01 Feb 2021