Title
SonicRAG : High Fidelity Sound Effects Synthesis Based on Retrival Augmented Generation Yu-Ren Guo Wen-Kai Tai 45 0 0 06 May 2025
VocalNet: Speech LLM with Multi-Token Prediction for Faster and High-Quality Generation Yuhao Wang Heyang Liu Ziyang Cheng Ronghua Wu Qunshan Gu Yanfeng Wang Yu Wang 96 0 0 05 Apr 2025
NOTA: Multimodal Music Notation Understanding for Visual Large Language Model Mingni Tang Jiajia Li Lu Yang Zhiqiang Zhang Jinghao Tian Z. Li L. Zhang P. Wang 51 0 0 17 Feb 2025
DuplexMamba: Enhancing Real-time Speech Conversations with Duplex and Streaming Capabilities Xiangyu Lu Wang Xu Haoyu Wang Hongyun Zhou Haiyan Zhao Conghui Zhu T. Zhao M. Yang Mamba AuLLM 63 0 0 16 Feb 2025
Learning Musical Representations for Music Performance Question Answering Xingjian Diao Chunhui Zhang Tingxuan Wu Ming Cheng Z. Ouyang Weiyi Wu Jiang Gui 65 5 0 10 Feb 2025
DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data Ke-Han Lu Zhehuai Chen Szu-Wei Fu Chao-Han Huck Yang Jagadeesh Balam Boris Ginsburg Yu-Te Wang Hung-yi Lee AuLLM SyDa 104 5 0 28 Jan 2025
A Comprehensive Survey of Foundation Models in Medicine Wasif Khan Seowung Leem Kyle B. See Joshua K. Wong Shaoting Zhang R. Fang AI4CE LM&MA VLM 97 17 0 17 Jan 2025
Spider: Any-to-Many Multimodal LLM Jinxiang Lai Jie Zhang Jun Liu Jian Li Xiaocheng Lu Song Guo MLLM 56 2 0 14 Nov 2024
Robust 3D Point Clouds Classification based on Declarative Defenders Kaidong Li Tianxiao Zhang Cuncong Zhong Z. Zhang G. Wang 3DPC 34 1 0 13 Oct 2024
Audio-Agent: Leveraging LLMs For Audio Generation, Editing and Composition Zixuan Wang Chi-Keung Tang Chi-Keung Tang DiffM VGen LLMAG 41 4 0 04 Oct 2024
Recent Advances in Speech Language Models: A Survey Wenqian Cui Dianzhi Yu Xiaoqi Jiao Ziqiao Meng Guangyan Zhang Qichao Wang Yiwen Guo Irwin King AuLLM 59 14 0 01 Oct 2024
Speechworthy Instruction-tuned Language Models Hyundong Justin Cho Nicolaas Jedema Leonardo F. R. Ribeiro Karishma Sharma Pedro Szekely Alessandro Moschitti Ruben Janssen Jonathan May ALM 40 1 0 23 Sep 2024
A Study on Zero-shot Non-intrusive Speech Assessment using Large Language Models Ryandhimas E. Zezario Sabato Marco Siniscalchi Hsin-Min Wang Yu Tsao 26 2 0 16 Sep 2024
Audio-visual training for improved grounding in video-text LLMs Shivprasad Sagare Hemachandran S Kinshuk Sarabhai Prashant Ullegaddi SA Rajeshkumar 27 0 0 21 Jul 2024
AudioBench: A Universal Benchmark for Audio Large Language Models Bin Wang Xunlong Zou Geyu Lin S. Zhuohan Liu Wenyu Zhang Zhengyuan Liu AiTi Aw Nancy F. Chen AuLLM ELM LM&MA 85 20 0 23 Jun 2024
SpeechVerse: A Large-scale Generalizable Audio Language Model Nilaksh Das Saket Dingliwal S. Ronanki Rohit Paturi David Huang ... Monica Sunkara S. Srinivasan Kyu J. Han Katrin Kirchhoff Katrin Kirchhoff 39 37 0 14 May 2024
Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations Guan-Ting Lin Cheng-Han Chiang Hung-yi Lee 34 22 0 20 Feb 2024
Detecting Multimedia Generated by Large AI Models: A Survey Li Lin Neeraj Gupta Yue Zhang Hainan Ren Chun-Hao Liu Feng Ding Xin Eric Wang X. Li Luisa Verdoliva Shu Hu 75 56 0 22 Jan 2024
HeadArtist: Text-conditioned 3D Head Generation with Self Score Distillation Hongyu Liu Xuan Wang Ziyu Wan Yujun Shen Yibing Song Jing Liao Qifeng Chen DiffM 36 17 0 12 Dec 2023
SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translation Zhehuai Chen He Huang A. Andrusenko Oleksii Hrinchuk Krishna C. Puvvada Jason Chun Lok Li Subhankar Ghosh Jagadeesh Balam Boris Ginsburg LRM 21 48 0 13 Oct 2023
Towards Robust Multi-Modal Reasoning via Model Selection Xiangyan Liu Rongxue Li Wei Ji Tao Lin LLMAG LRM 27 3 0 12 Oct 2023
Joint Audio and Speech Understanding Yuan Gong Alexander H. Liu Hongyin Luo Leonid Karlinsky James R. Glass AuLLM 26 66 0 25 Sep 2023
Refashioning Emotion Recognition Modelling: The Advent of Generalised Large Models Zixing Zhang Liyizhe Peng Tao Pang Jing Han Huan Zhao Bjorn W. Schuller 32 12 0 21 Aug 2023
TableGPT: Towards Unifying Tables, Nature Language and Commands into One GPT Liangyu Zha Junlin Zhou Liyao Li Rui Wang Qingyi Huang ... Xing-yan Deng J. Xu Haobo Wang Gang Chen J. Zhao RALM LMTD 32 42 0 17 Jul 2023
Mini-Giants: "Small" Language Models and Open Source Win-Win Zhengping Zhou Lezhi Li Xinxi Chen Andy Li SyDa ALM MoE 24 6 0 17 Jul 2023
AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation Rongjie Huang Huadai Liu Xize Cheng Yi Ren Lin Li ... Jinzheng He Lichao Zhang Jinglin Liu Xiaoyue Yin Zhou Zhao 58 8 0 24 May 2023
SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities Dong Zhang Shimin Li Xin Zhang Jun Zhan Pengyu Wang Yaqian Zhou Xipeng Qiu AuLLM MLLM 43 287 0 18 May 2023
Augmented Large Language Models with Parametric Knowledge Guiding Ziyang Luo Can Xu Pu Zhao Xiubo Geng Chongyang Tao Jing Ma Qingwei Lin Daxin Jiang KELM RALM 35 44 0 08 May 2023
AlignSTS: Speech-to-Singing Conversion via Cross-Modal Alignment Ruiqi Li Rongjie Huang Lichao Zhang Jinglin Liu Zhou Zhao 23 4 0 08 May 2023
Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models Rongjie Huang Jia-Bin Huang Dongchao Yang Yi Ren Luping Liu Mingze Li Zhenhui Ye Jinglin Liu Xiaoyue Yin Zhou Zhao DiffM 140 315 0 30 Jan 2023
NoreSpeech: Knowledge Distillation based Conditional Diffusion Model for Noise-robust Expressive TTS Dongchao Yang Songxiang Liu Jianwei Yu Helin Wang Chao Weng Yuexian Zou DiffM VLM 29 18 0 04 Nov 2022
TF-GridNet: Making Time-Frequency Domain Models Great Again for Monaural Speaker Separation Zhong-Qiu Wang Samuele Cornell Shukjae Choi Younglo Lee Byeonghak Kim Shinji Watanabe 66 95 0 08 Sep 2022
GenerSpeech: Towards Style Transfer for Generalizable Out-Of-Domain Text-to-Speech Rongjie Huang Yi Ren Jinglin Liu Chenye Cui Zhou Zhao OODD VLM 115 34 0 15 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,881 0 04 Mar 2022
Improving the Performance of Automated Audio Captioning via Integrating the Acoustic and Semantic Information Zhongjie Ye Helin Wang Dongchao Yang Yuexian Zou 32 27 0 12 Oct 2021
Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks Siddharth Dalmia Brian Yan Vikas Raunak Florian Metze Shinji Watanabe 27 30 0 02 May 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,424 0 23 Jan 2020