Audio Captioning Transformer

21 July 2021

Papers citing "Audio Captioning Transformer"

50 / 63 papers shown

Title
TACOS: Temporally-aligned Audio CaptiOnS for Language-Audio Pretraining Paul Primus Florian Schmid Gerhard Widmer CLIP AI4TS VLM 28 0 0 12 May 2025
A Survey on Cross-Modal Interaction Between Music and Multimodal Data Sifei Li Mining Tan Feier Shen Minyan Luo Zijiao Yin Fan Tang W. Dong Changsheng Xu 57 0 0 17 Apr 2025
Unbiased Sliced Wasserstein Kernels for High-Quality Audio Captioning Manh Luong Khai Nguyen Dinh Q. Phung Gholamreza Haffari Lizhen Qu 47 0 0 08 Feb 2025
Audio-Language Models for Audio-Centric Tasks: A survey Yi Su Jisheng Bai Qisheng Xu Kele Xu Yong Dou AuLLM 99 2 0 28 Jan 2025
Audio-Language Datasets of Scenes and Events: A Survey Gijs Wijngaard Elia Formisano Michele Esposito M. Dumontier 79 2 0 10 Jan 2025
Lyra: An Efficient and Speech-Centric Framework for Omni-Cognition Zhisheng Zhong Chengyao Wang Yuqi Liu Senqiao Yang Longxiang Tang ... Shaozuo Yu Sitong Wu Eric Lo Shu-Lin Liu Jiaya Jia AuLLM 103 6 0 12 Dec 2024
MuMu-LLaMA: Multi-modal Music Understanding and Generation via Large Language Models Shansong Liu Atin Sakkeer Hussain Qilong Wu Chenshuo Sun Ying Shan AuLLM 61 3 0 09 Dec 2024
Towards Diverse and Efficient Audio Captioning via Diffusion Models Manjie Xu Chenxing Li Xinyi Tu Yong Ren Ruibo Fu Wei Liang Dong Yu DiffM 41 1 0 14 Sep 2024
EnCLAP++: Analyzing the EnCLAP Framework for Optimizing Automated Audio Captioning Performance Jaeyeon Kim Minjeon Jeon Jaeyoon Jung Sang Hoon Woo Jinjoo Lee 26 2 0 02 Sep 2024
Expanding on EnCLAP with Auxiliary Retrieval Model for Automated Audio Captioning Jaeyeon Kim Jaeyoon Jung Minjeong Jeon Sang Hoon Woo Jinjoo Lee 24 1 0 02 Sep 2024
Dissecting Temporal Understanding in Text-to-Audio Retrieval Andreea-Maria Oncescu João F. Henriques A. Sophia Koepke 24 2 0 01 Sep 2024
Efficient Audio Captioning with Encoder-Level Knowledge Distillation Xuenan Xu Haohe Liu Mengyue Wu Wenwu Wang Mark D. Plumbley 40 1 0 19 Jul 2024
AVCap: Leveraging Audio-Visual Features as Text Tokens for Captioning Jongsuk Kim Jiwon Shin Junmo Kim 26 1 0 10 Jul 2024
Zero-Shot Audio Captioning Using Soft and Hard Prompts Yiming Zhang Xuenan Xu Ruoyi Du Haohe Liu Yuan Dong Zheng-Hua Tan Wenwu Wang Zhanyu Ma VLM 33 4 0 10 Jun 2024
Distance Sampling-based Paraphraser Leveraging ChatGPT for Text Data Manipulation Yoori Oh Yoseob Han Kyogu Lee 37 1 0 01 May 2024
SoundingActions: Learning How Actions Sound from Narrated Egocentric Videos Changan Chen Kumar Ashutosh Rohit Girdhar David F. Harwath Kristen Grauman EgoV SSL 26 6 0 08 Apr 2024
EnCLAP: Combining Neural Audio Codec and Audio-Text Joint Embedding for Automated Audio Captioning Jaeyeon Kim Jaeyoon Jung Jinjoo Lee Sang Hoon Woo CLIP VLM 16 21 0 31 Jan 2024
Zero-shot audio captioning with audio-language model guidance and audio context keywords Leonard Salewski Stefan Fauth A. Sophia Koepke Zeynep Akata 19 10 0 14 Nov 2023
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model Avamarie Brueggeman Andrea Madotto Zhaojiang Lin Tushar Nagarajan Matt Smith ... Peyman Heidari Yue Liu Kavya Srinet Babak Damavandi Anuj Kumar MLLM 29 92 0 27 Sep 2023
Weakly-supervised Automated Audio Captioning via text only training Theodoros Kouzelis V. Katsouros CLIP 25 6 0 21 Sep 2023
RECAP: Retrieval-Augmented Audio Captioning Sreyan Ghosh Sonal Kumar Chandra Kiran Reddy Evuru R. Duraiswami Dinesh Manocha VLM 62 17 0 18 Sep 2023
MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response Zihao Deng Yi Ma Yudong Liu Rongchen Guo Ge Zhang Wenhu Chen Wenhao Huang Emmanouil Benetos MLLM AuLLM 26 18 0 15 Sep 2023
Audio Difference Learning for Audio Captioning Tatsuya Komatsu Yusuke Fujita K. Takeda T. Toda 37 3 0 15 Sep 2023
Zero-Shot Audio Captioning via Audibility Guidance Tal Shaharabany Ariel Shaulov Lior Wolf 13 4 0 07 Sep 2023
Generating Realistic Images from In-the-wild Sounds Taegyeong Lee Jeonghun Kang Hyeonyu Kim Taehwan Kim DiffM 24 1 0 05 Sep 2023
CoNeTTE: An efficient Audio Captioning system leveraging multiple datasets with Task Embedding Etienne Labbé Thomas Pellegrini J. Pinquier 12 10 0 01 Sep 2023
Audio Difference Captioning Utilizing Similarity-Discrepancy Disentanglement Daiki Takeuchi Yasunori Ohishi Daisuke Niizumi Noboru Harada K. Kashino 17 6 0 23 Aug 2023
Music Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioning Shansong Liu Atin Sakkeer Hussain Chenshuo Sun Yin Shan MLLM 24 27 0 22 Aug 2023
Separate Anything You Describe Xubo Liu Qiuqiang Kong Yan Zhao Haohe Liu Yiitan Yuan Yuzhuo Liu Rui Xia Yuxuan Wang Mark D. Plumbley Wenwu Wang VLM 22 43 0 09 Aug 2023
UnIVAL: Unified Model for Image, Video, Audio and Language Tasks Mustafa Shukor Corentin Dancette Alexandre Ramé Matthieu Cord MoMe MLLM 50 42 0 30 Jul 2023
Improving Audio Caption Fluency with Automatic Error Correction Hanxue Zhang Zeyu Xie Xuenan Xu Mengyue Wu K. Yu 18 0 0 16 Jun 2023
Enhance Temporal Relations in Audio Captioning with Sound Event Detection Zeyu Xie Xuenan Xu Mengyue Wu K. Yu 11 10 0 02 Jun 2023
Dual Transformer Decoder based Features Fusion Network for Automated Audio Captioning Jianyuan Sun Xubo Liu Xinhao Mei V. Kılıç Mark D. Plumbley Wenwu Wang 12 3 0 30 May 2023
Listen, Think, and Understand Yuan Gong Hongyin Luo Alexander H. Liu Leonid Karlinsky James R. Glass ELM MLLM LRM 29 135 0 18 May 2023
Multitask learning in Audio Captioning: a sentence embedding regression loss acts as a regularizer Etienne Labbé J. Pinquier Thomas Pellegrini 33 5 0 02 May 2023
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset Sihan Chen Xingjian He Longteng Guo Xinxin Zhu Weining Wang Jinhui Tang Jinhui Tang VLM 26 102 0 17 Apr 2023
Efficient Audio Captioning Transformer with Patchout and Text Guidance Thodoris Kouzelis Grigoris Bastas Athanasios Katsamanis Alexandros Potamianos ViT 10 6 0 06 Apr 2023
Prefix tuning for automated audio captioning Minkyu Kim Kim Sung-Bin Tae-Hyun Oh 8 42 0 30 Mar 2023
WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research Xinhao Mei Chutong Meng Haohe Liu Qiuqiang Kong Tom Ko Chengqi Zhao Mark D. Plumbley Yuexian Zou Wenwu Wang 43 192 0 30 Mar 2023
Fine-grained Audible Video Description Xuyang Shen Dong Li Jinxing Zhou Zhen Qin Bowen He ... Yuchao Dai Lingpeng Kong Meng Wang Yu Qiao Yiran Zhong VGen 36 11 0 27 Mar 2023
eP-ALM: Efficient Perceptual Augmentation of Language Models Mustafa Shukor Corentin Dancette Matthieu Cord MLLM VLM 24 29 0 20 Mar 2023
MAViL: Masked Audio-Video Learners Po-Yao (Bernie) Huang Vasu Sharma Hu Xu Chaitanya K. Ryali Haoqi Fan Yanghao Li Shang-Wen Li Gargi Ghosh Jitendra Malik Christoph Feichtenhofer 19 50 0 15 Dec 2022
Towards Generating Diverse Audio Captions via Adversarial Training Xinhao Mei Xubo Liu Jianyuan Sun Mark D. Plumbley Wenwu Wang DiffM 25 2 0 05 Dec 2022
Exploring Train and Test-Time Augmentations for Audio-Language Learning Eungbeom Kim Jinhee Kim Yoori Oh Kyungsu Kim Minju Park Jaeheon Sim J. Lee Kyogu Lee 10 10 0 31 Oct 2022
Visually-Aware Audio Captioning With Adaptive Audio-Visual Attention Xubo Liu Qiushi Huang Xinhao Mei Haohe Liu Qiuqiang Kong ... Yu Zhang Lilian H. Y. Tang Mark D. Plumbley Volkan Kilicc Wenwu Wang 36 18 0 28 Oct 2022
Automated Audio Captioning via Fusion of Low- and High- Dimensional Features Jianyuan Sun Xubo Liu Xinhao Mei Mark D. Plumbley V. Kılıç Wenwu Wang 17 3 0 10 Oct 2022
Text-to-Audio Grounding Based Novel Metric for Evaluating Audio Caption Similarity Swapnil Bhosale Rupayan Chakraborty Sunil Kumar Kopparapu 9 1 0 03 Oct 2022
Diffsound: Discrete Diffusion Model for Text-to-sound Generation Dongchao Yang Jianwei Yu Helin Wang Wen Wang Chao Weng Yuexian Zou Dong Yu DiffM 25 295 0 20 Jul 2022
Automated Audio Captioning and Language-Based Audio Retrieval Clive Gomes Hyejin Park Patrick Kollman Yi-Zhe Song Iffanice Houndayi Ankit Parag Shah 20 1 0 08 Jul 2022
Event-related data conditioning for acoustic event classification Yuanbo Hou Dick Botteldooren 9 3 0 16 Jun 2022