Clotho: An Audio Captioning Dataset

21 October 2019

Papers citing "Clotho: An Audio Captioning Dataset"

50 / 259 papers shown

Title
CLAIR-A: Leveraging Large Language Models to Judge Audio Captions Tsung-Han Wu Joseph E. Gonzalez Trevor Darrell David M. Chan 22 2 0 19 Sep 2024
Enhancing Low-Resource Language and Instruction Following Capabilities of Audio Language Models Potsawee Manakul Guangzhi Sun Warit Sirichotedumrong Kasima Tharnpipitchai Kunat Pipatanakul AuLLM 36 4 0 17 Sep 2024
Language-Queried Target Sound Extraction Without Parallel Training Data Hao Ma Zhiyuan Peng Xu Li Yukai Li Mingjie Shao Qiuqiang Kong Ju Liu VLM 69 1 0 14 Sep 2024
Recall: Empowering Multimodal Embedding for Edge Devices Dongqi Cai Shangguang Wang Chen Peng Zeling Zhang Mengwei Xu 27 3 0 09 Sep 2024
EnCLAP++: Analyzing the EnCLAP Framework for Optimizing Automated Audio Captioning Performance Jaeyeon Kim Minjeon Jeon Jaeyoon Jung Sang Hoon Woo Jinjoo Lee 26 2 0 02 Sep 2024
Expanding on EnCLAP with Auxiliary Retrieval Model for Automated Audio Captioning Jaeyeon Kim Jaeyoon Jung Minjeong Jeon Sang Hoon Woo Jinjoo Lee 24 1 0 02 Sep 2024
Dissecting Temporal Understanding in Text-to-Audio Retrieval Andreea-Maria Oncescu João F. Henriques A. Sophia Koepke 26 2 0 01 Sep 2024
SpeechCraft: A Fine-grained Expressive Speech Dataset with Natural Language Description Zeyu Jin Jia Jia Qixin Wang Kehan Li Shuoyi Zhou Songtao Zhou Xiaoyu Qin Zhiyong Wu 27 10 0 24 Aug 2024
On Class Separability Pitfalls In Audio-Text Contrastive Zero-Shot Learning Tiago Tavares Fabio Ayres Zhepei Wang Paris Smaragdis VLM 31 2 0 23 Aug 2024
Estimated Audio-Caption Correspondences Improve Language-Based Audio Retrieval Paul Primus Florian Schmid Gerhard Widmer 29 2 0 21 Aug 2024
Start from Video-Music Retrieval: An Inter-Intra Modal Loss for Cross Modal Retrieval Zeyu Chen Pengfei Zhang Kai Ye Wei Dong Xin Feng Yana Zhang 41 0 0 28 Jul 2024
Audio Entailment: Assessing Deductive Reasoning for Audio Understanding Soham Deshmukh Shuo Han Hazim T. Bukhari Benjamin Elizalde Hannes Gamper Rita Singh Bhiksha Raj ReLM LRM AuLLM 27 7 0 25 Jul 2024
Computer Audition: From Task-Specific Machine Learning to Foundation Models Andreas Triantafyllopoulos Iosif Tsangko Alexander Gebhard A. Mesaros Tuomas Virtanen Björn Schuller 45 4 0 22 Jul 2024
OmniBind: Large-scale Omni Multimodal Representation via Binding Spaces Zehan Wang Ziang Zhang Hang Zhang Luping Liu Rongjie Huang Xize Cheng Hengshuang Zhao Zhou Zhao 43 9 0 16 Jul 2024
Qwen2-Audio Technical Report Yunfei Chu Jin Xu Qian Yang Haojie Wei Xipin Wei ... Yuanjun Lv Jinzheng He Junyang Lin Chang Zhou Jingren Zhou AuLLM VLM 37 104 0 15 Jul 2024
S3: A Simple Strong Sample-effective Multimodal Dialog System Elisei Rykov Egor Malkershin Alexander Panchenko 20 0 0 26 Jun 2024
Fusing Audio and Metadata Embeddings Improves Language-based Audio Retrieval Paul Primus Gerhard Widmer 49 3 0 22 Jun 2024
GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities Sreyan Ghosh Sonal Kumar Ashish Seth Chandra Kiran Reddy Evuru Utkarsh Tyagi S. Sakshi Oriol Nieto R. Duraiswami Dinesh Manocha AuLLM LRM 46 36 0 17 Jun 2024
Performance Improvement of Language-Queried Audio Source Separation Based on Caption Augmentation From Large Language Models for DCASE Challenge 2024 Task 9 Do Hyun Lee Yoonah Song Hong Kook Kim 22 2 0 17 Jun 2024
MINT: a Multi-modal Image and Narrative Text Dubbing Dataset for Foley Audio Content Planning and Generation Ruibo Fu Shuchen Shi Hongming Guo Tao Wang Chunyu Qiang ... Zhiyong Wang Yukun Liu Xuefei Liu Shuai Zhang Guanjun Li VGen 30 0 0 15 Jun 2024
Explore the Limits of Omni-modal Pretraining at Scale Yiyuan Zhang Handong Li Jing Liu Xiangyu Yue VLM LRM 49 1 0 13 Jun 2024
Zero-Shot Audio Captioning Using Soft and Hard Prompts Yiming Zhang Xuenan Xu Ruoyi Du Haohe Liu Yuan Dong Zheng-Hua Tan Wenwu Wang Zhanyu Ma VLM 33 4 0 10 Jun 2024
Soundscape Captioning using Sound Affective Quality Network and Large Language Model Yuanbo Hou Qiaoqiao Ren A. Mitchell Wenwu Wang Jian Kang Tony Belpaeme Dick Botteldooren 34 3 0 09 Jun 2024
AudioLCM: Text-to-Audio Generation with Latent Consistency Models Huadai Liu Rongjie Huang Yang Liu Hengyuan Cao Jialei Wang Xize Cheng Siqi Zheng Zhou Zhao 68 8 0 01 Jun 2024
Listenable Maps for Zero-Shot Audio Classifiers Francesco Paissan Luca Della Libera Mirco Ravanelli Cem Subakan 32 4 0 27 May 2024
Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs Mustafa Shukor Matthieu Cord 68 5 0 26 May 2024
A Survey of Multimodal Large Language Model from A Data-centric Perspective Tianyi Bai Hao Liang Binwang Wan Yanran Xu Xi Li ... Ping-Chia Huang Jiulong Shan Conghui He Binhang Yuan Wentao Zhang 47 36 0 26 May 2024
Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts Yunxin Li Shenyuan Jiang Baotian Hu Longyue Wang Wanqi Zhong Wenhan Luo Lin Ma Min-Ling Zhang MoE 46 28 0 18 May 2024
Revisiting Deep Audio-Text Retrieval Through the Lens of Transportation Manh Luong Khai Nguyen Nhat Ho Reza Haf D.Q. Phung Lizhen Qu 30 12 0 16 May 2024
FreeBind: Free Lunch in Unified Multimodal Space via Knowledge Fusion Zehan Wang Ziang Zhang Xize Cheng Rongjie Huang Luping Liu ... Haifeng Huang Yang Zhao Tao Jin Peng Gao Zhou Zhao 31 8 0 08 May 2024
Distance Sampling-based Paraphraser Leveraging ChatGPT for Text Data Manipulation Yoori Oh Yoseob Han Kyogu Lee 37 1 0 01 May 2024
T-CLAP: Temporal-Enhanced Contrastive Language-Audio Pretraining Yiitan Yuan Zhuo Chen Xubo Liu Haohe Liu Xuenan Xu Dongya Jia Yuanzhe Chen Mark D. Plumbley Wenwu Wang CLIP VLM 40 9 0 27 Apr 2024
Hear Me, See Me, Understand Me: Audio-Visual Autism Behavior Recognition Shijian Deng Erin E. Kosloski Siddhi Patel Zeke A. Barnett Yiyang Nan ... William T. Doan Matthew Wang Harsh Singh P. Rollins Yapeng Tian 31 4 0 22 Mar 2024
InternVideo2: Scaling Video Foundation Models for Multimodal Video Understanding Yi Wang Kunchang Li Xinhao Li Jiashuo Yu Yinan He ... Hongjie Zhang Yifei Huang Yu Qiao Yali Wang Limin Wang 34 44 0 22 Mar 2024
Building speech corpus with diverse voice characteristics for its prompt-based representation Aya Watanabe Shinnosuke Takamichi Yuki Saito Wataru Nakata Detai Xin Hiroshi Saruwatari 27 0 0 20 Mar 2024
Refining Knowledge Transfer on Audio-Image Temporal Agreement for Audio-Text Cross Retrieval Shunsuke Tsubaki Daisuke Niizumi Daiki Takeuchi Yasunori Ohishi Noboru Harada Keisuke Imoto 19 1 0 16 Mar 2024
Multiscale Matching Driven by Cross-Modal Similarity Consistency for Audio-Text Retrieval Qian Wang Jia-Chen Gu Zhen-Hua Ling 35 2 0 15 Mar 2024
A Detailed Audio-Text Data Simulation Pipeline using Single-Event Sounds Xuenan Xu Xiaohang Xu Zeyu Xie Pingyue Zhang Mengyue Wu Kai Yu 28 6 0 07 Mar 2024
A SOUND APPROACH: Using Large Language Models to generate audio descriptions for egocentric text-audio retrieval Andreea-Maria Oncescu João F. Henriques Andrew Zisserman Samuel Albanie A. Sophia Koepke 23 5 0 29 Feb 2024
Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing? Marco Gaido Sara Papi Matteo Negri L. Bentivogli 41 12 0 19 Feb 2024
AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension Qian Yang Jin Xu Wenrui Liu Yunfei Chu Ziyue Jiang ... Yichong Leng Yuanjun Lv Zhou Zhao Chang Zhou Jingren Zhou LM&MA AuLLM ALM 44 57 0 12 Feb 2024
Cacophony: An Improved Contrastive Audio-Text Model Ge Zhu Jordan Darefsky Zhiyao Duan AuLLM 40 11 0 10 Feb 2024
Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities Zhifeng Kong Arushi Goel Rohan Badlani Wei Ping Rafael Valle Bryan Catanzaro AuLLM LM&MA MLLM 66 73 0 02 Feb 2024
EnCLAP: Combining Neural Audio Codec and Audio-Text Joint Embedding for Automated Audio Captioning Jaeyeon Kim Jaeyoon Jung Jinjoo Lee Sang Hoon Woo CLIP VLM 18 21 0 31 Jan 2024
A Survey on Data Augmentation in Large Model Era Yue Zhou Chenlu Guo Xu Wang Yi-Ju Chang Yuan Wu LM&MA VLM 49 23 0 27 Jan 2024
CoAVT: A Cognition-Inspired Unified Audio-Visual-Text Pre-Training Model for Multimodal Processing Xianghu Yue Xiaohai Tian Lu Lu Malu Zhang Zhizheng Wu Haizhou Li 39 0 0 22 Jan 2024
Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data Yuhui Zhang Elaine Sui Serena Yeung-Levy 29 9 0 16 Jan 2024
GroundingGPT:Language Enhanced Multi-modal Grounding Model Zhaowei Li Qi Xu Dong Zhang Hang Song Yiqing Cai ... Junting Pan Zefeng Li Van Tu Vu Zhida Huang Tao Wang 28 37 0 11 Jan 2024
Learning Audio Concepts from Counterfactual Natural Language A. Vosoughi Luca Bondi Ho-Hsiang Wu Chenliang Xu CML 47 3 0 10 Jan 2024
Towards Weakly Supervised Text-to-Audio Grounding Xuenan Xu Ziyang Ma Mengyue Wu Kai Yu AI4TS 28 9 0 05 Jan 2024