v1v2 (latest)

Visually Grounded Neural Syntax Acquisition

7 June 2019

Jiayuan Mao

Papers citing "Visually Grounded Neural Syntax Acquisition"

50 / 57 papers shown

Title
Reframing linguistic bootstrapping as joint inference using visually-grounded grammar induction models Eva Portelance Siva Reddy Timothy J. O'Donnell 73 3 0 17 Jun 2024
Learning Language Structures through Grounding Freda Shi 80 2 0 14 Jun 2024
Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations Ziqiao Ma Zekun Wang Joyce Chai 146 4 0 22 May 2024
A Multimodal In-Context Tuning Approach for E-Commerce Product Description Generation Yunxin Li Baotian Hu Wenhan Luo Lin Ma Yuxin Ding Min Zhang 125 1 0 21 Feb 2024
OLViT: Multi-Modal State Tracking via Attention-Based Embeddings for Video-Grounded Dialog Adnen Abdessaied Manuel von Hochmeister Andreas Bulling 83 2 0 20 Feb 2024
Mitigating Open-Vocabulary Caption Hallucinations Assaf Ben-Kish Moran Yanuka Morris Alper Raja Giryes Hadar Averbuch-Elor MLLM VLM 123 6 0 06 Dec 2023
Towards Vision Enhancing LLMs: Empowering Multimodal Knowledge Storage and Sharing in LLMs Yunxin Li Baotian Hu Wei Wang Xiaochun Cao Min Zhang 77 5 0 27 Nov 2023
On the Transferability of Visually Grounded PCFGs Yanpeng Zhao Ivan Titov 59 1 0 21 Oct 2023
Audio-Visual Neural Syntax Acquisition Cheng-I Jeff Lai Freda Shi Puyuan Peng Yoon Kim Kevin Gimpel ... David D. Cox David Harwath Yang Zhang Karen Livescu James R. Glass CLIP NAI 71 1 0 11 Oct 2023
Ensemble Distillation for Unsupervised Constituency Parsing Behzad Shayegh Yanshuai Cao Xiaodan Zhu Jackie C.K. Cheung Lili Mou 146 5 0 03 Oct 2023
A Joint Study of Phrase Grounding and Task Performance in Vision and Language Models Noriyuki Kojima Hadar Averbuch-Elor Yoav Artzi 76 2 0 06 Sep 2023
A Multi-Modal Context Reasoning Approach for Conditional Inference on Joint Textual and Visual Clues Yunxin Li Baotian Hu Xinyu Chen Yuxin Ding Lin Ma Min Zhang LRM 93 15 0 08 May 2023
Is BERT Blind? Exploring the Effect of Vision-and-Language Pretraining on Visual Language Understanding Morris Alper Michael Fiman Hadar Averbuch-Elor VLM LRM 80 15 0 21 Mar 2023
Cascading and Direct Approaches to Unsupervised Constituency Parsing on Spoken Sentences Yuan Tseng Cheng-I Jeff Lai Hung-yi Lee SSL 73 4 0 15 Mar 2023
HiCLIP: Contrastive Language-Image Pretraining with Hierarchy-aware Attention Shijie Geng Jianbo Yuan Yu Tian Yuxiao Chen Yongfeng Zhang CLIP VLM 72 46 0 06 Mar 2023
Does Vision Accelerate Hierarchical Generalization of Neural Language Learners? Tatsuki Kuribayashi VLM 56 0 0 01 Feb 2023
How poor is the stimulus? Evaluating hierarchical generalization in neural networks trained on child-directed speech Aditya Yedetore Tal Linzen Robert Frank R. Thomas McCoy 74 19 0 26 Jan 2023
Universal Multimodal Representation for Language Understanding Zhuosheng Zhang Kehai Chen Rui Wang Masao Utiyama Eiichiro Sumita Z. Li Hai Zhao SSL 109 22 0 09 Jan 2023
Re-evaluating the Need for Multimodal Signals in Unsupervised Grammar Induction Boyi Li Rodolfo Corona K. Mangalam Catherine Chen Daniel Flaherty Serge Belongie Kilian Q. Weinberger Jitendra Malik Trevor Darrell Dan Klein 86 1 0 20 Dec 2022
Multilingual Multimodality: A Taxonomical Survey of Datasets, Techniques, Challenges and Opportunities Khyathi Chandu A. Geramifard 76 3 0 30 Oct 2022
Learning a Grammar Inducer from Massive Uncurated Instructional Videos Songyang Zhang Linfeng Song Lifeng Jin Haitao Mi Kun Xu Dong Yu Jiebo Luo 107 5 0 22 Oct 2022
Visualize Before You Write: Imagination-Guided Open-Ended Text Generation Wanrong Zhu An Yan Yujie Lu Wenda Xu Xinze Wang Miguel P. Eckstein William Yang Wang 130 36 0 07 Oct 2022
VALHALLA: Visual Hallucination for Machine Translation Yi Li Yikang Shen Yoon Kim Chun-Fu Chen Rogerio Feris David D. Cox Nuno Vasconcelos MLLM 144 40 0 31 May 2022
Unsupervised Slot Schema Induction for Task-oriented Dialog Dian Yu Mingqiu Wang Yuan Cao Izhak Shafran Laurent El Shafey H. Soltau 71 13 0 09 May 2022
Natural Language to Code Translation with Execution Freda Shi Daniel Fried Marjan Ghazvininejad Luke Zettlemoyer Sida I. Wang 129 129 0 25 Apr 2022
Imagination-Augmented Natural Language Understanding Yujie Lu Wanrong Zhu Xinze Wang Miguel P. Eckstein William Yang Wang 62 24 0 18 Apr 2022
Unsupervised Vision-Language Parsing: Seamlessly Bridging Visual Scene Graphs with Language Structures via Dependency Relationships Chao Lou Wenjuan Han Yuh-Chen Lin Zilong Zheng CoGe 84 10 0 27 Mar 2022
Finding Structural Knowledge in Multimodal-BERT Victor Milewski Miryam de Lhoneux Marie-Francine Moens 72 10 0 17 Mar 2022
Vision-Language Intelligence: Tasks, Representation Learning, and Large Models Feng Li Hao Zhang Yi-Fan Zhang Shixuan Liu Jian Guo L. Ni Pengchuan Zhang Lei Zhang AI4TS VLM 83 37 0 03 Mar 2022
Grammar-Based Grounded Lexicon Learning Jiayuan Mao Haoyue Shi Jiajun Wu R. Levy J. Tenenbaum NAI 95 15 0 17 Feb 2022
HAKE: A Knowledge Engine Foundation for Human Activity Understanding Yong-Lu Li Xinpeng Liu Xiaoqian Wu Yizhuo Li Zuoyu Qiu Liang Xu Yue Xu Haoshu Fang Cewu Lu 96 38 0 14 Feb 2022
Visually Grounded Concept Composition Bowen Zhang Hexiang Hu Linlu Qiu Peter Shaw Fei Sha CoGe 122 6 0 29 Sep 2021
Dependency Induction Through the Lens of Visual Perception Ruisi Su Shruti Rijhwani Hao Zhu Junxian He Xinyu Wang Yonatan Bisk Graham Neubig 70 3 0 20 Sep 2021
Improved Latent Tree Induction with Distant Supervision via Span Constraints Zhiyang Xu Andrew Drozdov Jay Yoon Lee Timothy J. O'Gorman Subendhu Rongali Dylan Finkbeiner S. Suresh Mohit Iyyer Andrew McCallum 72 9 0 10 Sep 2021
Sequence-to-Sequence Learning with Latent Neural Grammars Yoon Kim 168 40 0 02 Sep 2021
Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis Wei Han Hui Chen Alexander Gelbukh Amir Zadeh Louis-Philippe Morency Soujanya Poria 70 185 0 28 Jul 2021
Grounding 'Grounding' in NLP Khyathi Chandu Yonatan Bisk A. Black 96 54 0 04 Jun 2021
Neural Bi-Lexicalized PCFG Induction Aaron Courville Yanpeng Zhao Kewei Tu 76 23 0 31 May 2021
Learning Syntax from Naturally-Occurring Bracketings Tianze Shi Ozan Irsoy Igor Malioutov Lillian Lee 74 6 0 28 Apr 2021
Cetacean Translation Initiative: a roadmap to deciphering the communication of sperm whales Jacob Andreas Gašper Beguš M. Bronstein R. Diamant Denley Delaney ... D. Tchernov P. Tønnesen Antonio Torralba Daniel M. Vogt Robert J. Wood 60 10 0 17 Apr 2021
Video-aided Unsupervised Grammar Induction Songyang Zhang Linfeng Song Lifeng Jin Kun Xu Dong Yu Jiebo Luo 63 27 0 09 Apr 2021
VLGrammar: Grounded Grammar Induction of Vision and Language Yining Hong Qing Li Song-Chun Zhu Siyuan Huang VLM 89 25 0 24 Mar 2021
KANDINSKYPatterns -- An experimental exploration environment for Pattern Analysis and Machine Intelligence Andreas Holzinger Anna Saranti Heimo Mueller 114 10 0 28 Feb 2021
Data-efficient Alignment of Multimodal Sequences by Aligning Gradient Updates and Internal Feature Distributions Jianan Wang Boyang Albert Li Xiangyu Fan Jing-Hua Lin Yanwei Fu 54 2 0 15 Nov 2020
Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision Hao Tan Joey Tianyi Zhou CLIP 89 121 0 14 Oct 2020
On the Role of Supervision in Unsupervised Constituency Parsing Freda Shi Karen Livescu Kevin Gimpel SSL 72 22 0 06 Oct 2020
Visually Grounded Compound PCFGs Yanpeng Zhao Ivan Titov 80 45 0 25 Sep 2020
Analogical Reasoning for Visually Grounded Language Acquisition Bo Wu Haoyu Qin Alireza Zareian Carl Vondrick Shih-Fu Chang 46 9 0 22 Jul 2020
Tree-Augmented Cross-Modal Encoding for Complex-Query Video Retrieval Xun Yang Jianfeng Dong Yixin Cao Xun Wang Meng Wang Tat-Seng Chua 65 140 0 06 Jul 2020
The Importance of Category Labels in Grammar Induction with Child-directed Utterances Lifeng Jin William Schuler 41 3 0 20 Jun 2020