v1v2 (latest)

Generating Visual Stories with Grounded and Coreferent Characters

20 September 2024

Danyang Liu

Mirella Lapata

Frank Keller

ArXiv (abs)PDF HTML

Papers citing "Generating Visual Stories with Grounded and Coreferent Characters"

36 / 36 papers shown

Title
SCORE: Story Coherence and Retrieval Enhancement for AI Narratives Qiang Yi Yangfan He Jing Wang Xinyuan Song Shiyao Qian ... Menghao Huo Kuan Lu Jiaqi Chen Lewei He Tianyu Shi RALM 571 60 0 30 Mar 2025
Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators Yinhong Liu Han Zhou Zhijiang Guo Ehsan Shareghi Ivan Vulić Anna Korhonen Nigel Collier ALM 807 123 0 20 Jan 2025
Not (yet) the whole story: Evaluating Visual Storytelling Requires More than Measuring Coherence, Grounding, and Repetition Aditya K Surikuchi Raquel Fernández Sandro Pezzelle 160 9 0 05 Jul 2024
From Persona to Personalization: A Survey on Role-Playing Language Agents Jiangjie Chen Xintao Wang Rui Xu Siyu Yuan Yikai Zhang ... Caiyu Hu Siye Wu Scott Ren Ziquan Fu Yanghua Xiao 299 165 0 28 Apr 2024
Zero-Shot Character Identification and Speaker Prediction in Comics via Iterative Multimodal Fusion Yingxuan Li Ryota Hinami Kiyoharu Aizawa Yusuke Matsui 127 11 0 22 Apr 2024
Character is Destiny: Can Large Language Models Simulate Persona-Driven Decisions in Role-Playing? Rui Xu Xintao Wang Jiangjie Chen Siyu Yuan Xinfeng Yuan Jiaqing Liang Zulong Chen Xiaoqing Dong Yanghua Xiao 266 13 0 18 Apr 2024
SCO-VIST: Social Interaction Commonsense Knowledge-based Visual Storytelling Eileen Wang S. Han Josiah Poon 201 5 0 01 Feb 2024
[Lions: 1] and [Tigers: 2] and [Bears: 3], Oh My! Literary Coreference Annotation with LLMs Rebecca M. M. Hicke David M. Mimno 126 10 0 31 Jan 2024
GROOViST: A Metric for Grounding Objects in Visual StorytellingConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Aditya K Surikuchi Sandro Pezzelle Raquel Fernández 112 13 0 26 Oct 2023
Visual Storytelling with Question-Answer Plans Danyang Liu Mirella Lapata Frank Keller CoGe 164 9 0 08 Oct 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot ArenaNeural Information Processing Systems (NeurIPS), 2023 Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 2.2K 6,246 0 09 Jun 2023
MIMIC-IT: Multi-Modal In-Context Instruction Tuning Yue Liu Yuanhan Zhang Liangyu Chen Jinghao Wang Fanyi Pu Jingkang Yang Xuefei Liu Ziwei Liu MLLM VLM 227 287 0 08 Jun 2023
Segment AnythingIEEE International Conference on Computer Vision (ICCV), 2023 A. Kirillov Eric Mintun Nikhila Ravi Hanzi Mao Chloe Rolland ... Spencer Whitehead Alexander C. Berg Wan-Yen Lo Piotr Dollár Ross B. Girshick MLLM VLM 786 10,568 0 05 Apr 2023
Detecting and Grounding Important Characters in Visual StoriesAAAI Conference on Artificial Intelligence (AAAI), 2023 Danyang Liu Frank Keller 146 11 0 30 Mar 2023
Visual Writing Prompts: Character-Grounded Story Generation with Curated Image SequencesTransactions of the Association for Computational Linguistics (TACL), 2023 Xudong Hong A. Sayeed K. Mehra Vera Demberg Bernt Schiele VGen 110 52 0 20 Jan 2023
Few-Shot Character Understanding in Movies as an Assessment to Meta-Learning of Theory-of-MindInternational Conference on Machine Learning (ICML), 2022 Mo Yu Qiujing Wang Shunchi Zhang Yisi Sang Kangsheng Pu ... Han Wang Liyan Xu Jing Li Yue Yu Jie Zhou 181 21 0 09 Nov 2022
"Let Your Characters Tell Their Story": A Dataset for Character-Centric Narrative Understanding Faeze Brahman Meng Huang Oyvind Tafjord Chao Zhao Mrinmaya Sachan Snigdha Chaturvedi 147 61 0 12 Sep 2021
Plot and Rework: Modeling Storylines for Visual StorytellingFindings (Findings), 2021 Chi-Yang Hsu Yun-Wei Chu Ting-Hao 'Kenneth' Huang Lun-Wei Ku 139 34 0 14 May 2021
CLIPScore: A Reference-free Evaluation Metric for Image CaptioningConference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Jack Hessel Ari Holtzman Maxwell Forbes Ronan Le Bras Yejin Choi CLIP 779 2,180 0 18 Apr 2021
Perceiver: General Perception with Iterative AttentionInternational Conference on Machine Learning (ICML), 2021 Andrew Jaegle Felix Gimeno Andrew Brock Andrew Zisserman Oriol Vinyals João Carreira VLM ViT MDE 401 1,216 0 04 Mar 2021
Learning Transferable Visual Models From Natural Language SupervisionInternational Conference on Machine Learning (ICML), 2021 Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 1.9K 39,712 0 26 Feb 2021
Commonsense Knowledge Aware Concept Selection For Diverse and Informative Visual StorytellingAAAI Conference on Artificial Intelligence (AAAI), 2021 Hong Chen Yifei Huang Hiroya Takamura Hideki Nakayama DiffM 147 48 0 05 Feb 2021
MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence FrontiersNeural Information Processing Systems (NeurIPS), 2021 Krishna Pillutla Swabha Swayamdipta Rowan Zellers John Thickstun Sean Welleck Yejin Choi Zaïd Harchaoui 305 439 0 02 Feb 2021
End-to-End Object Detection with TransformersEuropean Conference on Computer Vision (ECCV), 2020 Nicolas Carion Francisco Massa Gabriel Synnaeve Nicolas Usunier Alexander Kirillov Sergey Zagoruyko ViT 3DV PINN 1.8K 16,030 0 26 May 2020
Knowledge-Enriched Visual StorytellingAAAI Conference on Artificial Intelligence (AAAI), 2019 Chao-Chun Hsu Zi-Yuan Chen Chi-Yang Hsu Chih-Chia Li Tzu-Yuan Lin Ting-Hao 'Kenneth' Huang Lun-Wei Ku DiffM 126 52 0 03 Dec 2019
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionAnnual Meeting of the Association for Computational Linguistics (ACL), 2019 M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 714 11,902 0 29 Oct 2019
Character-Centric Storytelling Aditya K Surikuchi Jorma T. Laaksonen 113 5 0 17 Sep 2019
What Makes A Good Story? Designing Composite Rewards for Visual StorytellingAAAI Conference on Artificial Intelligence (AAAI), 2019 Junjie Hu Yu Cheng Zhe Gan Jingjing Liu Jianfeng Gao Graham Neubig 149 71 0 11 Sep 2019
The Steep Road to Happily Ever After: An Analysis of Current Visual Storytelling Models Yatri Modi Natalie Parde 99 16 0 06 Apr 2019
Plan-And-Write: Towards Better Automatic Storytelling Lili Yao Nanyun Peng R. Weischedel Kevin Knight Dongyan Zhao Rui Yan 263 438 0 14 Nov 2018
Contextualize, Show and Tell: A Neural Visual Storyteller Diana Gonzalez-Rico Gibran Fuentes Pineda 82 37 0 03 Jun 2018
GLAC Net: GLocal Attention Cascading Networks for Multi-image Cued Story Generation Taehyeong Kim Min-Oh Heo Seonil Son Kyoung-Wha Park Byoung-Tak Zhang 156 82 0 28 May 2018
UnitBox: An Advanced Object Detection Network Jiahui Yu Yuning Jiang Zinan Lin Zhimin Cao Thomas Huang CVBM 181 1,605 0 04 Aug 2016
Visual Relationship Detection with Language Priors Cewu Lu Ranjay Krishna Michael S. Bernstein Li Fei-Fei VLM 312 1,188 0 31 Jul 2016
Visual Storytelling Ting-Hao 'Kenneth' Huang Huang Francis Ferraro N. Mostafazadeh Ishan Misra ... C. L. Zitnick Devi Parikh Lucy Vanderwende Michel Galley Margaret Mitchell VGen 179 522 0 13 Apr 2016
FaceNet: A Unified Embedding for Face Recognition and Clustering Florian Schroff Dmitry Kalenichenko James Philbin 3DH 844 14,096 0 12 Mar 2015