Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generation

4 April 2023

Esa Rahtu

Shiníchi Satoh

Papers citing "Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generation"

43 / 43 papers shown

Title
Direct Motion Models for Assessing Generated Videos Kelsey R. Allen Carl Doersch Guangyao Zhou Mohammed Suhail Danny Driess ... Thomas Kipf Mehdi S. M. Sajjadi Kevin P. Murphy João Carreira Sjoerd van Steenkiste EGVM DiffM VGen 74 0 0 30 Apr 2025
Video-Bench: Human-Aligned Video Generation Benchmark Hui Han Siyuan Li Jiaqi Chen Yiwen Yuan Yuling Wu ... Y. Li J. Zhang Chi Zhang Li Li Yongxin Ni EGVM VGen 65 0 0 07 Apr 2025
SEED: Towards More Accurate Semantic Evaluation for Visual Brain Decoding Juhyeon Park P. Y. Kim Jiook Cha Shinjae Yoo Taesup Moon 48 0 0 09 Mar 2025
Scalable Ranked Preference Optimization for Text-to-Image Generation Shyamgopal Karthik Huseyin Coskun Zeynep Akata Sergey Tulyakov J. Ren Anil Kag EGVM 52 4 0 23 Oct 2024
Trustworthy Text-to-Image Diffusion Models: A Timely and Focused Survey Yi Zhang Zhen Chen Chih-Hong Cheng Wenjie Ruan Xiaowei Huang Dezong Zhao David Flynn Siddartha Khastgir Xingyu Zhao MedIm 30 3 0 26 Sep 2024
Rethinking Sparse Lexical Representations for Image Retrieval in the Age of Rising Multi-Modal Large Language Models K. Nakata Daisuke Miyashita Youyang Ng Yasuto Hoshi J. Deguchi 21 0 0 29 Aug 2024
ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation Shenghai Yuan Jinfa Huang Yongqi Xu Yaoyang Liu Shaofeng Zhang Yujun Shi Ruijie Zhu Xinhua Cheng Jiebo Luo Li Yuan EGVM VGen 66 34 0 26 Jun 2024
EVALALIGN: Supervised Fine-Tuning Multimodal LLMs with Human-Aligned Data for Evaluating Text-to-Image Models Zhiyu Tan Xiaomeng Yang Luozheng Qin Mengping Yang Cheng Zhang Hao Li 42 7 0 24 Jun 2024
GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation Baiqi Li Zhiqiu Lin Deepak Pathak Jiayao Li Yixin Fei ... Tiffany Ling Xide Xia Pengchuan Zhang Graham Neubig Deva Ramanan EGVM 44 24 0 19 Jun 2024
Consistency-diversity-realism Pareto fronts of conditional image generative models Pietro Astolfi Marlene Careil Melissa Hall Oscar Manas Matthew Muckley Jakob Verbeek Adriana Romero Soriano M. Drozdzal 45 10 0 14 Jun 2024
Rethinking Human Evaluation Protocol for Text-to-Video Models: Enhancing Reliability,Reproducibility, and Practicality Tianle Zhang Langtian Ma Yuchen Yan Yuchen Zhang Kai Wang ... Wenqi Shao Yang You Yu Qiao Ping Luo Kaipeng Zhang VGen 58 2 0 13 Jun 2024
GAIA: Rethinking Action Quality Assessment for AI-Generated Videos Zijian Chen Wei Sun Yuan Tian Jun Jia Zicheng Zhang Jiarui Wang Ru Huang Xiongkuo Min Guangtao Zhai Wenjun Zhang EGVM 48 10 0 10 Jun 2024
GenAI Arena: An Open Evaluation Platform for Generative Models Dongfu Jiang Max W.F. Ku Tianle Li Yuansheng Ni Shizhuo Sun Rongqi Fan Wenhu Chen EGVM 39 20 0 06 Jun 2024
ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization L. Eyring Shyamgopal Karthik Karsten Roth Alexey Dosovitskiy Zeynep Akata 74 16 0 06 Jun 2024
Greedy Growing Enables High-Resolution Pixel-Based Diffusion Models C. N. Vasconcelos Abdullah Rashwan Austin Waters Trevor Walker Keyang Xu Jimmy Yan ... Wenlei Zhou Kevin Swersky David J. Fleet Jason Baldridge Oliver Wang 39 3 0 27 May 2024
Erasing Concepts from Text-to-Image Diffusion Models with Few-shot Unlearning Masane Fuchi Tomohiro Takagi DiffM VLM 45 13 0 12 May 2024
DOCCI: Descriptions of Connected and Contrasting Images Yasumasa Onoe Sunayana Rane Zachary Berger Yonatan Bitton Jaemin Cho ... Zarana Parekh Jordi Pont-Tuset Garrett Tanzer Su Wang Jason Baldridge 31 48 0 30 Apr 2024
PKU-AIGIQA-4K: A Perceptual Quality Assessment Database for Both Text-to-Image and Image-to-Image AI-Generated Images Jiquan Yuan Fanyi Yang Jihe Li Xinyan Cao Jinming Che Jinlong Lin Xixin Cao EGVM 21 2 0 29 Apr 2024
ANCHOR: LLM-driven News Subject Conditioning for Text-to-Image Synthesis Aashish Anantha Ramakrishnan Sharon X. Huang Dongwon Lee 24 0 0 15 Apr 2024
Evaluating Text-to-Visual Generation with Image-to-Text Generation Zhiqiu Lin Deepak Pathak Baiqi Li Jiayao Li Xide Xia Graham Neubig Pengchuan Zhang Deva Ramanan EGVM 37 125 0 01 Apr 2024
PFStorer: Personalized Face Restoration and Super-Resolution Tuomas Varanka Tapani Toivonen Soumya Tripathy Guoying Zhao Erman Acar DiffM 34 2 0 13 Mar 2024
Evaluating Text-to-Image Generative Models: An Empirical Study on Human Image Synthesis Mu-Hwa Chen Yi Liu Jian Yi Changran Xu Qiuxia Lai Hongliang Wang Tsung-Yi Ho Qiang Xu EGVM 19 7 0 08 Mar 2024
Towards A Better Metric for Text-to-Video Generation Jay Zhangjie Wu Guian Fang Haoning Wu Xintao Wang Yixiao Ge ... Rui Zhao Weisi Lin Wynne Hsu Ying Shan Mike Zheng Shou VGen 22 34 0 15 Jan 2024
M^2ConceptBase: A Fine-Grained Aligned Concept-Centric Multimodal Knowledge Base Zhiwei Zha Jiaan Wang Zhixu Li Xiangru Zhu Wei Song Yanghua Xiao VLM 20 2 0 16 Dec 2023
Rich Human Feedback for Text-to-Image Generation Youwei Liang Junfeng He Gang Li Peizhao Li Arseniy Klimovskiy ... Yiwen Luo Yang Li Kai Kohlhoff Deepak Ramachandran Vidhya Navalpakkam EGVM 14 66 0 15 Dec 2023
Exploring the Naturalness of AI-Generated Images Zijian Chen Wei Sun Haoning Wu Zicheng Zhang Jun Jia ... Fengyu Sun Shangling Jui Xiongkuo Min Guangtao Zhai Wenjun Zhang 11 19 0 09 Dec 2023
FERGI: Automatic Annotation of User Preferences for Text-to-Image Generation from Spontaneous Facial Expression Reaction Shuangquan Feng Junhua Ma Virginia R. de Sa EGVM 10 0 0 05 Dec 2023
Stable Diffusion Exposed: Gender Bias from Prompt to Image Yankun Wu Yuta Nakashima Noa Garcia 23 16 0 05 Dec 2023
VBench: Comprehensive Benchmark Suite for Video Generative Models Ziqi Huang Yinan He Jiashuo Yu Fan Zhang Chenyang Si ... Xinyuan Chen Limin Wang Dahua Lin Yu Qiao Ziwei Liu VGen 59 346 0 29 Nov 2023
Fair Text-to-Image Diffusion via Fair Mapping Jia Li Lijie Hu Jingfeng Zhang Tianhang Zheng Hua Zhang Di Wang 36 12 0 29 Nov 2023
PKU-I2IQA: An Image-to-Image Quality Assessment Database for AI Generated Images Jiquan Yuan Xinyan Cao Changjin Li Fanyi Yang Jinlong Lin Xixin Cao EGVM 24 18 0 27 Nov 2023
The Challenges of Image Generation Models in Generating Multi-Component Images Tham Yik Foong Shashank Kotyan Poyuan Mao Danilo Vasconcellos Vargas EGVM 34 1 0 22 Nov 2023
Holistic Evaluation of Text-To-Image Models Tony Lee Michihiro Yasunaga Chenlin Meng Yifan Mai Joon Sung Park ... Jun-Yan Zhu Fei-Fei Li Jiajun Wu Stefano Ermon Percy Liang 139 125 0 07 Nov 2023
Hierarchical Evaluation Framework: Best Practices for Human Evaluation I. Bojić Jessica Chen Si Yuan Chang Qi Chwen Ong Shafiq R. Joty Josip Car 12 5 0 03 Oct 2023
VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning Han Lin Abhaysinh Zala Jaemin Cho Mohit Bansal LM&Ro VGen DiffM 32 74 0 26 Sep 2023
Let's ViCE! Mimicking Human Cognitive Behavior in Image Generation Evaluation Federico Betti Jacopo Staiano Lorenzo Baraldi Lorenzo Baraldi Rita Cucchiara N. Sebe EGVM 8 6 0 18 Jul 2023
Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models G. Stein Jesse C. Cresswell Rasa Hosseinzadeh Yi Sui Brendan Leigh Ross Valentin Villecroze Zhaoyan Liu Anthony L. Caterini J. E. T. Taylor G. Loaiza-Ganem EGVM 17 94 0 07 Jun 2023
If at First You Don't Succeed, Try, Try Again: Faithful Diffusion-based Text-to-Image Generation by Selection Shyamgopal Karthik Karsten Roth Massimiliano Mancini Zeynep Akata 24 20 0 22 May 2023
X-IQE: eXplainable Image Quality Evaluation for Text-to-Image Generation with Visual Large Language Models Yixiong Chen Li Liu C. Ding 15 21 0 18 May 2023
ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation Jiazheng Xu Xiao Liu Yuchen Wu Yuxuan Tong Qinkai Li Ming Ding Jie Tang Yuxiao Dong 17 306 0 12 Apr 2023
The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation Marzena Karpinska Nader Akoury Mohit Iyyer 204 106 0 14 Sep 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 253 4,735 0 24 Feb 2021
How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models Ahmed Alaa B. V. Breugel Evgeny S. Saveliev M. Schaar 40 186 0 17 Feb 2021