Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study

30 March 2023

Yong Cao

Li Zhou

Seolhwa Lee

Laura Cabello

Min Chen

Daniel Hershcovich

ArXiv PDF HTML

Papers citing "Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study"

50 / 102 papers shown

Title
The Echoes of Multilinguality: Tracing Cultural Value Shifts during LM Fine-tuning Rochelle Choenni Anne Lauscher Ekaterina Shutova 26 7 0 21 May 2024
Tagengo: A Multilingual Chat Dataset P. Devine 21 3 0 21 May 2024
"They are uncultured": Unveiling Covert Harms and Social Threats in LLM Generated Conversations Preetam Prabhu Srikar Dammu Hayoung Jung Anjali Singh Monojit Choudhury Tanushree Mitra 19 8 0 08 May 2024
The high dimensional psychological profile and cultural bias of ChatGPT Hang Yuan Zhongyue Che Shao Li Yue Zhang Xiaomeng Hu Siyang Luo 22 2 0 06 May 2024
A Comprehensive Survey on Evaluating Large Language Model Applications in the Medical Industry Yining Huang Keke Tang Meilian Chen Boyuan Wang ELM LM&MA 19 13 0 24 Apr 2024
CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies Weiyan Shi Ryan Li Yutong Zhang Caleb Ziems Chunhua yu R. Horesh Rogério Abreu de Paula Diyi Yang 28 26 0 23 Apr 2024
Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approaches Pablo Biedma Xiaoyuan Yi Linus Huang Maosong Sun Xing Xie PILM 32 3 0 19 Apr 2024
CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs' (Lack of) Multicultural Knowledge Yu Ying Chiu Amirhossein Ajalloeian Maria Antoniak Chan Young Park Shuyue Stella Li Mehar Bhatia Sahithya Ravi Yulia Tsvetkov Vered Shwartz Yejin Choi 36 20 0 10 Apr 2024
ChatGPT Role-play Dataset: Analysis of User Motives and Model Naturalness Sabrina Bodmer Ameeta Agrawal Judit Dombi Tetyana Sydorenko Jung In Lee 19 4 0 26 Mar 2024
Evaluating LLMs for Gender Disparities in Notable Persons L. Rhue Sofie Goethals Arun Sundararajan 26 4 0 14 Mar 2024
Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews Weixin Liang Zachary Izzo Yaohui Zhang Haley Lepp Hancheng Cao ... Haotian Ye Sheng Liu Zhi Huang Daniel A. McFarland James Y. Zou DeLMO 68 78 0 11 Mar 2024
Towards Measuring and Modeling "Culture" in LLMs: A Survey Muhammad Farid Adilazuarda Sagnik Mukherjee Pradhyumna Lavania Siddhant Singh Alham Fikri Aji Jacki OÑeill Ashutosh Modi Monojit Choudhury 50 53 0 05 Mar 2024
Exploring Multilingual Concepts of Human Value in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages? Shaoyang Xu Weilong Dong Zishan Guo Xinwei Wu Deyi Xiong 33 6 0 28 Feb 2024
DOSA: A Dataset of Social Artifacts from Different Indian Geographical Subcultures Agrima Seth Sanchit Ahuja Kalika Bali Sunayana Sitaram 30 8 0 23 Feb 2024
Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performance Ziqi Yin Hao Wang Kaito Horio Daisuke Kawahara Satoshi Sekine 44 20 0 22 Feb 2024
KorNAT: LLM Alignment Benchmark for Korean Social Values and Common Knowledge Jiyoung Lee Minwoo Kim Seungho Kim Junghwan Kim Seunghyun Won Hwaran Lee Edward Choi ALM 18 11 0 21 Feb 2024
Investigating Cultural Alignment of Large Language Models Badr AlKhamissi Muhammad N. ElNokrashy Mai AlKhamissi Mona T. Diab 29 42 0 20 Feb 2024
Stick to Your Role! Context-dependence and Stability of Personal Value Expression in Large Language Models Grgur Kovač Rémy Portelas Masataka Sawayama Peter Ford Dominey Pierre-Yves Oudeyer LLMAG 24 1 0 19 Feb 2024
CultureLLM: Incorporating Cultural Differences into Large Language Models Cheng-rong Li Mengzhou Chen Jindong Wang Sunayana Sitaram Xing Xie VLM 49 17 0 09 Feb 2024
Exploring Visual Culture Awareness in GPT-4V: A Comprehensive Probing Yong Cao Wenyan Li Jiaang Li Yifei Yuan Antonia Karamolegkou Daniel Hershcovich VLM 20 7 0 08 Feb 2024
Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs Simone Balloccu Patrícia Schmidtová Mateusz Lango Ondrej Dusek SILM ELM PILM 16 152 0 06 Feb 2024
Generative AI to Generate Test Data Generators Benoit Baudry K. Etemadi Sen Fang Yogya Gamage Yi Liu Yuxin Liu Martin Monperrus Javier Ron André Silva Deepika Tiwari 19 5 0 31 Jan 2024
Appropriateness of LLM-equipped Robotic Well-being Coach Language in the Workplace: A Qualitative Evaluation Micol Spitale Minja Axelsson Hatice Gunes 11 9 0 26 Jan 2024
Bridging Cultural Nuances in Dialogue Agents through Cultural Value Surveys Yong Cao Min Chen Daniel Hershcovich 16 5 0 18 Jan 2024
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems Tianyu Cui Yanling Wang Chuanpu Fu Yong Xiao Sijia Li ... Junwu Xiong Xinyu Kong Zujie Wen Ke Xu Qi Li 52 56 0 11 Jan 2024
Exploring the Capabilities of ChatGPT in Ancient Chinese Translation and Person Name Recognition Shijing Si Siqing Zhou Le Tang Xiaoqing Cheng Yugui Zhang 18 0 0 23 Dec 2023
From Bytes to Biases: Investigating the Cultural Self-Perception of Large Language Models Wolfgang Messner Tatum Greene Josephine Matalone 19 4 0 21 Dec 2023
Culturally Responsive Artificial Intelligence -- Problems, Challenges and Solutions Natalia O.zegalska-Lukasik Szymon Lukasik 14 2 0 13 Dec 2023
CDEval: A Benchmark for Measuring the Cultural Dimensions of Large Language Models Yuhang Wang Yanxu Zhu Chao Kong Shuyu Wei Xiaoyuan Yi Xing Xie Jitao Sang ALM VLM ELM 8 5 0 28 Nov 2023
Cultural Bias and Cultural Alignment of Large Language Models Yan Tao Olga Viberg Ryan S. Baker René F. Kizilcec ELM 21 72 0 23 Nov 2023
Cultural Adaptation of Recipes Yong Cao Yova Kementchedjhieva Ruixiang Cui Antonia Karamolegkou Li Zhou Megan Dare Lucia Donatelli Daniel Hershcovich 18 5 0 26 Oct 2023
GD-COMET: A Geo-Diverse Commonsense Inference Model Mehar Bhatia Vered Shwartz 16 6 0 23 Oct 2023
Denevil: Towards Deciphering and Navigating the Ethical Values of Large Language Models via Instruction Learning Shitong Duan Xiaoyuan Yi Peng Zhang T. Lu Xing Xie Ning Gu 11 9 0 17 Oct 2023
Cultural Compass: Predicting Transfer Learning Success in Offensive Language Detection with Cultural Features Li Zhou Antonia Karamolegkou Wenyu Chen Daniel Hershcovich 40 11 0 10 Oct 2023
Navigating Cultural Chasms: Exploring and Unlocking the Cultural POV of Text-To-Image Models Mor Ventura Eyal Ben-David Anna Korhonen Roi Reichart 21 12 0 03 Oct 2023
People's Perceptions Toward Bias and Related Concepts in Large Language Models: A Systematic Review Lu Wang Max Song R. Rezapour Bum Chul Kwon Jina Huh-Yoo AI4CE 22 1 0 25 Sep 2023
Are Multilingual LLMs Culturally-Diverse Reasoners? An Investigation into Multicultural Proverbs and Sayings Chen Cecilia Liu Fajri Koto Timothy Baldwin Iryna Gurevych LRM 30 17 0 15 Sep 2023
A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time Yeqi Gao Zhao-quan Song Weixin Wang Junze Yin 18 25 0 14 Sep 2023
Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensions Reem I. Masoud Ziquan Liu Martin Ferianc Philip C. Treleaven Miguel R. D. Rodrigues 11 49 0 25 Aug 2023
LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzles Shulin Huang Shirong Ma Yinghui Li Mengzuo Huang Wuhe Zou Weidong Zhang Haitao Zheng LLMAG LRM 24 26 0 21 Aug 2023
What has ChatGPT read? The origins of archaeological citations used by a generative artificial intelligence application D. Spennemann 14 2 0 07 Aug 2023
Large Language Models as Superpositions of Cultural Perspectives Grgur Kovač Masataka Sawayama Rémy Portelas Cédric Colas Peter Ford Dominey Pierre-Yves Oudeyer LLMAG 20 32 0 15 Jul 2023
A Survey on Evaluation of Large Language Models Yu-Chu Chang Xu Wang Jindong Wang Yuanyi Wu Linyi Yang ... Yue Zhang Yi-Ju Chang Philip S. Yu Qian Yang Xingxu Xie ELM LM&MA ALM 58 1,464 0 06 Jul 2023
Multilingual Language Models are not Multicultural: A Case Study in Emotion Shreya Havaldar Sunny Rai Bhumika Singhal Langchen Liu Langchen Liu Sharath Chandra Guntuku Lyle Ungar 18 56 0 03 Jul 2023
Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias Yue Yu Yuchen Zhuang Jieyu Zhang Yu Meng Alexander Ratner Ranjay Krishna Jiaming Shen Chao Zhang ALM 28 200 0 28 Jun 2023
This Land is {Your, My} Land: Evaluating Geopolitical Biases in Language Models Bryan Li Samar Haider Chris Callison-Burch 8 15 0 24 May 2023
Having Beer after Prayer? Measuring Cultural Bias in Large Language Models Tarek Naous Michael Joseph Ryan Alan Ritter Wei-ping Xu 24 82 0 23 May 2023
CHEAT: A Large-scale Dataset for Detecting ChatGPT-writtEn AbsTracts Peipeng Yu Jiahan Chen Xuan Feng Zhihua Xia 55 39 0 24 Apr 2023
Does Moral Code Have a Moral Code? Probing Delphi's Moral Philosophy Kathleen C. Fraser S. Kiritchenko Esma Balkir 99 36 0 25 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,730 0 04 Mar 2022