How Can We Know What Language Models Know?

28 November 2019

Graham Neubig

Papers citing "How Can We Know What Language Models Know?"

50 / 241 papers shown

Title
Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey Garima Agrawal Tharindu Kumarage Zeyad Alghami Huanmin Liu 37 81 0 14 Nov 2023
Evaluating General-Purpose AI with Psychometrics Xiting Wang Liming Jiang Jose Hernandez-Orallo David Stillwell Luning Sun Fang Luo Xing Xie AI4MH ELM 30 12 0 25 Oct 2023
Language Models Hallucinate, but May Excel at Fact Verification Jian-Yu Guan Jesse Dodge David Wadden Minlie Huang Hao Peng LRM HILM 26 28 0 23 Oct 2023
Tree Prompting: Efficient Task Adaptation without Fine-Tuning John X. Morris Chandan Singh Alexander M. Rush Jianfeng Gao Yuntian Deng VLM LRM 21 17 0 21 Oct 2023
Who Are All The Stochastic Parrots Imitating? They Should Tell Us! Sagi Shaier Lawrence E Hunter K. Wense 34 3 0 16 Oct 2023
Decomposed Prompt Tuning via Low-Rank Reparameterization Yao Xiao Lu Xu Jiaxi Li Wei Lu Xiaoli Li VLM 17 6 0 16 Oct 2023
In-Context Learning for Few-Shot Molecular Property Prediction Christopher Fifty J. Leskovec Sebastian Thrun 36 5 0 13 Oct 2023
Enhancing Text-based Knowledge Graph Completion with Zero-Shot Large Language Models: A Focus on Semantic Enhancement Rui Yang Jiahao Zhu Jianping Man Li Fang Yi Zhou 24 17 0 12 Oct 2023
Prompt-and-Align: Prompt-Based Social Alignment for Few-Shot Fake News Detection Jiaying Wu Xinyu Chen Haobin Yang Qi Zhao Yuhui Shi AAML 24 11 0 28 Sep 2023
Quantifying and Analyzing Entity-level Memorization in Large Language Models Zhenhong Zhou Jiuyang Xiang Chao-Yi Chen Sen Su PILM 38 8 0 30 Aug 2023
Mobile Foundation Model as Firmware Jinliang Yuan Chenchen Yang Dongqi Cai Shihe Wang Xin Yuan ... Di Zhang Hanzi Mei Xianqing Jia Shangguang Wang Mengwei Xu 40 19 0 28 Aug 2023
Semantic Consistency for Assuring Reliability of Large Language Models Harsh Raj Vipul Gupta Domenic Rosati S. Majumdar HILM 104 14 0 17 Aug 2023
A Knowledge-enhanced Two-stage Generative Framework for Medical Dialogue Information Extraction Zefa Hu Ziyi Ni Jing Shi Shuang Xu Bo Xu MedIm 35 1 0 30 Jul 2023
Pseudo Outlier Exposure for Out-of-Distribution Detection using Pretrained Transformers Jaeyoung Kim Kyuheon Jung Dongbin Na Sion Jang Eunbin Park Sungchul Choi OODD 24 6 0 18 Jul 2023
Training Models to Generate, Recognize, and Reframe Unhelpful Thoughts Mounica Maddela Megan Ung Jing Xu Andrea Madotto H. Foran Y-Lan Boureau LRM 28 21 0 06 Jul 2023
NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Models Kai Mei Zheng Li Zhenting Wang Yang Zhang Shiqing Ma AAML SILM 34 48 0 28 May 2023
Plug-and-Play Document Modules for Pre-trained Models Chaojun Xiao Zhengyan Zhang Xu Han Chi-Min Chan Yankai Lin Zhiyuan Liu Xiangyang Li Zhonghua Li Zhao Cao Maosong Sun KELM 22 5 0 28 May 2023
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models Yuhui Zhang Michihiro Yasunaga Zhengping Zhou Jeff Z. HaoChen James Y. Zou Percy Liang Serena Yeung 44 7 0 27 May 2023
Hierarchical Verbalizer for Few-Shot Hierarchical Text Classification Kexi Ji Yixin Lian Jingsheng Gao Baoyuan Wang VLM 19 21 0 26 May 2023
CTQScorer: Combining Multiple Features for In-context Example Selection for Machine Translation Aswanth Kumar Ratish Puduppully Raj Dabre Anoop Kunchukuttan 36 12 0 23 May 2023
Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models Alfonso Amayuelas Kyle Wong Liangming Pan Wenhu Chen W. Wang 37 26 0 23 May 2023
Natural Language Decompositions of Implicit Content Enable Better Text Representations Alexander Miserlis Hoyle Rupak Sarkar Pranav Goel Philip Resnik AI4CE 38 12 0 23 May 2023
Language-Agnostic Bias Detection in Language Models with Bias Probing Abdullatif Köksal Omer F. Yalcin Ahmet Akbiyik M. Kilavuz Anna Korhonen Hinrich Schütze 25 1 0 22 May 2023
Flatness-Aware Prompt Selection Improves Accuracy and Sample Efficiency Lingfeng Shen Weiting Tan Boyuan Zheng Daniel Khashabi VLM 39 6 0 18 May 2023
Statistical Knowledge Assessment for Large Language Models Qingxiu Dong Jingjing Xu Lingpeng Kong Zhifang Sui Lei Li HILM 36 6 0 17 May 2023
Active Retrieval Augmented Generation Zhengbao Jiang Frank F. Xu Luyu Gao Zhiqing Sun Qian Liu Jane Dwivedi-Yu Yiming Yang Jamie Callan Graham Neubig RALM 9 252 0 11 May 2023
Domain Mastery Benchmark: An Ever-Updating Benchmark for Evaluating Holistic Domain Knowledge of Large Language Model--A Preliminary Release Zhouhong Gu Xiaoxuan Zhu Haoning Ye Lin Zhang Zhuozhi Xiong Zihan Li Qi He Sihang Jiang Hongwei Feng Yanghua Xiao ELM ALM 47 2 0 23 Apr 2023
TransHP: Image Classification with Hierarchical Prompting Wenhao Wang Yifan Sun W. Li Yi Yang 19 16 0 13 Apr 2023
Defense-Prefix for Preventing Typographic Attacks on CLIP Hiroki Azuma Yusuke Matsui VLM AAML 20 17 0 10 Apr 2023
Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models Emilio Ferrara SILM 21 247 0 07 Apr 2023
Vita-CLIP: Video and text adaptive CLIP via Multimodal Prompting Syed Talal Wasim Muzammal Naseer Salman Khan F. Khan M. Shah VLM VPVLM 30 73 0 06 Apr 2023
PromptORE -- A Novel Approach Towards Fully Unsupervised Relation Extraction Pierre-Yves Genest Pierre-Edouard Portier Elöd Egyed-Zsigmond L. Goix LRM 22 12 0 24 Mar 2023
Visually-Prompted Language Model for Fine-Grained Scene Graph Generation in an Open World Qifan Yu Juncheng Li Yuehua Wu Siliang Tang Wei Ji Yueting Zhuang 30 34 0 23 Mar 2023
Exploring Structured Semantic Prior for Multi Label Recognition with Incomplete Labels Zixuan Ding Ao Wang Hui Chen Q. Zhang Pengzhang Liu Yongjun Bao Weipeng P. Yan Jungong Han 19 27 0 23 Mar 2023
Diversity-Aware Meta Visual Prompting Qidong Huang Xiaoyi Dong Dongdong Chen Weiming Zhang Feifei Wang Gang Hua Neng H. Yu VLM VPVLM 41 52 0 14 Mar 2023
Rethinking Visual Prompt Learning as Masked Visual Token Modeling Ning Liao Bowen Shi Xiaopeng Zhang Min Cao Junchi Yan Qi Tian VLM 34 7 0 09 Mar 2023
Class Cardinality Comparison as a Fermi Problem Shrestha Ghosh Simon Razniewski G. Weikum 42 2 0 08 Mar 2023
The Contribution of Knowledge in Visiolinguistic Learning: A Survey on Tasks and Challenges Maria Lymperaiou Giorgos Stamou VLM 29 4 0 04 Mar 2023
Dynamic Benchmarking of Masked Language Models on Temporal Concept Drift with Multiple Views Katerina Margatina Shuai Wang Yogarshi Vyas Neha Ann John Yassine Benajiba Miguel Ballesteros 15 15 0 23 Feb 2023
Active Prompting with Chain-of-Thought for Large Language Models Shizhe Diao Pengcheng Wang Yong Lin Tong Zhang ReLM KELM LLMAG LRM 29 119 0 23 Feb 2023
Can discrete information extraction prompts generalize across language models? Nathanaël Carraz Rakotonirina Roberto Dessì Fabio Petroni Sebastian Riedel Marco Baroni 25 7 0 20 Feb 2023
RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions Yunlong Wang Shuyuan Shen Brian Y. Lim 28 88 0 19 Feb 2023
LabelPrompt: Effective Prompt-based Learning for Relation Classification W. Zhang Xiaoning Song Zhenhua Feng Tianyang Xu Xiaojun Wu VLM 32 4 0 16 Feb 2023
Controlling for Stereotypes in Multimodal Language Model Evaluation Manuj Malik Richard Johansson 18 1 0 03 Feb 2023
The Exploration of Knowledge-Preserving Prompts for Document Summarisation Cheng Chen Wei Emma Zhang A. Shakeri Makhmoor Fiza KELM 27 0 0 27 Jan 2023
Probing Out-of-Distribution Robustness of Language Models with Parameter-Efficient Transfer Learning Hyunsoo Cho Choonghyun Park Junyeop Kim Hyuhng Joon Kim Kang Min Yoo Sang-goo Lee OODD 23 3 0 27 Jan 2023
Understanding Finetuning for Factual Knowledge Extraction from Language Models Mehran Kazemi Sid Mittal Deepak Ramachandran KELM 34 10 0 26 Jan 2023
Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models Zhiqiu Lin Samuel Yu Zhiyi Kuang Deepak Pathak Deva Ramana VLM 15 100 0 16 Jan 2023
Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering Zhiyong Wu Yaoxiang Wang Jiacheng Ye Lingpeng Kong 39 119 0 20 Dec 2022
DSI++: Updating Transformer Memory with New Documents Sanket Vaibhav Mehta Jai Gupta Yi Tay Mostafa Dehghani Vinh Q. Tran J. Rao Marc Najork Emma Strubell Donald Metzler CLL 30 39 0 19 Dec 2022