Kformer: Knowledge Injection in Transformer Feed-Forward Layers

15 January 2022

Huajun Chen

Ningyu Zhang

KELM

MedIm

ArXiv PDF HTML

Papers citing "Kformer: Knowledge Injection in Transformer Feed-Forward Layers"

35 / 35 papers shown

Title
Question-guided Knowledge Graph Re-scoring and Injection for Knowledge Graph Question Answering Yu Zhang Kehai Chen Xuefeng Bai zhao kang Quanjiang Guo Min Zhang 15 5 0 02 Oct 2024
Knowledge Graph Tuning: Real-time Large Language Model Personalization based on Human Feedback Jingwei Sun Zhixu Du Yiran Chen KELM 24 2 0 30 May 2024
Erasing Concepts from Text-to-Image Diffusion Models with Few-shot Unlearning Masane Fuchi Tomohiro Takagi DiffM VLM 43 13 0 12 May 2024
Personalized LLM Response Generation with Parameterized Memory Injection Kai Zhang Lizhi Qing Yangyang Kang 23 11 0 04 Apr 2024
Collaborative Knowledge Infusion for Low-resource Stance Detection Ming Yan Joey Tianyi Zhou Ivor W. Tsang 14 2 0 28 Mar 2024
DREditor: An Time-efficient Approach for Building a Domain-specific Dense Retrieval Model Chen Huang Duanyu Feng Wenqiang Lei Jiancheng Lv 49 1 0 23 Jan 2024
A Comprehensive Study of Knowledge Editing for Large Language Models Ningyu Zhang Yunzhi Yao Bo Tian Peng Wang Shumin Deng ... Lei Liang Zhiqiang Zhang Xiao-Jun Zhu Jun Zhou Huajun Chen KELM 26 76 0 02 Jan 2024
Unveiling Multilinguality in Transformer Models: Exploring Language Specificity in Feed-Forward Networks Sunit Bhattacharya Ondrej Bojar 19 7 0 24 Oct 2023
Improving Factual Consistency for Knowledge-Grounded Dialogue Systems via Knowledge Enhancement and Alignment Boyang Xue Weichao Wang Hongru Wang Fei Mi Rui Wang Yasheng Wang Lifeng Shang Xin Jiang Qun Liu Kam-Fai Wong KELM HILM 211 15 0 12 Oct 2023
Multi-Modal Knowledge Graph Transformer Framework for Multi-Modal Entity Alignment Qian Li Cheng Ji Shu Guo Zhaoji Liang Lihong Wang Jianxin Li 15 8 0 10 Oct 2023
The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" Lukas Berglund Meg Tong Max Kaufmann Mikita Balesni Asa Cooper Stickland Tomasz Korbak Owain Evans LRM 20 233 0 21 Sep 2023
On the Relationship between Skill Neurons and Robustness in Prompt Tuning Leon Ackermann Xenia Ohmer AAML 11 0 0 21 Sep 2023
"Merge Conflicts!" Exploring the Impacts of External Distractors to Parametric Knowledge Graphs Cheng Qian Xinran Zhao Sherry Tongshuang Wu KELM 14 8 0 15 Sep 2023
Taken out of context: On measuring situational awareness in LLMs Lukas Berglund Asa Cooper Stickland Mikita Balesni Max Kaufmann Meg Tong Tomasz Korbak Daniel Kokotajlo Owain Evans LLMAG LRM 8 60 0 01 Sep 2023
Evaluating the Ripple Effects of Knowledge Editing in Language Models Roi Cohen Eden Biran Ori Yoran Amir Globerson Mor Geva KELM 33 155 0 24 Jul 2023
Shuo Wen Jie Zi: Rethinking Dictionaries and Glyphs for Chinese Language Pre-training Yuxuan Wang Jianghui Wang Dongyan Zhao Zilong Zheng 15 4 0 30 May 2023
Editing Common Sense in Transformers Anshita Gupta Debanjan Mondal Akshay Krishna Sheshadri Wenlong Zhao Xiang Lorraine Li Sarah Wiegreffe Niket Tandon KELM 26 20 0 24 May 2023
Editing Large Language Models: Problems, Methods, and Opportunities Yunzhi Yao Peng Wang Bo Tian Shuyang Cheng Zhoubo Li Shumin Deng Huajun Chen Ningyu Zhang KELM 30 275 0 22 May 2023
Decouple knowledge from parameters for plug-and-play language modeling Xin Cheng Yankai Lin Xiuying Chen Dongyan Zhao Rui Yan KELM 14 2 0 19 May 2023
Knowledge Rumination for Pre-trained Language Models Yunzhi Yao Peng Wang Shengyu Mao Chuanqi Tan Fei Huang Huajun Chen Ningyu Zhang KELM 14 3 0 15 May 2023
RECKONING: Reasoning through Dynamic Knowledge Encoding Zeming Chen Gail Weiss E. Mitchell Asli Celikyilmaz Antoine Bosselut KELM LRM 14 11 0 10 May 2023
A Study on ReLU and Softmax in Transformer Kai Shen Junliang Guo Xuejiao Tan Siliang Tang Rui Wang Jiang Bian 11 51 0 13 Feb 2023
A Survey of Knowledge Enhanced Pre-trained Language Models Linmei Hu Zeyi Liu Ziwang Zhao Lei Hou Liqiang Nie Juanzi Li KELM VLM 17 121 0 11 Nov 2022
An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks Yuxiang Wu Yu Zhao Baotian Hu Pasquale Minervini Pontus Stenetorp Sebastian Riedel RALM KELM 43 42 0 30 Oct 2022
Parameter-Efficient Tuning Makes a Good Classification Head Zhuoyi Yang Ming Ding Yanhui Guo Qingsong Lv Jie Tang VLM 27 13 0 30 Oct 2022
Mass-Editing Memory in a Transformer Kevin Meng Arnab Sen Sharma A. Andonian Yonatan Belinkov David Bau KELM VLM 25 518 0 13 Oct 2022
Calibrating Factual Knowledge in Pretrained Language Models Qingxiu Dong Damai Dai Yifan Song Jingjing Xu Zhifang Sui Lei Li KELM 225 81 0 07 Oct 2022
Neural Knowledge Bank for Pretrained Transformers Damai Dai Wen-Jie Jiang Qingxiu Dong Yajuan Lyu Qiaoqiao She Zhifang Sui KELM 8 21 0 31 Jul 2022
FL-Tuning: Layer Tuning for Feed-Forward Network in Transformer Jingping Liu Yuqiu Song Kui Xue Hongli Sun Chao Wang Lihan Chen Haiyun Jiang Jiaqing Liang Tong Ruan 36 1 0 30 Jun 2022
Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space Mor Geva Avi Caciularu Ke Wang Yoav Goldberg KELM 11 332 0 28 Mar 2022
OntoProtein: Protein Pretraining With Gene Ontology Embedding Ningyu Zhang Zhen Bi Xiaozhuan Liang Shuyang Cheng Haosen Hong Shumin Deng J. Lian Qiang Zhang Huajun Chen 91 86 0 23 Jan 2022
Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention Yichong Xu Chenguang Zhu Shuohang Wang Siqi Sun Hao Cheng Xiaodong Liu Jianfeng Gao Pengcheng He Michael Zeng Xuedong Huang LRM 246 54 0 06 Dec 2021
K-BERT: Enabling Language Representation with Knowledge Graph Weijie Liu Peng Zhou Zhe Zhao Zhiruo Wang Qi Ju Haotang Deng Ping Wang 220 773 0 17 Sep 2019
Knowledge Enhanced Contextual Word Representations Matthew E. Peters Mark Neumann IV RobertL.Logan Roy Schwartz Vidur Joshi Sameer Singh Noah A. Smith 215 651 0 09 Sep 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 396 2,576 0 03 Sep 2019