SpikeGPT: Generative Pre-trained Language Model with Spiking Neural
Networks

SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks

27 February 2023

Rui-Jie Zhu

Jason Eshraghian

Papers citing "SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks"

15 / 15 papers shown

Title
SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking Xingrun Xing Boyan Gao Zheng Zhang David A. Clifton Shitao Xiao LI DU Guoqi Li Jiajun Zhang 45 5 0 05 Jul 2024
SQUAT: Stateful Quantization-Aware Training in Recurrent Spiking Neural Networks Sreyes P. Venkatesh Razvan Marinescu Jason Eshraghian MQ 25 5 0 15 Apr 2024
Neuromorphic Intermediate Representation: A Unified Instruction Set for Interoperable Brain-Inspired Computing Jens Egholm Pedersen Steven Abreu Matthias Jobst Gregor Lenz Vittorio Fra ... Gianvito Urgese Sadasivan Shankar Terrence C. Stewart Jason Eshraghian Sadique Sheik 25 30 0 24 Nov 2023
DISTA: Denoising Spiking Transformer with intrinsic plasticity and spatiotemporal attention Boxun Xu Hejia Geng Yuxuan Yin Peng Li 6 2 0 15 Nov 2023
Spike-driven Transformer Man Yao Jiakui Hu Zhaokun Zhou Liuliang Yuan Yonghong Tian Boxing Xu Guoqi Li 21 109 0 04 Jul 2023
A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation Xiaowei Huang Wenjie Ruan Wei Huang Gao Jin Yizhen Dong ... Sihao Wu Peipei Xu Dengyu Wu André Freitas Mustafa A. Mustafa ALM 27 81 0 19 May 2023
The MiniPile Challenge for Data-Efficient Language Models Jean Kaddour MoE ALM 8 41 0 17 Apr 2023
Online Transformers with Spiking Neurons for Fast Prosthetic Hand Control Nathan Leroux Jan Finkbeiner Emre Neftci 15 9 0 21 Mar 2023
Spikformer: When Spiking Neural Network Meets Transformer Zhaokun Zhou Yuesheng Zhu Chao He Yaowei Wang Shuicheng Yan Yonghong Tian Liuliang Yuan 140 231 0 29 Sep 2022
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 240 453 0 24 Sep 2022
The fine line between dead neurons and sparsity in binarized spiking neural networks Jason Eshraghian Wei D. Lu 20 19 0 28 Jan 2022
Deep Residual Learning in Spiking Neural Networks Wei Fang Zhaofei Yu Yanqing Chen Tiejun Huang T. Masquelier Yonghong Tian 121 470 0 08 Feb 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 242 1,977 0 31 Dec 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 3,054 0 23 Jan 2020
Convolutional Neural Networks for Sentence Classification Yoon Kim AILaw VLM 238 13,283 0 25 Aug 2014