A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA
Through Sparse Attention and Dynamic Pipelining

A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining

7 August 2022

Hongwu Peng

Caiwen Ding

Papers citing "A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining"

6 / 6 papers shown

Title
Accel-GCN: High-Performance GPU Accelerator Design for Graph Convolution Networks Xiaoru Xie Hongwu Peng Amit Hasan Shaoyi Huang Jiahui Zhao Haowen Fang Wei Zhang Tong Geng O. Khan Caiwen Ding GNN 30 30 0 22 Aug 2023
AutoReP: Automatic ReLU Replacement for Fast Private Network Inference Hongwu Peng Shaoyi Huang Tong Zhou Yukui Luo Chenghong Wang ... Tony Geng Kaleel Mahmood Wujie Wen Xiaolin Xu Caiwen Ding OffRL 32 38 0 20 Aug 2023
Dynamic Sparse Training via Balancing the Exploration-Exploitation Trade-off Shaoyi Huang Bowen Lei Dongkuan Xu Hongwu Peng Yue Sun Mimi Xie Caiwen Ding 15 19 0 30 Nov 2022
PolyMPCNet: Towards ReLU-free Neural Architecture Search in Two-party Computation Based Private Inference Hongwu Peng Shangli Zhou Yukui Luo Shijin Duan Nuo Xu ... Tong Geng Ang Li Wujie Wen Xiaolin Xu Caiwen Ding 21 3 0 20 Sep 2022
Towards Sparsification of Graph Neural Networks Hongwu Peng Deniz Gurevin Shaoyi Huang Tong Geng Weiwen Jiang O. Khan Caiwen Ding GNN 30 24 0 11 Sep 2022
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 254 2,013 0 28 Jul 2020