Structured Transforms for Small-Footprint Deep Learning

6 October 2015

Sanjiv Kumar

Papers citing "Structured Transforms for Small-Footprint Deep Learning"

45 / 45 papers shown

Title
Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models Xubin Wang Zhiqing Tang Jianxiong Guo Tianhui Meng Chenhao Wang Tian-sheng Wang Weijia Jia 57 1 0 08 Mar 2025
Symmetry-Based Structured Matrices for Efficient Approximately Equivariant Networks Ashwin Samudre Mircea Petrache Brian D. Nord Shubhendu Trivedi 50 2 0 18 Sep 2024
Simple Hardware-Efficient Long Convolutions for Sequence Modeling Daniel Y. Fu Elliot L. Epstein Eric N. D. Nguyen A. Thomas Michael Zhang Tri Dao Atri Rudra Christopher Ré 16 52 0 13 Feb 2023
Arithmetic Circuits, Structured Matrices and (not so) Deep Learning Atri Rudra 13 1 0 24 Jun 2022
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness Tri Dao Daniel Y. Fu Stefano Ermon Atri Rudra Christopher Ré VLM 87 2,045 0 27 May 2022
Monarch: Expressive Structured Matrices for Efficient and Accurate Training Tri Dao Beidi Chen N. Sohoni Arjun D Desai Michael Poli Jessica Grogan Alexander Liu Aniruddh Rao Atri Rudra Christopher Ré 24 87 0 01 Apr 2022
Mixed Precision Low-bit Quantization of Neural Network Language Models for Speech Recognition Junhao Xu Jianwei Yu Shoukang Hu Xunying Liu Helen Meng MQ 27 13 0 29 Nov 2021
CHIP: CHannel Independence-based Pruning for Compact Neural Networks Yang Sui Miao Yin Yi Xie Huy Phan S. Zonouz Bo Yuan VLM 33 129 0 26 Oct 2021
ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression Weiyue Su Xuyi Chen Shi Feng Jiaxiang Liu Weixin Liu Yu Sun Hao Tian Hua Wu Haifeng Wang 34 13 0 04 Jun 2021
FNet: Mixing Tokens with Fourier Transforms James Lee-Thorp Joshua Ainslie Ilya Eckstein Santiago Ontanon 47 517 0 09 May 2021
GST: Group-Sparse Training for Accelerating Deep Reinforcement Learning Juhyoung Lee Sangyeob Kim Sangjin Kim Wooyoung Jo H. Yoo OffRL 21 9 0 24 Jan 2021
Sparse Linear Networks with a Fixed Butterfly Structure: Theory and Practice Nir Ailon Omer Leibovitch Vineet Nair 15 14 0 17 Jul 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 19 2,843 0 09 Jun 2020
A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects Zewen Li Wenjie Yang Shouheng Peng Fan Liu HAI 3DV 54 2,600 0 01 Apr 2020
Communication-Efficient Edge AI: Algorithms and Systems Yuanming Shi Kai Yang Tao Jiang Jun Zhang Khaled B. Letaief GNN 23 326 0 22 Feb 2020
Depthwise Non-local Module for Fast Salient Object Detection Using a Single Thread Haofeng Li Guanbin Li Binbin Yang Guanqi Chen Liang Lin Yizhou Yu ObjD 46 28 0 22 Jan 2020
Discrimination-aware Network Pruning for Deep Model Compression Jing Liu Bohan Zhuang Zhuangwei Zhuang Yong Guo Junzhou Huang Jin-Hui Zhu Mingkui Tan CVBM 19 119 0 04 Jan 2020
Iteratively Training Look-Up Tables for Network Quantization Fabien Cardinaux Stefan Uhlich K. Yoshiyama Javier Alonso García Lukas Mauch Stephen Tiedemann Thomas Kemp Akira Nakamura MQ 27 16 0 12 Nov 2019
Survey on Deep Neural Networks in Speech and Vision Systems M. Alam Manar D. Samad Lasitha Vidyaratne Alexander M. Glandon Khan M. Iftekharuddin 3DV VLM AI4TS 34 205 0 16 Aug 2019
Compressing RNNs for IoT devices by 15-38x using Kronecker Products Urmish Thakker Jesse G. Beu Dibakar Gope Chu Zhou Igor Fedorov Ganesh S. Dasika Matthew Mattina 24 36 0 07 Jun 2019
Butterfly Transform: An Efficient FFT Based Neural Architecture Design Keivan Alizadeh-Vahid Anish K. Prabhu Ali Farhadi Mohammad Rastegari 32 50 0 05 Jun 2019
CircConv: A Structured Convolution with Low Complexity Siyu Liao Zhe Li Liang Zhao Qinru Qiu Yanzhi Wang Bo Yuan 19 18 0 28 Feb 2019
Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization Hesham Mostafa Xin Wang 37 307 0 15 Feb 2019
Understanding and Training Deep Diagonal Circulant Neural Networks Alexandre Araujo Benjamin Négrevergne Y. Chevaleyre Jamal Atif 27 4 0 29 Jan 2019
Pre-Defined Sparse Neural Networks with Hardware Acceleration Sourya Dey Kuan-Wen Huang P. Beerel K. Chugg 41 24 0 04 Dec 2018
Building Efficient Deep Neural Networks with Unitary Group Convolutions Ritchie Zhao Yuwei Hu Jordan Dotzel Christopher De Sa Zhiru Zhang 21 28 0 19 Nov 2018
Learning Compressed Transforms with Low Displacement Rank Anna T. Thomas Albert Gu Tri Dao Atri Rudra Christopher Ré 27 40 0 04 Oct 2018
Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech Recognition Chun-Fu Chen Quanfu Fan Neil Rohit Mallinar Tom Sercu Rogerio Feris 20 96 0 10 Jul 2018
Resource-Efficient Neural Architect Yanqi Zhou S. Ebrahimi Sercan Ö. Arik Haonan Yu Hairong Liu G. Diamos 22 64 0 12 Jun 2018
Tensorial Neural Networks: Generalization of Neural Networks and Application to Model Compression Jiahao Su Jingling Li Bobby Bhattacharjee Furong Huang 16 20 0 25 May 2018
Data-Dependent Coresets for Compressing Neural Networks with Applications to Generalization Bounds Cenk Baykal Lucas Liebenwein Igor Gilitschenski Dan Feldman Daniela Rus 25 79 0 15 Apr 2018
FFT-Based Deep Learning Deployment in Embedded Systems Sheng Lin Ning Liu M. Nazemi Hongjia Li Caiwen Ding Yanzhi Wang Massoud Pedram 41 52 0 13 Dec 2017
Trace norm regularization and faster inference for embedded speech recognition RNNs Markus Kliegl Siddharth Goyal Kexin Zhao Kavya Srinet M. Shoeybi 35 8 0 25 Oct 2017
Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting Sercan Ö. Arik Markus Kliegl R. Child Joel Hestness Andrew Gibiansky Christopher Fougner R. Prenger Adam Coates 21 180 0 15 Mar 2017
Theoretical Properties for Neural Networks with Weight Matrices of Low Displacement Rank Liang Zhao Siyu Liao Yanzhi Wang Zhe Li Jian Tang Victor Pan Bo Yuan 31 61 0 01 Mar 2017
Fully-adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification Y. Lu Abhishek Kumar Shuangfei Zhai Yu Cheng T. Javidi Rogerio Feris 3DH 21 384 0 16 Nov 2016
Small-footprint Highway Deep Neural Networks for Speech Recognition Liang Lu Steve Renals 30 15 0 18 Oct 2016
Knowledge Distillation for Small-footprint Highway Networks Liang Lu Michelle Guo Steve Renals 16 73 0 02 Aug 2016
Structured Convolution Matrices for Energy-efficient Deep learning R. Appuswamy T. Nayak John V. Arthur S. K. Esser P. Merolla J. McKinstry T. Melano M. Flickner D. Modha 33 11 0 08 Jun 2016
TripleSpin - a generic compact paradigm for fast machine learning computations K. Choromanski Francois Fagan Cédric Gouy-Pailler Anne Morvan Vikas Sindhwani Jamal Atif 29 7 0 29 May 2016
Composition of Deep and Spiking Neural Networks for Very Low Bit Rate Speech Coding Milos Cernak Alexandros Lazaridis Afsaneh Asaei Philip N. Garner 11 29 0 15 Apr 2016
Learning Compact Recurrent Neural Networks Zhiyun Lu Vikas Sindhwani Tara N. Sainath 12 86 0 09 Apr 2016
Personalized Speech recognition on mobile devices Ian McGraw Rohit Prabhavalkar R. Álvarez Montse Gonzalez Arenas Kanishka Rao ... O. Alsharif Hasim Sak A. Gruenstein F. Beaufays Carolina Parada 15 183 0 10 Mar 2016
Scalable and Sustainable Deep Learning via Randomized Hashing Ryan Spring Anshumali Shrivastava 23 132 0 26 Feb 2016
ACDC: A Structured Efficient Linear Layer Marcin Moczulski Misha Denil J. Appleyard Nando de Freitas 30 98 0 18 Nov 2015