Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation

15 August 2013

Aaron Courville

Papers citing "Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation"

50 / 1,869 papers shown

Title
BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocation Peng Xu Wenqi Shao Yonghong Tian Shitao Tang Kai-Chuang Zhang Peng Gao Fengwei An Yu Qiao Ping Luo MoE 35 27 0 18 Feb 2024
Turn Waste into Worth: Rectifying Top- $k$ Router of MoE Zhiyuan Zeng Qipeng Guo Zhaoye Fei Zhangyue Yin Yunhua Zhou Linyang Li Tianxiang Sun Hang Yan Dahua Lin Xipeng Qiu MoE MoMe 33 4 0 17 Feb 2024
EdgeQAT: Entropy and Distribution Guided Quantization-Aware Training for the Acceleration of Lightweight LLMs on the Edge Xuan Shen Zhenglun Kong Changdi Yang Zhaoyang Han Lei Lu ... Zhihao Shu Wei Niu Miriam Leeser Pu Zhao Yanzhi Wang MQ 51 18 0 16 Feb 2024
BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation Dayou Du Yijia Zhang Shijie Cao Jiaqi Guo Ting Cao Xiaowen Chu Ningyi Xu MQ 46 30 0 16 Feb 2024
Symbolic Autoencoding for Self-Supervised Sequence Learning Mohammad Hossein Amani Nicolas Mario Baldwin Amin Mansouri Martin Josifoski Maxime Peyrard Robert West 26 1 0 16 Feb 2024
Conditional Information Gain Trellis Ufuk Can Biçici Tuna Han Salih Meral L. Akarun 34 2 0 13 Feb 2024
L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models Hyesung Jeon Yulhwa Kim Jae-Joon Kim MQ 29 4 0 07 Feb 2024
ApiQ: Finetuning of 2-Bit Quantized Large Language Model Baohao Liao Christian Herold Shahram Khadivi Christof Monz CLL MQ 47 12 0 07 Feb 2024
BiLLM: Pushing the Limit of Post-Training Quantization for LLMs Wei Huang Yangdong Liu Haotong Qin Ying Li Shiming Zhang Xianglong Liu Michele Magno Xiaojuan Qi MQ 82 69 0 06 Feb 2024
See More Details: Efficient Image Super-Resolution by Experts Mining Eduard Zamfir Zongwei Wu Nancy Mehta Yulun Zhang Radu Timofte SupR 48 10 0 05 Feb 2024
InterpretCC: Intrinsic User-Centric Interpretability through Global Mixture of Experts Vinitra Swamy Syrielle Montariol Julian Blackwell Jibril Frej Martin Jaggi Tanja Kaser 46 3 0 05 Feb 2024
Quantized Approximately Orthogonal Recurrent Neural Networks Armand Foucault Franck Mamalet Franccois Malgouyres MQ 34 1 0 05 Feb 2024
Variational DAG Estimation via State Augmentation With Stochastic Permutations Edwin V. Bonilla P. Elinas He Zhao Maurizio Filippone V. Kitsios Terry O'Kane CML 45 3 0 04 Feb 2024
Sample Complexity of Algorithm Selection Using Neural Networks and Its Applications to Branch-and-Cut Hongyu Cheng Sammy Khalife Barbara Fiedorowicz Amitabh Basu 9 1 0 04 Feb 2024
MixedNUTS: Training-Free Accuracy-Robustness Balance via Nonlinearly Mixed Classifiers Yatong Bai Mo Zhou Vishal M. Patel Somayeh Sojoudi AAML 29 6 0 03 Feb 2024
Spiking Music: Audio Compression with Event Based Auto-encoders Martim Lisboa Guillaume Bellec 40 2 0 02 Feb 2024
A Differentiable Partially Observable Generalized Linear Model with Forward-Backward Message Passing Chengrui Li Weihan Li Yule Wang Anqi Wu 31 1 0 02 Feb 2024
Neural Language of Thought Models Yi-Fu Wu Minseung Lee Sungjin Ahn MLLM VLM 80 6 0 02 Feb 2024
Lightweight Pixel Difference Networks for Efficient Visual Representation Learning Z. Su Jiehua Zhang Longguang Wang Hua Zhang Zhen Liu M. Pietikäinen Li Liu 38 20 0 01 Feb 2024
Robustly overfitting latents for flexible neural image compression Yura Perugachi-Diaz Arwin Gansekoele Sandjai Bhulai 46 1 0 31 Jan 2024
Trainable Fixed-Point Quantization for Deep Learning Acceleration on FPGAs Dingyi Dai Yichi Zhang Jiahao Zhang Zhanqiu Hu Yaohui Cai Qi Sun Zhiru Zhang MQ 69 5 0 31 Jan 2024
Forecasting VIX using Bayesian Deep Learning Héctor J. Hortúa Andrés Mora-Valencia BDL OOD 28 4 0 30 Jan 2024
X-PEFT: eXtremely Parameter-Efficient Fine-Tuning for Extreme Multi-Profile Scenarios Namju Kwak Taesup Kim MoE 29 0 0 29 Jan 2024
A Comprehensive Survey of Compression Algorithms for Language Models Seungcheol Park Jaehyeon Choi Sojin Lee U. Kang MQ 32 12 0 27 Jan 2024
Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators Yaniv Blumenfeld Itay Hubara Daniel Soudry 47 3 0 25 Jan 2024
Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models T. Golling Lukas Heinrich Michael Kagan Samuel Klein Matthew Leigh Margarita Osadchy J. A. Raine 36 24 0 24 Jan 2024
Text-to-Image Cross-Modal Generation: A Systematic Review Maciej Żelaszczyk Jacek Mańdziuk 35 3 0 21 Jan 2024
Neglected Hessian component explains mysteries in Sharpness regularization Yann N. Dauphin Atish Agarwala Hossein Mobahi FAtt 46 7 0 19 Jan 2024
A2Q+: Improving Accumulator-Aware Weight Quantization Ian Colbert Alessandro Pappalardo Jakoba Petri-Koenig Yaman Umuroglu MQ 29 4 0 19 Jan 2024
Adaptive Self-training Framework for Fine-grained Scene Graph Generation Kibum Kim Kanghoon Yoon Yeonjun In Jinyoung Moon Donghyun Kim Chanyoung Park 41 8 0 18 Jan 2024
Optimization of Discrete Parameters Using the Adaptive Gradient Method and Directed Evolution Andrei Beinarovich Sergey Stepanov Alexander Zaslavsky 46 0 0 12 Jan 2024
A foundation for exact binarized morphological neural networks T. Aouad Hugues Talbot 16 1 0 08 Jan 2024
Training a General Spiking Neural Network with Improved Efficiency and Minimum Latency Yunpeng Yao Man Wu Zheng Chen Renyuan Zhang 35 0 0 05 Jan 2024
Retraining-free Model Quantization via One-Shot Weight-Coupling Learning Chen Tang Yuan Meng Jiacheng Jiang Shuzhao Xie Rongwei Lu Xinzhu Ma Zhi Wang Wenwu Zhu MQ 24 8 0 03 Jan 2024
Compact Neural Graphics Primitives with Learned Hash Probing Towaki Takikawa Thomas Müller Merlin Nimier-David Alex Evans Sanja Fidler Alec Jacobson Alexander Keller 27 18 0 28 Dec 2023
City-on-Web: Real-time Neural Rendering of Large-scale Scenes on the Web Kaiwen Song Xiaoyi Zeng Chenqu Ren Juyong Zhang AI4CE 43 10 0 27 Dec 2023
Compressing Image-to-Image Translation GANs Using Local Density Structures on Their Learned Manifold Alireza Ganjdanesh Shangqian Gao Hirad Alipanah Heng-Chiao Huang GAN 32 6 0 22 Dec 2023
ARBiBench: Benchmarking Adversarial Robustness of Binarized Neural Networks Peng Zhao Jiehua Zhang Bowen Peng Longguang Wang Yingmei Wei Yu Liu Li Liu AAML 32 0 0 21 Dec 2023
DSFormer: Effective Compression of Text-Transformers by Dense-Sparse Weight Factorization Rahul Chand Yashoteja Prabhu Pratyush Kumar 20 3 0 20 Dec 2023
Sign Language Production with Latent Motion Transformer Pan Xie Taiying Peng Yao Du Qipeng Zhang SLR 27 3 0 20 Dec 2023
SCoTTi: Save Computation at Training Time with an adaptive framework Ziyu Li Enzo Tartaglione Van-Tam Nguyen 42 0 0 19 Dec 2023
Continual Learning: Forget-free Winning Subnetworks for Video Representations Haeyong Kang Jaehong Yoon Sung Ju Hwang Chang D. Yoo CLL 39 2 0 19 Dec 2023
Multiple Hypothesis Dropout: Estimating the Parameters of Multi-Modal Output Distributions David D. Nguyen David Liebowitz Surya Nepal S. Kanhere OOD UQCV 45 0 0 18 Dec 2023
Learning to Act without Actions Dominik Schmidt Minqi Jiang OffRL 34 31 0 17 Dec 2023
Learning Interpretable Queries for Explainable Image Classification with Information Pursuit Stefan Kolek Aditya Chattopadhyay Kwan Ho Ryan Chan Héctor Andrade-Loarca Gitta Kutyniok René Vidal 29 2 0 16 Dec 2023
Adaptive Computation Modules: Granular Conditional Computation For Efficient Inference Bartosz Wójcik Alessio Devoto Karol Pustelnik Pasquale Minervini Simone Scardapane 23 5 0 15 Dec 2023
One Self-Configurable Model to Solve Many Abstract Visual Reasoning Problems Mikolaj Malkiñski Jacek Mańdziuk 32 4 0 15 Dec 2023
End-to-End Training of Neural Networks for Automotive Radar Interference Mitigation Christian Oswald Máté Tóth Paul Meissner Franz Pernkopf AAML 26 3 0 15 Dec 2023
USM-Lite: Quantization and Sparsity Aware Fine-tuning for Speech Recognition with Universal Speech Models Shaojin Ding David Qiu David Rim Yanzhang He Oleg Rybakov ... Tara N. Sainath Zhonglin Han Jian Li Amir Yazdanbakhsh Shivani Agrawal MQ 34 9 0 13 Dec 2023
Modality Plug-and-Play: Elastic Modality Adaptation in Multimodal LLMs for Embodied AI Kai Huang Boyuan Yang Wei Gao 37 1 0 13 Dec 2023