Deep ReLU Networks Have Surprisingly Few Activation Patterns

3 June 2019

Papers citing "Deep ReLU Networks Have Surprisingly Few Activation Patterns"

50 / 51 papers shown

Title
On Space Folds of ReLU Neural Networks Michal Lewandowski Hamid Eghbalzadeh Bernhard Heinzl Raphael Pisoni Bernhard A.Moser MLT 87 1 0 17 Feb 2025
A Relative Homology Theory of Representation in Neural Networks Kosio Beshkov 99 0 0 17 Feb 2025
Spectral complexity of deep neural networks Simmaco Di Lillo Domenico Marinucci Michele Salvi Stefano Vigogna BDL 82 1 0 15 May 2024
Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks Fanghui Liu L. Dadi V. Cevher 82 2 0 29 Apr 2024
Understanding the Feature Norm for Out-of-Distribution Detection Jaewoo Park Jacky Chen Long Chai Jaeho Yoon Andrew Beng Jin Teoh OODD 24 12 0 09 Oct 2023
Synthesizing Programmatic Policies with Actor-Critic Algorithms and ReLU Networks S. Orfanos Levi H. S. Lelis 19 6 0 04 Aug 2023
Understanding Activation Patterns in Artificial Neural Networks by Exploring Stochastic Processes S. Lehmler Muhammad Saif-ur-Rehman Tobias Glasmachers Ioannis Iossifidis 27 0 0 01 Aug 2023
PAC bounds of continuous Linear Parameter-Varying systems related to neural ODEs Dániel Rácz Mihaly Petreczky Bálint Daróczy 57 0 0 07 Jul 2023
Polyhedral Complex Extraction from ReLU Networks using Edge Subdivision Arturs Berzins 27 5 0 12 Jun 2023
Learning Prescriptive ReLU Networks Wei-Ju Sun Asterios Tsiourvas 21 2 0 01 Jun 2023
SkelEx and BoundEx: Natural Visualization of ReLU Neural Networks Pawel Pukowski Haiping Lu 28 0 0 09 May 2023
When Deep Learning Meets Polyhedral Theory: A Survey Joey Huchette Gonzalo Muñoz Thiago Serra Calvin Tsay AI4CE 94 32 0 29 Apr 2023
The Power of Typed Affine Decision Structures: A Case Study Gerrit Nolte Maximilian Schlüter Alnis Murtovi Bernhard Steffen AAML 20 3 0 28 Apr 2023
On the Lipschitz Constant of Deep Networks and Double Descent Matteo Gamba Hossein Azizpour Marten Bjorkman 31 7 0 28 Jan 2023
Neural networks learn to magnify areas near decision boundaries Jacob A. Zavatone-Veth Sheng Yang Julian Rubinfien Cengiz Pehlevan MLT AI4CE 25 6 0 26 Jan 2023
Towards Rigorous Understanding of Neural Networks via Semantics-preserving Transformations Maximilian Schlüter Gerrit Nolte Alnis Murtovi Bernhard Steffen 29 6 0 19 Jan 2023
Getting Away with More Network Pruning: From Sparsity to Geometry and Linear Regions Junyang Cai Khai-Nguyen Nguyen Nishant Shrestha Aidan Good Ruisen Tu Xin Yu Shandian Zhe Thiago Serra MLT 40 7 0 19 Jan 2023
Expected Gradients of Maxout Networks and Consequences to Parameter Initialization Hanna Tseran Guido Montúfar ODL 30 0 0 17 Jan 2023
Understanding the Spectral Bias of Coordinate Based MLPs Via Training Dynamics J. Lazzari Xiuwen Liu 26 3 0 14 Jan 2023
Effects of Data Geometry in Early Deep Learning Saket Tiwari George Konidaris 79 7 0 29 Dec 2022
Interpreting Neural Networks through the Polytope Lens Sid Black Lee D. Sharkey Léo Grinsztajn Eric Winsor Daniel A. Braun ... Kip Parker Carlos Ramón Guevara Beren Millidge Gabriel Alfour Connor Leahy FAtt MILM 31 22 0 22 Nov 2022
Towards Reliable Neural Specifications Chuqin Geng Nham Le Xiaojie Xu Zhaoyue Wang A. Gurfinkel X. Si AAML 36 10 0 28 Oct 2022
Improved Bounds on Neural Complexity for Representing Piecewise Linear Functions Kuan-Lin Chen H. Garudadri Bhaskar D. Rao 11 18 0 13 Oct 2022
Curved Representation Space of Vision Transformers Juyeop Kim Junha Park Songkuk Kim Jongseok Lee ViT 38 6 0 11 Oct 2022
Magnitude and Angle Dynamics in Training Single ReLU Neurons Sangmin Lee Byeongsu Sim Jong Chul Ye MLT 96 6 0 27 Sep 2022
Verifying Attention Robustness of Deep Neural Networks against Semantic Perturbations S. Munakata Caterina Urban Haruki Yokoyama Koji Yamamoto Kazuki Munakata AAML 17 4 0 13 Jul 2022
Exact Spectral Norm Regularization for Neural Networks Anton Johansson Claes Strannegård Niklas Engsner P. Mostad AAML 13 2 0 27 Jun 2022
On the Number of Regions of Piecewise Linear Neural Networks Alexis Goujon Arian Etemadi M. Unser 44 13 0 17 Jun 2022
Lower and Upper Bounds for Numbers of Linear Regions of Graph Convolutional Networks Hao Chen Yu Wang Huan Xiong GNN 16 6 0 01 Jun 2022
The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networks Xin Yu Thiago Serra Srikumar Ramalingam Shandian Zhe 42 48 0 09 Mar 2022
Neural Architecture Search for Spiking Neural Networks Youngeun Kim Yuhang Li Hyoungseob Park Yeshwanth Venkatesha Priyadarshini Panda 26 88 0 23 Jan 2022
A Simple and Efficient Sampling-based Algorithm for General Reachability Analysis T. Lew Lucas Janson Riccardo Bonalli Marco Pavone 24 18 0 10 Dec 2021
Lottery Tickets with Nonzero Biases Jonas Fischer Advait Gadhikar R. Burkholz 22 6 0 21 Oct 2021
Expressivity of Neural Networks via Chaotic Itineraries beyond Sharkovsky's Theorem Clayton Sanford Vaggos Chatziafratis 16 1 0 19 Oct 2021
Training Deep Neural Networks with Joint Quantization and Pruning of Weights and Activations Xinyu Zhang Ian Colbert Ken Kreutz-Delgado Srinjoy Das MQ 32 11 0 15 Oct 2021
An Embedding of ReLU Networks and an Analysis of their Identifiability Pierre Stock Rémi Gribonval 31 17 0 20 Jul 2021
Constrained Feedforward Neural Network Training via Reachability Analysis Long Kiu Chung Adam Dai Derek Knowles Shreyas Kousik Grace Gao 16 8 0 16 Jul 2021
Random Neural Networks in the Infinite Width Limit as Gaussian Processes Boris Hanin BDL 32 43 0 04 Jul 2021
Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sums Guido Montúfar Yue Ren Leon Zhang 13 39 0 16 Apr 2021
Deep ReLU Networks Preserve Expected Length Boris Hanin Ryan Jeong David Rolnick 26 14 0 21 Feb 2021
A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks Asaf Noy Yi Tian Xu Y. Aflalo Lihi Zelnik-Manor R. L. Jin 36 3 0 12 Jan 2021
Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks Quynh N. Nguyen Marco Mondelli Guido Montúfar 25 81 0 21 Dec 2020
Dissipative Deep Neural Dynamical Systems Ján Drgoňa Soumya Vasisht Aaron Tuor D. Vrabie 21 7 0 26 Nov 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 34 79 0 17 Sep 2020
Lossless Compression of Deep Neural Networks Thiago Serra Abhinav Kumar Srikumar Ramalingam 24 56 0 01 Jan 2020
Large Scale Model Predictive Control with Neural Networks and Primal Active Sets Steven W. Chen Tianyu Wang Nikolay Atanasov Vijay Kumar M. Morari 15 86 0 23 Oct 2019
Finite Depth and Width Corrections to the Neural Tangent Kernel Boris Hanin Mihai Nica MDE 24 150 0 13 Sep 2019
Optimal Function Approximation with Relu Neural Networks Bo Liu Yi Liang 25 33 0 09 Sep 2019
Greedy Shallow Networks: An Approach for Constructing and Training Neural Networks Anton Dereventsov Armenak Petrosyan Clayton Webster 15 9 0 24 May 2019
Deep Neural Network Approximation Theory Dennis Elbrächter Dmytro Perekrestenko Philipp Grohs Helmut Bölcskei 14 207 0 08 Jan 2019