Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark

29 September 2021

Papers citing "Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark"

18 / 18 papers shown

Title
Advancing Constrained Monotonic Neural Networks: Achieving Universal Approximation Beyond Bounded Activations Davide Sartor Alberto Sinigaglia Gian Antonio Susto 24 0 0 05 May 2025
Grounding Text-to-Image Diffusion Models for Controlled High-Quality Image Generation Ahmad Süleyman Göksel Biricik 41 2 0 15 Jan 2025
Cauchy activation function and XNet Xin Li Zhihong Xia Hongkun Zhang 26 4 0 28 Sep 2024
Kolmogorov-Arnold Networks in Low-Data Regimes: A Comparative Study with Multilayer Perceptrons Farhad Pourkamali-Anaraki 28 5 0 16 Sep 2024
Controlled Learning of Pointwise Nonlinearities in Neural-Network-Like Architectures Michael Unser Alexis Goujon Stanislas Ducotterd 24 2 0 23 Aug 2024
A Survey on Failure Analysis and Fault Injection in AI Systems Guangba Yu Gou Tan Haojia Huang Zhenyu Zhang Pengfei Chen Roberto Natella Zibin Zheng 34 3 0 28 Jun 2024
BrowNNe: Brownian Nonlocal Neurons & Activation Functions Sriram Nagaraj Truman Hickok 21 0 0 21 Jun 2024
M3BAT: Unsupervised Domain Adaptation for Multimodal Mobile Sensing with Multi-Branch Adversarial Training L. Meegahapola Hamza Hassoune D. Gática-Pérez 32 8 0 26 Apr 2024
Neural Redshift: Random Networks are not Random Functions Damien Teney A. Nicolicioiu Valentin Hartmann Ehsan Abbasnejad 89 18 0 04 Mar 2024
Speech Swin-Transformer: Exploring a Hierarchical Transformer with Shifted Windows for Speech Emotion Recognition Yong Wang Cheng Lu Hailun Lian Yan Zhao Bjorn Schuller Yuan Zong Wenming Zheng 13 10 0 19 Jan 2024
An extended asymmetric sigmoid with Perceptron (SIGTRON) for imbalanced linear classification Hyenkyun Woo 13 0 0 26 Dec 2023
From Alexnet to Transformers: Measuring the Non-linearity of Deep Neural Networks with Affine Optimal Transport Quentin Bouniot I. Redko Anton Mallasto Charlotte Laclau Karol Arndt Oliver Struckmeier Markus Heinonen Ville Kyrki Samuel Kaski 52 2 0 17 Oct 2023
HRTF upsampling with a generative adversarial network using a gnomonic equiangular projection Aidan O. T. Hogg Mads Jenkins Helena E. Liu Isaac Squires Samuel J. Cooper L. Picinali 15 13 0 09 Jun 2023
Review and Comparison of Commonly Used Activation Functions for Deep Neural Networks Tomasz Szandała 54 269 0 15 Oct 2020
Dynamic ReLU Yinpeng Chen Xiyang Dai Mengchen Liu Dongdong Chen Lu Yuan Zicheng Liu 161 157 0 22 Mar 2020
Learning Neural Activations F. Minhas Amina Asif 11 2 0 27 Dec 2019
diffGrad: An Optimization Method for Convolutional Neural Networks S. Dubey Soumendu Chakraborty S. K. Roy Snehasis Mukherjee S. Singh B. B. Chaudhuri ODL 59 181 0 12 Sep 2019
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 948 20,214 0 17 Apr 2017